AWS Senior Data Engineer

2 days ago

Solapur, Maharashtra, India InfoCepts Full time

Key Result Areas and Activities:

  • Study existing technology landscape and understand current data integration framework and do impact assessment for the requirements.
  • Develop spark jobs using Scala for new project requirements.
  • Enhance existing spark jobs for any ongoing product enhancement.
  • Performance tuning of spark jobs, stress testing etc.
  • Create new data pipelines for developed/enhanced spark jobs using AWS Lambda or Apache Airflow
  • Responsible for database design process, logical design, physical design, star schema, snowflake schema etc.
  • Analyze data processing, integration, modelling and reporting requirements & Define data loading strategies considering volume, data types, frequency and analytics specifications.
  • Ensure optimal balance between cost and performance.
  • Project documentation, Adheres to Quality guidelines & Schedules.
  • Works hand in hand with PM for successful delivery of project and provide Estimation, scoping, scheduling assistance.
  • Manage build phase and quality assure code to ensure fulfilling requirements and adhering to Cloud architecture. Resolve difficult design and develop issues.

Work and Technical Experience:

Must-Have:

  • Overall, 7-9 years of IT Exp
  • 5+ years of AWS related project
  • Good to have Associate Level and Professional Level AWS Certification
  • In depth knowledge of following AWS Services required: S3, EC2, EMR, Severless, Athena, AWS Glue, Lambda, Step Functions
  • Cloud Databases (Must have) – AWS Aurora, Singlestore, RedShift, Snowflake
  • Big Data (Must have) - Hadoop, Hive, Spark, YARN
  • Programming Language (Must have) – Scala, Python, Shell Scripts, PySpark
  • Operating System (Must have) - Any flavor of Linux, Windows
  • Must have very strong SQL Skills
  • Orchestration Tools (Must have): Apache Airflow
  • Expertise in developing ETL workflows comprising complex transformations like SCD, deduplications, aggregations etc.
  • Should have thorough conceptual understanding of AWS VPC, Subnets, Security Groups, & Route Tables
  • Should be a quick and self-learner and be ready to adapt to new AWS Services or new Big Data Technologies as and when required

Qualifications:

  • Bachelor's degree in computer science, engineering, or related field (Master's degree is a plus)
  • Demonstrated continued learning through one or more technical certifications or related methods
  • Minimum 5 years of experience on Cloud related projects

Qualities:

  • Hold strong technical knowledge and experience
  • Should have the capability to deep dive and research in various technical related fields
  • Self-motivated and focused on delivering outcomes for a fast-growing team and firm
  • Able to communicate persuasively through speaking, writing, and client presentations
  • Able to consult, write, and present persuasively
  • Able to work in a self-organized and cross-functional team
  • Able to iterate based on new information, peer reviews, and feedback
  • Prior experience of working in a large media company would be added advantage

  • AWS Data Engineering Expert

    6 days ago

    Solapur, Maharashtra, India beBeeDataPipeline Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Senior AWS Cloud Architect PositionWe are looking for a skilled Senior AWS Cloud Architect to lead the design, development, and deployment of end-to-end data pipelines on AWS cloud infrastructure.The ideal candidate will have hands-on experience in designing and building large-scale data processing pipelines in a production environment.A strong proficiency...

  • Senior Data Engineer Specialist

    6 days ago

    Solapur, Maharashtra, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,25,00,000

    Job Title: Senior Data Engineering SpecialistOverviewWe are seeking a highly skilled and experienced Data Engineer to join our team. As a Senior Data Engineer, you will play a key role in designing, developing, and maintaining our data infrastructure.Key Responsibilities:Design and Develop Scalable Data Pipelines: Utilize PySpark to create efficient and...

  • AWS Data Architect

    2 days ago

    Solapur, Maharashtra, India Bajaj Finserv Full time

    AWS Data Architecture - Sr Data EngineerExperience : 8-10 yearsLocation - GuragonMust : AWS Data Architecture; Need to do a lot of internal collaborations/reporting; Close coordination with CITO for any strategic initiatives.Key Responsibilities:Design end-to-end data architectures using AWS-native services such as Glue, Redshift, Athena, EMR, S3, Lambda,...

  • AWS Data Platform Expert

    1 week ago

    Solapur, Maharashtra, India beBeeRedshift Full time ₹ 12,00,000 - ₹ 16,00,000

    Job Title: Senior AWS Redshift EngineerRole Overview:We are seeking a highly skilled Senior Redshift Engineer to join our team and support our clients in architecting, optimizing, and scaling AWS Redshift environments.Key Responsibilities:Lead architecture review and validation for multiple AWS Redshift clustersValidate sizing assumptions and provide...

  • Senior Data Engineer

    7 days ago

    Solapur, Maharashtra, India beBeeDataEngineering Full time US$ 90,000 - US$ 1,35,000

    Improve your data engineering skills with this exciting opportunity to design and implement robust data pipelines using AWS, Python, and SQL.About the RoleThe role requires strong technical expertise in designing, building, and optimizing data warehousing and data lake solutions. You will work closely with the team to gather and analyze data from various...

  • AWS Cloud Data Specialist

    6 days ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: AWS Data Engineer – Oil & Gas DomainWe are seeking an experienced AWS data professional with expertise in the oil and gas industry to design, build, and optimize scalable data pipelines and cloud solutions.This role involves working with domain experts, data scientists, and business analysts to translate oil and gas business requirements into...

  • AWS Cloud Engineer

    6 days ago

    Solapur, Maharashtra, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    AWS Cloud Engineer - AI Integration SpecialistJob Description:We are seeking a highly skilled AWS Cloud Engineer with expertise in Artificial Intelligence (AI) integration to join our team. The successful candidate will be responsible for designing, building, and maintaining scalable and secure cloud-based solutions that leverage the power of AI.Key...

  • Senior Full Stack Backend Developer – AWS Architect + + DB Engineering

    1 week ago

    Solapur, Maharashtra, India Uplevyl Full time

    Senior Full Stack Backend Developer (AWS Architect + Node.js + DB Engineer)Experience Required: 6+ YearsDomain: AI-Powered Social Platform / Media / E-CommerceAbout the RoleWe are seeking a highly skilled Senior Full Stack Backend Developer with proven expertise across AWS architecture, Node.js backend development, and scalable database design. You'll be...

  • Senior Data Engineer

    7 days ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Senior Data EngineerAbout the RoleWe are seeking a highly skilled and experienced Senior Data Engineer to drive the technical depth, accuracy, and instructional value of our training content.This role is responsible for defining and validating learning content aligned to industry-recognized certification paths, providing technical guidance on data...

  • Senior Data Engineer

    13 hours ago

    Solapur, Maharashtra, India Celebal Technologies Full time

    Job Description- Senior Data Engineer (Teradata Migration Specialist)About Us Celebal Technologies is a leading Solution Service company that provides services in the fields of Data Science, Big Data, Enterprise Cloud, and Automation. We are at the forefront of leveraging cutting-edge technologies to drive innovation and enhance our business processes. As...