AWS Data Engineer

3 days ago


Andhra Pradesh, India Inityinfotech Full time ₹ 15,00,000 - ₹ 25,00,000 per year

AWS Data Engineer

Experience :- 5+ years

Location : Remote

Job Description

Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.

Writing reusable, testable, and efficient code

Integration of data storage solutions in spark – especially with AWS S3 object storage. Performance tuning of pySpark scripts.

Need to ensure overall build delivery quality is good and on-time delivery is done at all times.

Should be able to handle meetings with customers with ease.

Need to have excellent communication skills to interact with the customer.

Be a team player and willing to work in an onsite-offshore model, mentor other folks in the team (onsite as well as offshore)

*
5+ years of experience in programming with python. Strong proficiency in python

*Familiarity with functional programming concepts

*
3+ years of hands-on experience in developing ETL data pipelines using pySpark on AWS EMR

*Experience in building pipelines and data lake for large enterprises on AWS

*Good understanding of Spark's Dataframe and API

*Experience in configuring EMR clusters on AWS

*Experience in dealing with AWS S3 object storage from Spark.

*Experience in troubleshooting spark jobs. Knowledge of monitoring spark jobs using Spark UI

*Performance tuning of Spark jobs.

*Understanding fundamental design principles behind business processes

Process Knowledge and Expertise:

Demonstrated experience in change management processes, including understanding of governance frameworks and preparation of supporting artefacts required for approvals.

Strong clarity on the path to production, with hands-on involvement in deployments, testing cycles, and obtaining business sign-offs.

Proven track record in technical solution design, with the ability to provide architectural guidance and support implementation strategies.

Databricks-Specific Skills

:

Experience in at least developing and delivering end-to-end Proof of Concept (POC) solutions covering the below:

Basic proficiency in Databricks, including creating jobs and configuring clusters.

Exposure to connecting external data sources (e.g., Amazon S3) to Databricks.

Understanding of Unity Catalog and its role in data governance.

Familiarity with notebook orchestration and implementing modular code structures to enhance scalability and maintainability.



  • Andhra Pradesh, India Growel Softech Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    - Primary/Mandatory Experience - Experience building data pipeline in Python along with AWS services (S3 SNS CloudWatch Lambda Step Function etc) - Proficient in AWS Server less technologies - Technical knowledge of Extract/Transform/Load (ETL) solutions for running analytic projects on the cloud - Candidate must have hands-on technical experience on the AWS...

  • AWS Tester

    2 weeks ago


    Andhra Pradesh, India Virtusa Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role : AWS TesterWe are looking for a highly skilled Automation QA Engineer with hands-on experience in AWS and Big Data ecosystems to join our Quality Engineering team. The ideal candidate will be responsible for developing automated test frameworks and validating large-scale data processing pipelines in cloud-based environments. This role requires strong...

  • Data Engineer

    22 hours ago


    Andhra Pradesh, India Growel Softech Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    :- Location :Any location in India Years of Experience:Level 2 (3 to ?5 Years) Data Engineer About The Role : for Data Journey USBIE Experience developing and deploying application code using SQL Python spark. 3-5 years experience developing and deploying data pipeline in cloud. 3-5 years experience in AWS using glue athena lambda secrets manager redshift...

  • AWS Data Engineer

    4 days ago


    uttar pradesh, India Coforge Full time

    We are seeking a skilled Data Engineer (AWS- Scala Mandatory) to join our dynamic team. This role involves building and maintaining robust data pipelines, designing scalable databases, and collaborating with stakeholders to deliver high-quality data solutions.ResponsibilitiesBuild and maintain data pipelines using AWS Glue, Python, EMR/Spark, and...

  • QA Aws

    1 week ago


    Andhra Pradesh, India Virtusa Full time

    Key Responsibilities Design, implement, and execute test strategies for APIs, services, data pipelines, and user interfaces. Perform data validation and reconciliation across AWS-native data stores (such as S3, DynamoDB, RDS, Redshift, etc.). Develop automated tests for backend services, APIs, and front-end user interfaces. Collaborate with developers and...

  • Aws Devops Banking

    4 weeks ago


    Noida, Uttar Pradesh, India NTT DATA Full time

    Req ID 328257 NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a AWS Devops Banking to join our team in Noida Uttar Pradesh IN-UP India IN Responsibilities Expertise in Devops Design...


  • Andhra Pradesh, India The Cigna Group Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Position OverviewEvernorth, a leading Health Services company, is looking for exceptional data engineers/developers in our Data and Analytics organization. In this role, you will actively participate with your development team on initiatives that support Evernorth's strategic goals as well as subject matter experts to understand business logic you will be...

  • Java Aws Architect

    5 days ago


    Andhra Pradesh, India Virtusa Full time

    15 years of Architecting & Developing Experience working in Big Data Technologies using Apache, Hadoop, Spark & SQL etc. 5 years of strong Development/ Coding experience using Spark with Python Minimum 3 Years experience working on Databricks on AWS based Cloud Services 5 years of strong experience in Big Data design, modelling & Development such as star...

  • Big Data

    5 days ago


    Andhra Pradesh, India Virtusa Full time

    Experience At least 5 years of experience in AWS based projects. Technical skills Proficiency in Python and PySpark for data engineering tasks. Big Data Strong knowledge of Big Data technologies and data warehousing concepts. AWS services Experience with AWS Data Engineering stack, including S3, RDS, Athena, Glue, Lambda, and Step Functions. SQL Strong SQL...


  • Andhra Pradesh, India The Cigna Group Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Application Development Sr.Analyst – Data EngineeringPosition OverviewWe're seeking a versatile Data Engineer to contribute to all aspects of our customer data platform. You'll work on various data engineering tasks, from designing data pipelines to implementing data integration solutions, playing a crucial role in enhancing our data infrastructure.Roles &...