AWS Data Engineer

4 days ago

Andhra Pradesh, India Inityinfotech Full time ₹ 15,00,000 - ₹ 25,00,000 per year

AWS Data Engineer

Experience :- 5+ years

Location : Remote

Job Description

Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.

Writing reusable, testable, and efficient code

Integration of data storage solutions in spark – especially with AWS S3 object storage. Performance tuning of pySpark scripts.

Need to ensure overall build delivery quality is good and on-time delivery is done at all times.

Should be able to handle meetings with customers with ease.

Need to have excellent communication skills to interact with the customer.

Be a team player and willing to work in an onsite-offshore model, mentor other folks in the team (onsite as well as offshore)

*
5+ years of experience in programming with python. Strong proficiency in python

*Familiarity with functional programming concepts

*
3+ years of hands-on experience in developing ETL data pipelines using pySpark on AWS EMR

*Experience in building pipelines and data lake for large enterprises on AWS

*Good understanding of Spark's Dataframe and API

*Experience in configuring EMR clusters on AWS

*Experience in dealing with AWS S3 object storage from Spark.

*Experience in troubleshooting spark jobs. Knowledge of monitoring spark jobs using Spark UI

*Performance tuning of Spark jobs.

*Understanding fundamental design principles behind business processes

Process Knowledge and Expertise:

Demonstrated experience in change management processes, including understanding of governance frameworks and preparation of supporting artefacts required for approvals.

Strong clarity on the path to production, with hands-on involvement in deployments, testing cycles, and obtaining business sign-offs.

Proven track record in technical solution design, with the ability to provide architectural guidance and support implementation strategies.

Databricks-Specific Skills

:

Experience in at least developing and delivering end-to-end Proof of Concept (POC) solutions covering the below:

Basic proficiency in Databricks, including creating jobs and configuring clusters.

Exposure to connecting external data sources (e.g., Amazon S3) to Databricks.

Understanding of Unity Catalog and its role in data governance.

Familiarity with notebook orchestration and implementing modular code structures to enhance scalability and maintainability.

  • Aws Data

    4 days ago

    Andhra Pradesh, India Virtusa Full time

    P3-C3-STS AWS Data Engineer Design and build scalable data pipelines using AWS services like AWS Glue, Amazon Redshift, and S3. Develop efficient ETL processes for data extraction, transformation, and loading into data warehouses and lakes. Automate repetitive tasks and build reusable frameworks to improve efficiency. Skill AWS Data Engineer - AWS Glue,...

  • IN | Ameritas: AWS Data Engineer: Offshore

    3 days ago

    Andhra Pradesh, India Growel Softech Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    - Primary/Mandatory Experience - Experience building data pipeline in Python along with AWS services (S3 SNS CloudWatch Lambda Step Function etc) - Proficient in AWS Server less technologies - Technical knowledge of Extract/Transform/Load (ETL) solutions for running analytic projects on the cloud - Candidate must have hands-on technical experience on the AWS...

  • AWS Tester

    2 weeks ago

    Andhra Pradesh, India Virtusa Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role : AWS TesterWe are looking for a highly skilled Automation QA Engineer with hands-on experience in AWS and Big Data ecosystems to join our Quality Engineering team. The ideal candidate will be responsible for developing automated test frameworks and validating large-scale data processing pipelines in cloud-based environments. This role requires strong...

  • Data Engineer

    2 days ago

    Andhra Pradesh, India Growel Softech Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    :- Location :Any location in India Years of Experience:Level 2 (3 to ?5 Years) Data Engineer About The Role : for Data Journey USBIE Experience developing and deploying application code using SQL Python spark. 3-5 years experience developing and deploying data pipeline in cloud. 3-5 years experience in AWS using glue athena lambda secrets manager redshift...

  • Aws Snowflake

    4 days ago

    Andhra Pradesh, India Virtusa Full time

    Design, develop, and maintain data pipelines and ETL processes using AWS and Snowflake. Implement data transformation workflows using DBT (Data Build Tool). Write efficient, reusable, and reliable code in Python. Optimize and tune data solutions for performance and scalability. Collaborate with data scientists, analysts, and other stakeholders to understand...

  • AWS Data Engineer

    5 days ago

    uttar pradesh, India Coforge Full time

    We are seeking a skilled Data Engineer (AWS- Scala Mandatory) to join our dynamic team. This role involves building and maintaining robust data pipelines, designing scalable databases, and collaborating with stakeholders to deliver high-quality data solutions.ResponsibilitiesBuild and maintain data pipelines using AWS Glue, Python, EMR/Spark, and...

  • AWS Data Engineer

    4 days ago

    uttar pradesh, India Coforge Full time

    We are seeking a skilled Data Engineer (AWS- Scala Mandatory) to join our dynamic team. This role involves building and maintaining robust data pipelines, designing scalable databases, and collaborating with stakeholders to deliver high-quality data solutions. Responsibilities Build and maintain data pipelines using AWS Glue , Python , EMR/Spark , and...

  • Aws Python

    6 days ago

    Andhra Pradesh, India Virtusa Full time

    JD for Data Engineer Python At least 5 to 8 years of experience in AWS Python programming and who can design, build, test & deploy the code. Should have experience in using following AWS services: AWS SQS, AWS MSK, AWS RDS Aurora DB, BOTO 3. Very strong SQL knowledge is a must, should be able to understand build complex queries. He/she should be closely...

  • Python Aws

    4 days ago

    Andhra Pradesh, India Virtusa Full time

    Python AWS Job Summary We are looking for a highly motivated and skilled Back End Engineer with expertise in Python FastAPI and AWS cloud services to join our engineering team This role focuses on building and deploying robust APIs and MCP Microservice Control Plane server components leveraging modern containerization and cloud native technologies Key...

  • Aws De

    6 days ago

    Andhra Pradesh, India Virtusa Full time

    Must Have Strong hands-on Experience in AWS Native Services Python, Pyspark Data Engineering(S3, EMR, Glue, Sqoop) Serverless Experience(Lambda, Step Functions) Strong SQL AWS Athena, Cloud Formation Templates, Crawler Strong Data analysis and troubleshooting skill Datamart / Data Warehousing Multi-Dimensional OLAP Star/Snowflake Schema Design **About...