AWS EMR Engineer

1 month ago


Anywhere in IndiaMultiple Locations capgemini Full time

Job Summary

Join Capgemini as an AWS EMR Engineer and take on the challenge of managing and optimizing big data processing workflows on Amazon EMR clusters. This role requires expertise in cloud computing, big data processing, and scripting/coding skills, with a focus on delivering efficient and scalable data processing solutions.

Key Responsibilities

- Minimum 2-6 years of experience in AWS technologies.

- Setting up and managing EMR clusters for processing large-scale data using frameworks like Apache Hadoop, Apache Spark, and Apache Hive.

- Configuring EMR clusters based on specific requirements, including choosing the appropriate instance types, storage configurations, and software settings.

- Implementing and optimizing data processing workflows on EMR clusters, leveraging distributed computing frameworks for tasks such as data cleansing, transformation, and analysis.

- Writing scripts and code to interact with EMR clusters, often using languages like Python, Java, or Scala, to develop and execute data processing jobs.

- Integrating EMR with other AWS services, such as Amazon S3 for storage, AWS Glue for ETL, AWS Lambda, and other complementary services to create end-to-end data pipelines.

- Optimizing cluster performance by fine-tuning configurations, adjusting resource allocation, and implementing best practices for efficient data processing.

- Implementing monitoring solutions to track cluster performance and troubleshoot issues, ensuring the reliability and availability of the big data processing environment.

- Implementing security measures to protect data within EMR clusters, configuring access controls, encryption, and ensuring compliance with security policies.

- Implementing automation for cluster provisioning, scaling, and decommissioning to streamline operations and improve efficiency.

- Overall, this role requires a combination of cloud computing knowledge, big data processing expertise, scripting/coding skills, and a good understanding of data engineering principles.

Preferred Qualifications

- AWS certifications, such as the "AWS Solution Architect - Associate" or relevant certifications demonstrating expertise in cloud computing, are often preferred.

- Good communication skills to interact with cross-functional teams, understand business requirements, and effectively convey technical information.

- Ability to collaborate with data engineers, data scientists, and other stakeholders in a team-oriented environment.

Technical Requirements

Languages: Python, SQL, UNIX Shell Script

Frameworks: Apache Spark

Platforms: Linux, Windows, AWS

Hadoop Tools: Hive, Spark (Spark SQL)

AWS Tools - EMR, Lambda, Glue, IAM, Redshift, Athena



  • Anywhere in India/Multiple Locations Capgemini Full time

    Roles & Responsibilities - AWS+EMR- Minimum 2-6 years of experience in AWS technologies.- Setting up and managing EMR clusters for processing large-scale data using frameworks like Apache Hadoop, Apache Spark, Apache Hive, etc.- Configuring EMR clusters based on specific requirements, including choosing the appropriate instance types, storage configurations,...

  • DevOps Engineer

    2 weeks ago


    Anywhere in India/Multiple Locations/Bangalore/Lucknow Useful BI Corporation Full time

    UsefulBI is looking for an 4+ years experienced DevOps Engineer with strong analytics skills to work with our team. This is key role in delivering high impact projects across financial, business intelligence with a focus on cloud-based solutions. The scope would be to a Perform server and chassis builds and configuration, and network switch configuration...

  • Data Engineer

    20 hours ago


    Anywhere in India/Multiple Locations Risk Resources LLP Full time

    Join Risk Resources LLP as a highly skilled Data Engineer specializing in designing, developing, and maintaining scalable data solutions on cloud platforms.About the RoleWe are seeking a seasoned professional with expertise in Snowflake, Azure, and AWS technologies to lead our data engineering efforts. As a key member of our team, you will be responsible for...

  • Data Engineer

    2 weeks ago


    Anywhere in India/Multiple Locations Risk Resources LLP Full time

    Overview : The Data Engineer specializing in Snowflake, Azure, and AWS technologies plays a crucial role in designing, developing, and maintaining scalable data solutions to meet the organization's evolving data needs. The role requires expertise in a range of data engineering tools and platforms for managing and transforming large sets of structured...

  • Aws data engineer

    4 days ago


    India Tata Consultancy Services Full time

    Dear Candidate, Greetings from TATA CONSULTANCY SERVICES LIMITED!!! Skill: AWS Data Engineer + Python & Scala Location: PAN INDIA Experience: 4 to 8 years Industry experience in Data Engineering on AWS cloud with glue, redshift , Athena experience. Ability to write high quality, maintainable, and robust code, often in SQL, Scala and Python....

  • AWS Data Engineer

    1 month ago


    india Vantage Point Consulting Inc. Full time

    AWS Data Engineer - Hadoop MigrationLocation: Chennai or BengaluruRole Type: Full-TimeWe’re seeking a skilled AWS Principal Data Architect to lead the migration of Hadoop DWH workloads from on-premise to AWS EMR. Join our team and help design and optimize scalable, secure, and resilient cloud data architectures for enterprise data processing.Key...

  • Big Data Engineer

    2 weeks ago


    Anywhere in India/Multiple Locations D-TechWorks Pvt Ltd Full time

    We are looking for Aws big data engineer In Remote , India Position : Aws big data engineerExperience : 7 to 9 YearLocation : Remote , India Mandate skills : BigData Python Pyspark AWS Glue Airflow Redshift Responsibilities :- Ensure effective Design, Development, Validation and Support activities in line with client needs and architectural requirements.,-...

  • AWS Data Engineer

    2 months ago


    india FLYONIT Full time

    Job Summary: We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing...

  • Aws data engineer

    2 months ago


    India FLYONIT Full time

    Job Summary:   We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for...

  • AWS Data Engineer

    2 weeks ago


    Anywhere in India/Multiple Locations Hum Technologies Full time

    Job Description : We are hiring for AWS Data Engineer with strong exp in AWS, SQL, Python , Spark & CI/CD for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : Remote/Hybrid Skills : AWS (5+ years), Python, PySpark, CI/CD, ETL/ELT, data pipelines, Jenkins, SQL, testing, Agile project...

  • AWS Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations IT Source Global Full time

    We have Immediate Openings on AWS data engineerJob Description :- Data Pipeline Development: Design and develop ETL processes using AWS Glue, Python, and PySpark to extract, transform, and load data from various sources into data lakes or data warehouses.- Data Integration: Integrate data from multiple sources, ensuring data quality, consistency, and...


  • Anywhere in India/Multiple Locations Freecharge Full time

    The role of a DevOps Engineer at FreeCharge DBAT team. We are looking for a highly motivated DevOps engineer who is proficient in implementing and managing DevOps practices and technologies. Your primary focus will be on designing and maintaining the infrastructure, tools, and processes to support the continuous integration, delivery, and deployment of our...

  • Big Data Engineer

    1 week ago


    Anywhere in India/Multiple Locations TalentXO Full time

    Required Qualifications : - 7-8 years of experience as a Data Engineer or in a similar role - Proficient in Python programming and PySpark for data processing and transformation - Extensive experience in designing and implementing data pipelines using AWS Glue and AWS MWAA (Airflow) - Strong SQL skills, including experience with AWS Redshift -...

  • Cloud Data Architect

    2 weeks ago


    Anywhere in India/Multiple Locations Risk Resources LLP Full time

    Company OverviewRisk Resources LLP is a dynamic and innovative company that specializes in providing cutting-edge data solutions to our clients.SalaryWe are offering a competitive salary of $140,000 - $160,000 per year, depending on experience.Job DescriptionThe Cloud Data Architect will play a crucial role in designing, developing, and maintaining scalable...

  • AWS Cloud Engineer

    2 months ago


    Anywhere in India/Multiple Locations SMARTWORK IT SERVICES Full time

    Cloud ArchitectAt SMARTWORK IT SERVICES, we're seeking a skilled Cloud Architect to design and implement scalable, secure, and high-performance cloud solutions using AWS services and best practices.Key Responsibilities:Cloud Architecture Design: Develop and maintain cloud architectures that meet business needs and comply with security best...

  • AppSierra Solutions

    2 weeks ago


    Anywhere in India/Multiple Locations AppSierra Solutions Pvt Ltd Full time

    Description :We are seeking an experienced AWS Data Engineer proficient in AWS technologies, PySpark, Glue, S3, and Terraform to join our innovative team. As an integral part of our data engineering group, you will be responsible for designing, building, and maintaining scalable data pipelines that facilitate seamless data extraction, transformation, and...


  • India RapidBrains Full time

    OverviewRapidBrains is seeking a skilled Software Engineer to join our team in developing cutting-edge Health Electronic Medical Record (EMR) products.About the RoleWe are looking for a talented professional with strong foundation in .NET, C#, Azure, and Blazor frameworks to work on innovative solutions using the latest technologies. If you have a passion...


  • Anywhere in India/Multiple Locations AppSierra Solutions Pvt Ltd Full time

    At AppSierra Solutions Pvt Ltd, we are seeking a highly skilled AWS Data Engineer to drive innovation in our team. With a strong focus on technical excellence and collaboration, this role demands a unique blend of analytical and problem-solving skills to design and implement efficient data processing solutions using AWS technologies.About the RoleWe estimate...

  • Java Developer

    2 months ago


    Anywhere in India/Multiple Locations SDNA Full time

    Job Description : - 6.5 years of IT experience in application development and support.- Strong Hands-On MUST Experience of a minimum of 2 years with React. This is a UI Specialist Role and React is the most important skill for the role.- Strong Hands-On Experience with Core Java, J2EE, JMS &.EJBs- Strong Hands-On Experience with Spring framework...


  • Anywhere in India/Multiple Locations NexGen Tech Solutions Full time

    Location : India (Remote). Experience : 12+ Years. Key Responsibilities : - Develop and maintain Python scripts and applications for data processing, automation, and integration tasks. - Design and optimize SQL/PLSQL queries and database interactions for high performance and reliability. - Configure and manage AWS services, with a strong emphasis on AWS...