Capgemini - AWS EMR Engineer

1 week ago


Anywhere in IndiaMultiple Locations Capgemini Full time

Roles & Responsibilities - AWS+EMR

- Minimum 2-6 years of experience in AWS technologies.

- Setting up and managing EMR clusters for processing large-scale data using frameworks like Apache Hadoop, Apache Spark, Apache Hive, etc.

- Configuring EMR clusters based on specific requirements, including choosing the appropriate instance types, storage configurations, and software settings.

- Implementing and optimizing data processing workflows on EMR clusters, leveraging distributed computing frameworks for tasks such as data cleansing, transformation, and analysis.

- Writing scripts and code to interact with EMR clusters, often using languages like Python, Java, or Scala, to develop and execute data processing jobs.

- Integrating EMR with other AWS services, such as Amazon S3 for storage, AWS Glue for ETL (Extract, Transform, Load), AWS Lambda and other complementary services to create end-to-end data pipelines.

- Optimizing cluster performance by fine-tuning configurations, adjusting resource allocation, and implementing best practices for efficient data processing.

- Implementing monitoring solutions to track cluster performance and troubleshoot issues, ensuring the reliability and availability of the big data processing environment.

- Implementing security measures to protect data within EMR clusters, configuring access controls, encryption, and ensuring compliance with security policies.

- Implementing automation for cluster provisioning, scaling, and decommissioning to streamline operations and improve efficiency.

- Overall, an AWS EMR role requires a combination of cloud computing knowledge, big data processing expertise, scripting/coding skills, and a good understanding of data engineering principles.

- AWS certifications, such as the "AWS Solution Architect - Associate" or relevant certifications demonstrating expertise in cloud computing, are often preferred.

- Good communication skills to interact with cross-functional teams, understand business requirements, and effectively convey technical information.

- Ability to collaborate with data engineers, data scientists, and other stakeholders in a team-oriented environment.

Tech Stack :

Languages : Python, SQL, UNIX Shell Script

Frameworks : Apache Spark

Platforms : Linux, Windows, AWS

Hadoop Tools : Hive, Spark (Spark SQL)

AWS Tools - EMR, Lambda, Glue, IAM, Redshift, Athena

(ref:hirist.tech)
  • AWS EMR Engineer

    1 month ago


    Anywhere in India/Multiple Locations capgemini Full time

    Job SummaryJoin Capgemini as an AWS EMR Engineer and take on the challenge of managing and optimizing big data processing workflows on Amazon EMR clusters. This role requires expertise in cloud computing, big data processing, and scripting/coding skills, with a focus on delivering efficient and scalable data processing solutions.Key Responsibilities- Minimum...


  • india Capgemini Full time

    About Capgemini Capgemini is a global leader in consulting, technology services, and digital transformation. With over 50 years of experience, we leverage technology to enable business transformation for clients across various industries. Our mission is to create and deliver business and technology solutions that fit our clients' needs and drive the results...


  • india Capgemini Engineering Full time

    At Capgemini, we work with the world’s leading brands to enhance and transform the way they do business. We do this with passion and by applying the human touch to business and technology. Given your work experience, we believe it would be a great opportunity for you to progress in your career and get the future you want at Capgemini. ROLE: Java BDD...


  • india Capgemini Full time

    Proficiency in Java and familiarity with Java frameworks such as Spring, Spring boot, Hibernate, or Java EE.Experience in Microservices, AWS.Experience with web technologies and frameworks (e.g., Angular, HTML, CSS, JavaScript, RESTful APIs).Strong understanding of object-oriented programming (OOP) principles and design patterns.Experience with relational...


  • india Capgemini Full time

    Proficiency in Java and familiarity with Java frameworks such as Spring, Spring boot, Hibernate, or Java EE. Experience in Microservices, AWS. Experience with web technologies and frameworks (e.g., Angular, HTML, CSS, JavaScript, RESTful APIs). Strong understanding of object-oriented programming (OOP) principles and design patterns. Experience with...


  • india Capgemini Full time

    Proficiency in Java and familiarity with Java frameworks such as Spring, Spring boot, Hibernate, or Java EE. Experience in Microservices, AWS. Experience with web technologies and frameworks (e.g., Angular, HTML, CSS, JavaScript, RESTful APIs). Strong understanding of object-oriented programming (OOP) principles and design patterns. Experience with...

  • DevOps Engineer

    5 days ago


    Anywhere in India/Multiple Locations/Bangalore/Lucknow Useful BI Corporation Full time

    UsefulBI is looking for an 4+ years experienced DevOps Engineer with strong analytics skills to work with our team. This is key role in delivering high impact projects across financial, business intelligence with a focus on cloud-based solutions. The scope would be to a Perform server and chassis builds and configuration, and network switch configuration...


  • india Capgemini Full time

    Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of...

  • Data Engineer

    1 week ago


    Anywhere in India/Multiple Locations Risk Resources LLP Full time

    Overview : The Data Engineer specializing in Snowflake, Azure, and AWS technologies plays a crucial role in designing, developing, and maintaining scalable data solutions to meet the organization's evolving data needs. The role requires expertise in a range of data engineering tools and platforms for managing and transforming large sets of structured...


  • india Capgemini Engineering Full time

    About the Role - As an experienced Scrum Master at Capgemini, you will lead and facilitate scrum and agile ceremonies for multiple web application development scrum teams, ensuring the delivery of our Development Backlog. The ideal candidate will coordinate and manage dependencies, assist in removing impediments, and maintain accurate tracking of all work...


  • Anywhere in India/Multiple Locations/Bangalore/Hyderabad capgemini Full time

    Job Description :- You will be part of the team verifying IPs and SoCs leading to first Si success.- Manage and lead a team of Verification engineers- IP verification is coverage driven using latest industry standard methodologies and HVLs.- Work involves defining verification strategy, writing test plans, developing efficient test benches and test cases.-...


  • India Capgemini Engineering Full time

    About the RoleWe are seeking a talented Front-end and Back-end Developer to join our team at Capgemini Engineering.Salary InformationThe estimated annual salary for this position is ₹1,200,000 - ₹1,800,000 based on industry standards and location.Job DescriptionThis role involves working as a full stack web developer using Angular and VueJS for front-end...

  • AWS Data Engineer

    4 weeks ago


    india Vantage Point Consulting Inc. Full time

    AWS Data Engineer - Hadoop MigrationLocation: Chennai or BengaluruRole Type: Full-TimeWe’re seeking a skilled AWS Principal Data Architect to lead the migration of Hadoop DWH workloads from on-premise to AWS EMR. Join our team and help design and optimize scalable, secure, and resilient cloud data architectures for enterprise data processing.Key...

  • Technical Expert

    1 day ago


    India Capgemini Engineering Full time

    At Capgemini Engineering, we're seeking an exceptional Technical Expert to lead our industry 4.0 initiatives in India.The estimated annual salary for this role is ₹800,000 - ₹1,200,000 depending on experience.Job Description:Develop and implement industrial application server solutions with a focus on historian and InTouch.Manage workflows, work tasks,...

  • Big Data Engineer

    7 days ago


    Anywhere in India/Multiple Locations D-TechWorks Pvt Ltd Full time

    We are looking for Aws big data engineer In Remote , India Position : Aws big data engineerExperience : 7 to 9 YearLocation : Remote , India Mandate skills : BigData Python Pyspark AWS Glue Airflow Redshift Responsibilities :- Ensure effective Design, Development, Validation and Support activities in line with client needs and architectural requirements.,-...

  • Aws data engineer

    2 months ago


    India FLYONIT Full time

    Job Summary:   We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for...

  • AWS Data Engineer

    2 months ago


    india FLYONIT Full time

    Job Summary: We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing...

  • AWS Data Engineer

    1 week ago


    Anywhere in India/Multiple Locations Hum Technologies Full time

    Job Description : We are hiring for AWS Data Engineer with strong exp in AWS, SQL, Python , Spark & CI/CD for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : Remote/Hybrid Skills : AWS (5+ years), Python, PySpark, CI/CD, ETL/ELT, data pipelines, Jenkins, SQL, testing, Agile project...

  • AWS Data Engineer

    1 month ago


    Anywhere in India/Multiple Locations IT Source Global Full time

    We have Immediate Openings on AWS data engineerJob Description :- Data Pipeline Development: Design and develop ETL processes using AWS Glue, Python, and PySpark to extract, transform, and load data from various sources into data lakes or data warehouses.- Data Integration: Integrate data from multiple sources, ensuring data quality, consistency, and...


  • india Capgemini Engineering Full time

    Capgemini Engineering is looking for Oracle BI Consultant.Responsabilities :Person will carry-out hands-on work design, development, testing, UAT, deployment, and provide support during regular hours and on call dutyResponsible for the day-to-day operations of the products in the familySupports the relationships with key software vendors, reviews their...