Capgemini - AWS EMR Engineer

2 months ago


Anywhere in IndiaMultiple Locations, IN Capgemini Full time

Roles & Responsibilities - AWS+EMR

- Minimum 2-6 years of experience in AWS technologies.

- Setting up and managing EMR clusters for processing large-scale data using frameworks like Apache Hadoop, Apache Spark, Apache Hive, etc.

- Configuring EMR clusters based on specific requirements, including choosing the appropriate instance types, storage configurations, and software settings.

- Implementing and optimizing data processing workflows on EMR clusters, leveraging distributed computing frameworks for tasks such as data cleansing, transformation, and analysis.

- Writing scripts and code to interact with EMR clusters, often using languages like Python, Java, or Scala, to develop and execute data processing jobs.

- Integrating EMR with other AWS services, such as Amazon S3 for storage, AWS Glue for ETL (Extract, Transform, Load), AWS Lambda and other complementary services to create end-to-end data pipelines.

- Optimizing cluster performance by fine-tuning configurations, adjusting resource allocation, and implementing best practices for efficient data processing.

- Implementing monitoring solutions to track cluster performance and troubleshoot issues, ensuring the reliability and availability of the big data processing environment.

- Implementing security measures to protect data within EMR clusters, configuring access controls, encryption, and ensuring compliance with security policies.

- Implementing automation for cluster provisioning, scaling, and decommissioning to streamline operations and improve efficiency.

- Overall, an AWS EMR role requires a combination of cloud computing knowledge, big data processing expertise, scripting/coding skills, and a good understanding of data engineering principles.

- AWS certifications, such as the "AWS Solution Architect - Associate" or relevant certifications demonstrating expertise in cloud computing, are often preferred.

- Good communication skills to interact with cross-functional teams, understand business requirements, and effectively convey technical information.

- Ability to collaborate with data engineers, data scientists, and other stakeholders in a team-oriented environment.

Tech Stack :

Languages : Python, SQL, UNIX Shell Script

Frameworks : Apache Spark

Platforms : Linux, Windows, AWS

Hadoop Tools : Hive, Spark (Spark SQL)

AWS Tools - EMR, Lambda, Glue, IAM, Redshift, Athena

(ref:hirist.tech)
  • Cloud Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations, IN GENPACT India Private Limited Full time

    Key Responsibilities :- Develop an innovative data landscape for driving the business with Analytical solutions up to real-time with state-of-the- art data & analytics technologies on AWS and RDBMS.- Design & Build pipelines with Continuous Delivery practices, to collect, normalize, index, integrate and publish data coming from multiple sources.- Build...

  • DevOps Engineer

    1 month ago


    Anywhere in India/Multiple Locations/Bangalore/Lucknow, IN Useful BI Corporation Full time

    UsefulBI is looking for an 4+ years experienced DevOps Engineer with strong analytics skills to work with our team. This is key role in delivering high impact projects across financial, business intelligence with a focus on cloud-based solutions. The scope would be to a Perform server and chassis builds and configuration, and network switch configuration per...

  • Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations, IN Risk Resources LLP Full time

    Overview :The Data Engineer specializing in Snowflake, Azure, and AWS technologies plays a crucial role in designing, developing, and maintaining scalable data solutions to meet the organization's evolving data needs. The role requires expertise in a range of data engineering tools and platforms for managing and transforming large sets of structured and...

  • AWS Practice Lead

    2 months ago


    Anywhere in India/Multiple Locations, IN Winfort Full time

    About us :We are a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. With strong over 50 years, we are trusted by our clients to unlock the value of technology to address the entire breadth of their...

  • Big Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations, IN D-TechWorks Pvt Ltd Full time

    We are looking for Aws big data engineer In Remote , India Position : Aws big data engineerExperience : 7 to 9 YearLocation : Remote , India Mandate skills : BigData Python Pyspark AWS Glue Airflow Redshift Responsibilities :- Ensure effective Design, Development, Validation and Support activities in line with client needs and architectural requirements.,-...

  • AWS Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations, IN Hum Technologies Full time

    Job Description :We are hiring for AWS Data Engineer with strong exp in AWS, SQL, Python , Spark & CI/CD for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : Remote/HybridSkills : AWS (5+ years), Python, PySpark, CI/CD, ETL/ELT, data pipelines, Jenkins, SQL, testing, Agile project management,...

  • AWS Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations, IN IT Source Global Full time

    We have Immediate Openings on AWS data engineerJob Description :- Data Pipeline Development: Design and develop ETL processes using AWS Glue, Python, and PySpark to extract, transform, and load data from various sources into data lakes or data warehouses.- Data Integration: Integrate data from multiple sources, ensuring data quality, consistency, and...


  • Anywhere in India/Multiple Locations, IN Freecharge Full time

    The role of a DevOps Engineer at FreeCharge DBAT team. We are looking for a highly motivated DevOps engineer who is proficient in implementing and managing DevOps practices and technologies. Your primary focus will be on designing and maintaining the infrastructure, tools, and processes to support the continuous integration, delivery, and deployment of our...

  • Big Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations, IN TalentXO Full time

    Required Qualifications :- 7-8 years of experience as a Data Engineer or in a similar role- Proficient in Python programming and PySpark for data processing and transformation- Extensive experience in designing and implementing data pipelines using AWS Glue and AWS MWAA (Airflow)- Strong SQL skills, including experience with AWS Redshift- Familiarity with...

  • UsefulBI Corporation

    2 months ago


    Anywhere in India/Multiple Locations, IN USEFULBI CORPORATION Full time

    Job Role : Solution Architect. Location : Remote. Job Description : - Minimum 8+years' Experience in Data Engineering. - Must have good knowledge and experience in Python. - Must have good Knowledge of Pyspark or Spark. - Must have good experience in AWS/Azure (Glue,- EMR). - Typically requires relevant analysis work and domain-area work experience. -...

  • AppSierra Solutions

    1 month ago


    Anywhere in India/Multiple Locations, IN AppSierra Solutions Pvt Ltd Full time

    Description :We are seeking an experienced AWS Data Engineer proficient in AWS technologies, PySpark, Glue, S3, and Terraform to join our innovative team. As an integral part of our data engineering group, you will be responsible for designing, building, and maintaining scalable data pipelines that facilitate seamless data extraction, transformation, and...


  • Anywhere in India/Multiple Locations, IN NucleusTeq Consulting Pvt. Ltd. Full time

    Job Description :Responsibilities :- Design, develop, and maintain scalable data pipelines and systems on AWS.- Implement data integration solutions using AWS services such as S3, Redshift, Glue, and Lambda.- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.- Optimize and tune data...

  • Robosoft Technologies

    2 months ago


    Anywhere in India/Multiple Locations, IN Robosoft Technologies Full time

    Job Description :- 5+ years of experience in AWS environment operation and management- Experience building and managing AWS environments- Experience in creating and using automated infrastructure as code like CloudFormation- Experience developing code in at least one high-level programming language- Experience with AWS services such as ECS, EC2(Auto scaling,...


  • Anywhere in India/Multiple Locations, IN NexGen Tech Solutions Full time

    Location : India (Remote). Experience : 12+ Years. Key Responsibilities : - Develop and maintain Python scripts and applications for data processing, automation, and integration tasks. - Design and optimize SQL/PLSQL queries and database interactions for high performance and reliability. - Configure and manage AWS services, with a strong emphasis on AWS...


  • Anywhere in India/Multiple Locations, IN BSRI Solutions Full time

    SALARY : 6LPA - 20LPAJob Description :- Design, document, develop and deploy architectural solutions for cloud-native web services, user interfaces, communications, and data storage services in AWS. - Use extensive knowledge of APIs to design RESTful services and integrate them with existing data providers, using JSON or XML as needed - Must have experience...


  • Anywhere in India/Multiple Locations, IN Excelra Knowledge Solutions Full time

    Data Engineer (Cloud & Ontologies)Job Description : Excelra is hiring for experienced Data Engineer with expertise in ontologies, AWS, Azure, and various data technologies. The role involves building and maintaining scalable data pipelines, working with cloud platforms, and supporting server and application deployment.Roles and Responsibilities : - Cloud...

  • Snowflake Architect

    2 months ago


    Anywhere in India/Multiple Locations, IN Response Informatics Full time

    Key Responsibilities :- Data Architecture Design : Lead the design and development of scalable, efficient, and secure data architectures using Snowflake, ensuring alignment with business objectives.- Data Integration : Utilize Fivetran for seamless data extraction, transformation, and loading (ETL) processes, ensuring that data from various sources is...

  • UsefulBI Corporation

    4 weeks ago


    Bangalore/Lucknow/Anywhere in India/Multiple Locations, IN Useful BI Corporation Full time

    UsefulBI is looking for an 4+ years experienced DevOps Engineer with strong analytics skills to work with our team. This is key role in delivering high impact projects across financial, business intelligence with a focus on cloud-based solutions. The scope would be to a Perform server and chassis builds and configuration, and network switch configuration per...

  • Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations, IN Innotatz IT Solutions Full time

    Job Description :We are looking for candidates who can join immediately. InnotatzIT Solutions is a global Software Development, IT Product Design and IT Consulting Company predominantly focused on Web Application Development, Digital Innovations & IT Recruitment Solutions. We also provide global digital consultancy that helps brands leverage design,...

  • AWS Cloud Engineer

    2 months ago


    Anywhere in India/Multiple Locations, IN SMARTWORK IT SERVICES Full time

    Key Responsibilities :Cloud Architecture Design : - Design and implement scalable, secure, and high-performance cloud solutions using AWS services and best practices.Infrastructure Management : - Manage and monitor AWS cloud infrastructure, including compute, storage, networking, and database resources.- Ensure optimal performance and...