Capgemini - AWS EMR Engineer
2 months ago
Roles & Responsibilities - AWS+EMR
- Minimum 2-6 years of experience in AWS technologies.
- Setting up and managing EMR clusters for processing large-scale data using frameworks like Apache Hadoop, Apache Spark, Apache Hive, etc.
- Configuring EMR clusters based on specific requirements, including choosing the appropriate instance types, storage configurations, and software settings.
- Implementing and optimizing data processing workflows on EMR clusters, leveraging distributed computing frameworks for tasks such as data cleansing, transformation, and analysis.
- Writing scripts and code to interact with EMR clusters, often using languages like Python, Java, or Scala, to develop and execute data processing jobs.
- Integrating EMR with other AWS services, such as Amazon S3 for storage, AWS Glue for ETL (Extract, Transform, Load), AWS Lambda and other complementary services to create end-to-end data pipelines.
- Optimizing cluster performance by fine-tuning configurations, adjusting resource allocation, and implementing best practices for efficient data processing.
- Implementing monitoring solutions to track cluster performance and troubleshoot issues, ensuring the reliability and availability of the big data processing environment.
- Implementing security measures to protect data within EMR clusters, configuring access controls, encryption, and ensuring compliance with security policies.
- Implementing automation for cluster provisioning, scaling, and decommissioning to streamline operations and improve efficiency.
- Overall, an AWS EMR role requires a combination of cloud computing knowledge, big data processing expertise, scripting/coding skills, and a good understanding of data engineering principles.
- AWS certifications, such as the "AWS Solution Architect - Associate" or relevant certifications demonstrating expertise in cloud computing, are often preferred.
- Good communication skills to interact with cross-functional teams, understand business requirements, and effectively convey technical information.
- Ability to collaborate with data engineers, data scientists, and other stakeholders in a team-oriented environment.
Tech Stack :
Languages : Python, SQL, UNIX Shell Script
Frameworks : Apache Spark
Platforms : Linux, Windows, AWS
Hadoop Tools : Hive, Spark (Spark SQL)
AWS Tools - EMR, Lambda, Glue, IAM, Redshift, Athena
-
Cloud Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN GENPACT India Private Limited Full timeKey Responsibilities :- Develop an innovative data landscape for driving the business with Analytical solutions up to real-time with state-of-the- art data & analytics technologies on AWS and RDBMS.- Design & Build pipelines with Continuous Delivery practices, to collect, normalize, index, integrate and publish data coming from multiple sources.- Build...
-
DevOps Engineer
1 month ago
Anywhere in India/Multiple Locations/Bangalore/Lucknow, IN Useful BI Corporation Full timeUsefulBI is looking for an 4+ years experienced DevOps Engineer with strong analytics skills to work with our team. This is key role in delivering high impact projects across financial, business intelligence with a focus on cloud-based solutions. The scope would be to a Perform server and chassis builds and configuration, and network switch configuration per...
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Risk Resources LLP Full timeOverview :The Data Engineer specializing in Snowflake, Azure, and AWS technologies plays a crucial role in designing, developing, and maintaining scalable data solutions to meet the organization's evolving data needs. The role requires expertise in a range of data engineering tools and platforms for managing and transforming large sets of structured and...
-
AWS Practice Lead
2 months ago
Anywhere in India/Multiple Locations, IN Winfort Full timeAbout us :We are a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. With strong over 50 years, we are trusted by our clients to unlock the value of technology to address the entire breadth of their...
-
Big Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN D-TechWorks Pvt Ltd Full timeWe are looking for Aws big data engineer In Remote , India Position : Aws big data engineerExperience : 7 to 9 YearLocation : Remote , India Mandate skills : BigData Python Pyspark AWS Glue Airflow Redshift Responsibilities :- Ensure effective Design, Development, Validation and Support activities in line with client needs and architectural requirements.,-...
-
AWS Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Hum Technologies Full timeJob Description :We are hiring for AWS Data Engineer with strong exp in AWS, SQL, Python , Spark & CI/CD for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : Remote/HybridSkills : AWS (5+ years), Python, PySpark, CI/CD, ETL/ELT, data pipelines, Jenkins, SQL, testing, Agile project management,...
-
AWS Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN IT Source Global Full timeWe have Immediate Openings on AWS data engineerJob Description :- Data Pipeline Development: Design and develop ETL processes using AWS Glue, Python, and PySpark to extract, transform, and load data from various sources into data lakes or data warehouses.- Data Integration: Integrate data from multiple sources, ensuring data quality, consistency, and...
-
Anywhere in India/Multiple Locations, IN Freecharge Full timeThe role of a DevOps Engineer at FreeCharge DBAT team. We are looking for a highly motivated DevOps engineer who is proficient in implementing and managing DevOps practices and technologies. Your primary focus will be on designing and maintaining the infrastructure, tools, and processes to support the continuous integration, delivery, and deployment of our...
-
Big Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN TalentXO Full timeRequired Qualifications :- 7-8 years of experience as a Data Engineer or in a similar role- Proficient in Python programming and PySpark for data processing and transformation- Extensive experience in designing and implementing data pipelines using AWS Glue and AWS MWAA (Airflow)- Strong SQL skills, including experience with AWS Redshift- Familiarity with...
-
UsefulBI Corporation
2 months ago
Anywhere in India/Multiple Locations, IN USEFULBI CORPORATION Full timeJob Role : Solution Architect. Location : Remote. Job Description : - Minimum 8+years' Experience in Data Engineering. - Must have good knowledge and experience in Python. - Must have good Knowledge of Pyspark or Spark. - Must have good experience in AWS/Azure (Glue,- EMR). - Typically requires relevant analysis work and domain-area work experience. -...
-
AppSierra Solutions
1 month ago
Anywhere in India/Multiple Locations, IN AppSierra Solutions Pvt Ltd Full timeDescription :We are seeking an experienced AWS Data Engineer proficient in AWS technologies, PySpark, Glue, S3, and Terraform to join our innovative team. As an integral part of our data engineering group, you will be responsible for designing, building, and maintaining scalable data pipelines that facilitate seamless data extraction, transformation, and...
-
NucleusTeq - AWS Data Engineer - Cloud Services
3 weeks ago
Anywhere in India/Multiple Locations, IN NucleusTeq Consulting Pvt. Ltd. Full timeJob Description :Responsibilities :- Design, develop, and maintain scalable data pipelines and systems on AWS.- Implement data integration solutions using AWS services such as S3, Redshift, Glue, and Lambda.- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.- Optimize and tune data...
-
Robosoft Technologies
2 months ago
Anywhere in India/Multiple Locations, IN Robosoft Technologies Full timeJob Description :- 5+ years of experience in AWS environment operation and management- Experience building and managing AWS environments- Experience in creating and using automated infrastructure as code like CloudFormation- Experience developing code in at least one high-level programming language- Experience with AWS services such as ECS, EC2(Auto scaling,...
-
Cloud/API Integration Engineer
2 months ago
Anywhere in India/Multiple Locations, IN NexGen Tech Solutions Full timeLocation : India (Remote). Experience : 12+ Years. Key Responsibilities : - Develop and maintain Python scripts and applications for data processing, automation, and integration tasks. - Design and optimize SQL/PLSQL queries and database interactions for high performance and reliability. - Configure and manage AWS services, with a strong emphasis on AWS...
-
Cloud Software Engineer
2 months ago
Anywhere in India/Multiple Locations, IN BSRI Solutions Full timeSALARY : 6LPA - 20LPAJob Description :- Design, document, develop and deploy architectural solutions for cloud-native web services, user interfaces, communications, and data storage services in AWS. - Use extensive knowledge of APIs to design RESTful services and integrate them with existing data providers, using JSON or XML as needed - Must have experience...
-
Excelra - Data Engineer - Python/AWS/Azure
2 months ago
Anywhere in India/Multiple Locations, IN Excelra Knowledge Solutions Full timeData Engineer (Cloud & Ontologies)Job Description : Excelra is hiring for experienced Data Engineer with expertise in ontologies, AWS, Azure, and various data technologies. The role involves building and maintaining scalable data pipelines, working with cloud platforms, and supporting server and application deployment.Roles and Responsibilities : - Cloud...
-
Snowflake Architect
2 months ago
Anywhere in India/Multiple Locations, IN Response Informatics Full timeKey Responsibilities :- Data Architecture Design : Lead the design and development of scalable, efficient, and secure data architectures using Snowflake, ensuring alignment with business objectives.- Data Integration : Utilize Fivetran for seamless data extraction, transformation, and loading (ETL) processes, ensuring that data from various sources is...
-
UsefulBI Corporation
4 weeks ago
Bangalore/Lucknow/Anywhere in India/Multiple Locations, IN Useful BI Corporation Full timeUsefulBI is looking for an 4+ years experienced DevOps Engineer with strong analytics skills to work with our team. This is key role in delivering high impact projects across financial, business intelligence with a focus on cloud-based solutions. The scope would be to a Perform server and chassis builds and configuration, and network switch configuration per...
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Innotatz IT Solutions Full timeJob Description :We are looking for candidates who can join immediately. InnotatzIT Solutions is a global Software Development, IT Product Design and IT Consulting Company predominantly focused on Web Application Development, Digital Innovations & IT Recruitment Solutions. We also provide global digital consultancy that helps brands leverage design,...
-
AWS Cloud Engineer
2 months ago
Anywhere in India/Multiple Locations, IN SMARTWORK IT SERVICES Full timeKey Responsibilities :Cloud Architecture Design : - Design and implement scalable, secure, and high-performance cloud solutions using AWS services and best practices.Infrastructure Management : - Manage and monitor AWS cloud infrastructure, including compute, storage, networking, and database resources.- Ensure optimal performance and...