AWS Data Engineer

1 month ago

india GCC client Full time

To drive the successful migration of on-premise Hadoop big data infrastructure to AWS.

This includes data migration, optimization, and integration tasks to ensure robust and scalable data solutions.

 

Responsibilities


  • Design and implement data migration strategies from on-premise Hadoop to AWS.
  • Develop and optimize Spark jobs for data processing on AWS EMR.
  • Manage and maintain S3 storage solutions for data lake architecture.
  • Configure and manage AWS Glue for ETL processes and data cataloging.
  • Ensure data quality and integrity throughout the migration process.
  • Collaborate with stakeholders to understand and document data requirements.
  • Perform data mapping and transformation tasks as part of the migration strategy.
  • Implement and manage AWS Lake Formation for data governance and security.
  • Troubleshoot and resolve data migration issues and performance bottlenecks.
  • Monitor and optimize data workflows for performance and cost efficiency.
  • Develop and maintain documentation for data processes and architecture.
  • Coordinate with cross-functional teams to align migration activities with project goals.
  • Provide technical guidance and support to junior team members.
  • Conduct performance tuning and optimization of data processing tasks.
  • Ensure compliance with data security and privacy policies.
  • Design and implement automated data pipelines using AWS services.
  • Perform regular data validation and verification checks.
  • Collaborate with AWS support and engineering teams as needed.
  • Stay up-to-date with the latest developments in big data and cloud technologies.
  • Assist in training and knowledge transfer activities for team members.


Qualifications

  • Bachelor’s or Master’s degree in Engineering, Computer Science, or a related field
  • 6-10 years of experience with the following technologies: Spark, Oracle 11g, Hive, HDFS, Hadoop, AWS services.


Primary skill set

  • Expertise in Apache Spark and AWS EMR.
  • Strong experience with AWS services such as S3, Glue, and Lake Formation.
  • Proficiency in HDFS, Hive, and data migration strategies.
  • Solid understanding of data processing and optimization techniques.

  • AWS Data Engineer

    3 months ago

    India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...

  • AWS Data Engineer

    2 months ago

    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    4 days ago

    india ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • Aws data engineer

    3 weeks ago

    India FLYONIT Full time

    Job Summary:   We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for...

  • AWS Data Engineer

    3 weeks ago

    india FLYONIT Full time

    Job Summary: We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing...

  • AWS Data Engineer

    3 weeks ago

    India Recro Full time

    AWS Data EngineerAt Recro, we're seeking a highly skilled AWS Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data pipelines using AWS services.Key Responsibilities:Design and implement data pipelines using AWS services such as AWS Glue,...

  • Data engineer

    3 weeks ago

    India NucleusTeq Full time

    Company Description Nucleus Teq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our...

  • Data Engineer

    3 weeks ago

    India NucleusTeq Full time

    Company Description NucleusTeq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our ability to...

  • Data Engineer

    3 weeks ago

    india NucleusTeq Full time

    Company DescriptionNucleusTeq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our ability to...

  • AWS Data Engineer

    3 weeks ago

    India Recro Full time

    Job Title: AWS Data EngineerLocation: RemoteExperience: 3-5 YearsJob Summary:We are seeking an experienced AWS Data Engineer to join our team at Recro. The ideal candidate will have a strong background in data engineering and AWS services, with a minimum of 3+ years of experience building and maintaining scalable data pipelines.Key Responsibilities:Design,...

  • AWS Data Engineer

    2 months ago

    India FLYONIT Full time

    Job Summary: We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing...

  • AWS Data Engineer

    2 months ago

    India FLYONIT Full time

    Job Summary:   We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for...

  • Data Engineer

    6 days ago

    india Adastra Full time

    Adastra India is a part of Adastra Group. The company was established in 2024 as an extension of Adastra Global Delivery in the APAC region. The company is focused on attracting, upskilling and utilizing local talent for projects delivered by Adastra in Europe and North America. The company has a local office in Pune hiring consultants locally and remotely...

  • Data Engineer

    6 days ago

    India Adastra Full time

    Adastra India is a part of Adastra Group. The company was established in 2024 as an extension of Adastra Global Delivery in the APAC region. The company is focused on attracting, upskilling and utilizing local talent for projects delivered by Adastra in Europe and North America. The company has a local office in Pune hiring consultants locally and remotely...

  • Data Engineer

    5 days ago

    india Adastra Full time

    Adastra India is a part of Adastra Group. The company was established in 2024 as an extension of Adastra Global Delivery in the APAC region. The company is focused on attracting, upskilling and utilizing local talent for projects delivered by Adastra in Europe and North America. The company has a local office in Pune hiring consultants locally and remotely...

  • AWS Data Engineer

    3 weeks ago

    india Programmers.io Full time

    AWS Data Engineer - MarketingKey Tasks:Analyze existing marketing data/tables and summarize data. Collaborating with existing DE team to ingest additional data sources for marketingDeveloping data workflows to process, clean & link / unify consumer data sourcesStrong in SQL and has experience in implementing data quality checksIngesting unified / clean data...

  • Aws data engineer

    3 weeks ago

    India Programmers.io Full time

    AWS Data Engineer - Marketing Key Tasks: Analyze existing marketing data/tables and summarize data. Collaborating with existing DE team to ingest additional data sources for marketing Developing data workflows to process, clean & link / unify consumer data sources Strong in SQL and has experience in implementing data quality checks Ingesting unified...

  • AWS Data Engineer

    3 weeks ago

    India Recro Full time

    Job Title: AWS Data Engineer Location - Remote Experience - 3+ Experience We are seeking a AWS Data Engineer with strong hands-on experience in data engineering techniques to create data pipelines and build data assets. The ideal candidate must possess deep expertise in PySpark, Python, SQL, and AWS services (Glue, S3, Redshift). The role...

  • AWS Data Engineer

    1 month ago

    india GCC client Full time

    To drive the successful migration of on-premise Hadoop big data infrastructure to AWS. This includes data migration, optimization, and integration tasks to ensure robust and scalable data solutions.   Responsibilities Design and implement data migration strategies from on-premise Hadoop to AWS. Develop and optimize Spark jobs for data processing on AWS...

  • Data Engineer

    4 weeks ago

    india NucleusTeq Full time

    Company Description NucleusTeq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our ability to...