AWS Data Engineer

3 months ago

India ITI Data Full time

Job Description

We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There is no requirement for Machine Learning skills. This is a high visibility, fast-paced key initiative will integrate data across internal and external sources, provide analytical insights, and integrate with the customer’s critical systems.

Key Responsibilities

• Design, build and unit test applications on Spark framework on Python.

• Build Python and PySpark based applications based on data in both Relational databases (e.g. Oracle), NoSQL databases (e.g. DynamoDB, MongoDB) and filesystems (e.g. S3, HDFS)

• Build AWS Lambda functions on Python runtime leveraging pandas, json, boto3, requests, avro libraries

• Build PySpark based data pipeline jobs on AWS Glue ETL requiring in-depth knowledge on AWS Glue Dynamic Frames and Options

• Build Python based event-driven integration with Kafka Topics, leveraging Confluent Kafka libraries

• Design and Build Generic, Reusable utility applications in Python

• Build the Python programs across Glue ETL jobs and Lambda functions

• Optimize performance for data access requirements by choosing the appropriate native Hadoop file formats (Avro, Parquet, ORC etc) and compression codec respectively.

• Design & Build S3 buckets, tiers, lifecycle policies, as strategic storage layer for "Data Lake"

• Optimize performance of Spark applications in Hadoop using configurations around Spark Context, Spark-SQL, Data Frame, and Pair RDD's

• Setup the Glue crawlers in order to catalog OracleDB tables, MongoDB collections and S3 objects

• Configure Athena tables and SQL views based on Glue Cataloged datasets

• Ability to monitor, troubleshoot and debug failures using AWS CloudWatch and Datadog

• Ability to solve complex data-driven scenarios and triage towards defects and production issues

• Participate in code release and production deployment.

  • AWS Data Engineer

    2 months ago

    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    3 months ago

    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    2 weeks ago

    india ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    2 weeks ago

    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • Aws data engineer

    1 month ago

    India FLYONIT Full time

    Job Summary:   We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for...

  • AWS Data Engineer

    1 month ago

    india FLYONIT Full time

    Job Summary: We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing...

  • Data engineer

    1 month ago

    India NucleusTeq Full time

    Company Description Nucleus Teq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our...

  • Data Engineer

    1 month ago

    India NucleusTeq Full time

    Company Description NucleusTeq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our ability to...

  • Data Engineer

    1 month ago

    india NucleusTeq Full time

    Company DescriptionNucleusTeq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our ability to...

  • AWS Data Engineer

    2 months ago

    India FLYONIT Full time

    Job Summary:   We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for...

  • AWS Data Engineer

    2 months ago

    India FLYONIT Full time

    Job Summary: We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing...

  • Data Engineer

    3 weeks ago

    india Adastra Full time

    Adastra India is a part of Adastra Group. The company was established in 2024 as an extension of Adastra Global Delivery in the APAC region. The company is focused on attracting, upskilling and utilizing local talent for projects delivered by Adastra in Europe and North America. The company has a local office in Pune hiring consultants locally and remotely...

  • Data Engineer

    3 weeks ago

    India Adastra Full time

    Adastra India is a part of Adastra Group. The company was established in 2024 as an extension of Adastra Global Delivery in the APAC region. The company is focused on attracting, upskilling and utilizing local talent for projects delivered by Adastra in Europe and North America. The company has a local office in Pune hiring consultants locally and remotely...

  • Data Engineer

    3 weeks ago

    india Adastra Full time

    Adastra India is a part of Adastra Group. The company was established in 2024 as an extension of Adastra Global Delivery in the APAC region. The company is focused on attracting, upskilling and utilizing local talent for projects delivered by Adastra in Europe and North America. The company has a local office in Pune hiring consultants locally and remotely...

  • Aws data engineer

    1 week ago

    India Recro Full time

    Job Title: AWS Data Engineer Location - Remote Experience - 3+ years Note- Looking for candidates who have experience working from the scratch and are currently serving their notice period, with availability to join within 10-15 days. We are seeking a Data Engineer with strong hands-on experience in data engineering techniques to create data...

  • AWS Data Engineer

    1 month ago

    india Programmers.io Full time

    AWS Data Engineer - MarketingKey Tasks:Analyze existing marketing data/tables and summarize data. Collaborating with existing DE team to ingest additional data sources for marketingDeveloping data workflows to process, clean & link / unify consumer data sourcesStrong in SQL and has experience in implementing data quality checksIngesting unified / clean data...

  • Aws data engineer

    1 month ago

    India Programmers.io Full time

    AWS Data Engineer - Marketing Key Tasks: Analyze existing marketing data/tables and summarize data. Collaborating with existing DE team to ingest additional data sources for marketing Developing data workflows to process, clean & link / unify consumer data sources Strong in SQL and has experience in implementing data quality checks Ingesting unified...

  • Data Engineer

    1 month ago

    India NucleusTeq Full time

    Company DescriptionNucleusTeq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our ability to...

  • Data Engineer

    1 month ago

    india NucleusTeq Full time

    Company Description NucleusTeq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our ability to...

  • AWS Data Engineer

    2 months ago

    india Programmers.io Full time

    AWS Data Engineer - Marketing Key Tasks: Analyze existing marketing data/tables and summarize data. Collaborating with existing DE team to ingest additional data sources for marketing Developing data workflows to process, clean & link / unify consumer data sources Strong in SQL and has experience in implementing data quality checks Ingesting unified /...