AWS Data Engineer

4 months ago

India ITI Data Full time

Job Description

We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There is no requirement for Machine Learning skills. This is a high visibility, fast-paced key initiative will integrate data across internal and external sources, provide analytical insights, and integrate with the customer’s critical systems.

Key Responsibilities

• Design, build and unit test applications on Spark framework on Python.

• Build Python and PySpark based applications based on data in both Relational databases (e.g. Oracle), NoSQL databases (e.g. DynamoDB, MongoDB) and filesystems (e.g. S3, HDFS)

• Build AWS Lambda functions on Python runtime leveraging pandas, json, boto3, requests, avro libraries

• Build PySpark based data pipeline jobs on AWS Glue ETL requiring in-depth knowledge on AWS Glue Dynamic Frames and Options

• Build Python based event-driven integration with Kafka Topics, leveraging Confluent Kafka libraries

• Design and Build Generic, Reusable utility applications in Python

• Build the Python programs across Glue ETL jobs and Lambda functions

• Optimize performance for data access requirements by choosing the appropriate native Hadoop file formats (Avro, Parquet, ORC etc) and compression codec respectively.

• Design & Build S3 buckets, tiers, lifecycle policies, as strategic storage layer for "Data Lake"

• Optimize performance of Spark applications in Hadoop using configurations around Spark Context, Spark-SQL, Data Frame, and Pair RDD's

• Setup the Glue crawlers in order to catalog OracleDB tables, MongoDB collections and S3 objects

• Configure Athena tables and SQL views based on Glue Cataloged datasets

• Ability to monitor, troubleshoot and debug failures using AWS CloudWatch and Datadog

• Ability to solve complex data-driven scenarios and triage towards defects and production issues

• Participate in code release and production deployment.

  • Aws data engineer

    2 weeks ago

    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on Py Spark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    3 months ago

    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    4 months ago

    India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...

  • AWS Data Engineer

    1 month ago

    india ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    4 weeks ago

    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • Data Warehouse Engineer | Data Warehouse Engineer | india

    2 weeks ago

    india Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.

  • Data Warehouse Engineer

    2 weeks ago

    India Data Warehouse Engineer Full time

    Experience : 2- 5 Years Primary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills. Good to have skills: Power BI, Databricks, Python.

  • Data Warehouse Expert for Analytics Projects

    2 weeks ago

    India Data Warehouse Engineer Full time

    About the RoleWe are seeking an experienced Data Warehouse Engineer to join our team. In this role, you will design, develop, and maintain data warehouses that support business intelligence and analytics initiatives.

  • Aws data engineer

    2 months ago

    India FLYONIT Full time

    Job Summary:   We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for...

  • AWS Data Engineer

    2 months ago

    india FLYONIT Full time

    Job Summary: We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing...

  • AWS Data Engineer

    3 months ago

    India FLYONIT Full time

    Job Summary:   We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for...

  • AWS Data Engineer

    3 months ago

    India FLYONIT Full time

    Job Summary: We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing...

  • Aws data engineer

    4 weeks ago

    India Recro Full time

    Job Title: AWS Data Engineer Location - Remote Experience - 3+ years Note- Looking for candidates who have experience working from the scratch and are currently serving their notice period, with availability to join within 10-15 days. We are seeking a Data Engineer with strong hands-on experience in data engineering techniques to create data...

  • AWS Data Engineer

    7 days ago

    India VicTree Solutions Full time

    Job Title: AWS Data Engineer (exp: 5+ years) Job Location: Pune, Maharashtra Contract Role Job Brief: We are seeking a skilled AWS Data Engineer to join our team. The ideal candidate will have a strong background in building, managing, and optimizing large-scale data pipelines and data lakes on AWS. This role requires hands- on experience with...

  • Cloud Data Engineering Lead

    1 week ago

    India ITI Data Full time

    ITI Data is seeking a highly skilled Cloud Data Engineering Lead to spearhead the design and development of scalable data pipelines on AWS. With a strong background in PySpark, Python, and AWS services, you will be responsible for building robust data integration solutions that enable data-driven insights across various internal and external sources.This is...

  • AWS Data Engineer

    4 weeks ago

    India Recro Full time

    Job Title: AWS Data Engineer Location - Remote Experience - 3+ years Note- Looking for candidates who have experience working from the scratch and are currently serving their notice period, with availability to join within 10-15 days. We are seeking a Data Engineer with strong hands-on experience in data engineering techniques to create data pipelines...

  • AWS Data Engineer

    4 weeks ago

    India Recro Full time

    Job Title: AWS Data EngineerLocation - Remote Experience - 3+ years Note- Looking for candidates who have experience working from the scratch and are currently serving their notice period, with availability to join within 10-15 days.We are seeking a Data Engineer with strong hands-on experience in data engineering techniques to create data pipelines and...

  • Lead data engineer

    3 weeks ago

    India Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...

  • AWS Data Engineer

    2 weeks ago

    India Recro Full time

    Job Title: Data Engineer Experience : 5+ years Location : Remote Job Type : Full-time Role Overview: We are seeking an experienced Data Engineer to develop and optimize data pipelines and manage cloud infrastructure. The ideal candidate has strong Python, Pyspark, and AWS skills, with a focus on performance optimization and clean, maintainable code. Key...

  • AWS Data Engineer

    2 weeks ago

    India Recro Full time

    Job Title: Data EngineerExperience: 5+ yearsLocation: RemoteJob Type: Full-timeRole Overview:We are seeking an experienced Data Engineer to develop and optimize data pipelines and manage cloud infrastructure. The ideal candidate has strong Python, Pyspark, and AWS skills, with a focus on performance optimization and clean, maintainable code.Key...