Current jobs related to Senior AWS Data Engineer - ChennaiBangaloreHyderabadPune - ProgressX

  • Senior Data Engineer

    1 month ago

    chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer

    3 weeks ago

    Chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Data Architect with AWS Expertise

    4 weeks ago

    Chennai, Tamil Nadu, India Wavicle Data Solutions Full time

    Job SummaryWe are seeking an experienced Senior AWS Data Engineer to lead our data engineering team at Wavicle Data Solutions. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS.Key ResponsibilitiesDesign and develop robust ETL pipelines on AWS using services like Glue,...

  • AWS Senior Data Engineer

    1 month ago

    Hyderabad, India I square soft Full time

    Job Title: AWS Senior Data EngineerAbout the Role:I Square Soft is seeking an experienced AWS Senior Data Engineer to lead the design, development, and deployment of large-scale data pipelines and architectures. The successful candidate will have a strong background in data engineering principles, data warehousing, and data modeling, with expertise in AWS...

  • Senior Data Engineer

    1 month ago

    Chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer

    1 month ago

    Chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer

    1 month ago

    chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer

    1 month ago

    Chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer AWS

    4 weeks ago

    Chennai, Tamil Nadu, India Fulcrum Digital Full time

    About the RoleFulcrum Digital is a cutting-edge digital transformation company providing innovative technology services to various industries. We are seeking a skilled Senior Data Engineer with expertise in AWS cloud technologies and data engineering.The ideal candidate will be proficient in a wide range of AWS services and tools, including Databricks, MWAA,...

  • Senior AWS Data Engineer

    1 month ago

    Hyderabad, India Manuh Technologies Full time

    About the RoleThe Senior AWS Data Engineer will lead the design and implementation of scalable, secure, and efficient data architectures on the AWS platform. This role requires a strong background in data engineering, cloud computing, and AWS services. The successful candidate will work closely with the development team to build, deploy, and maintain data...

  • Senior Data Engineer

    6 months ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior AWS Data Engineering

    1 week ago

    Pune, India Manresa Technologies Private Limited Full time

    Location: PuneTotal Experience Required: 5 YearsNotice Period: 15 Days MaximumRole Overview: We are looking for a Senior AWS Data Engineer with a strong background in Python, Apache Spark or PySpark, and SQL. The ideal candidate will have experience working with AWS cloud platforms, particularly with AWS Glue or AWS EMR, and Athena. The role involves working...

  • Data Engineer

    1 month ago

    Chennai/Bangalore/Pune/Hyderabad/Noida, IN MNR Solutions Full time

    Job Description :We are looking for an experienced AWS Data Engineer to join our team. The ideal candidate will have extensive knowledge of AWS data services and a strong background in data engineering. This role involves designing, implementing, and maintaining data pipelines and architectures to support business intelligence and analytics initiatives.Key...

  • AWS Cloud Data Engineer

    1 month ago

    Bangalore/Chennai/Hyderabad/Pune, India MNR Solutions Full time

    Job Title: AWS Data EngineerWe are seeking an experienced AWS Data Engineer to design, develop, and optimize data pipelines and ETL processes on AWS.Key Responsibilities:Design and develop data pipelines and ETL processes on AWS.Manage and maintain AWS RDS databases for smooth data operations.Leverage AWS services to manage, analyze, and visualize large...

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India Kloud9 Full time

    Job Description : As a Senior Data Engineer, you will partner with Software Engineers, Data Scientists and Business Intelligence Engineers. You will gain a deep understanding of our services and the data they produce and become our resident expert in how to transform that data into a format that is useful for analytics and business intelligence. You will...

  • Senior Data Engineer

    8 hours ago

    Bangalore, India Kloud9 Full time

    Job Description : As a Senior Data Engineer, you will partner with Software Engineers, Data Scientists and Business Intelligence Engineers. You will gain a deep understanding of our services and the data they produce and become our resident expert in how to transform that data into a format that is useful for analytics and business intelligence. You will...

  • AWS Data Engineer

    2 months ago

    Bangalore/Chennai/Hyderabad/Pune, IN MNR Solutions Full time

    Job Description :We are looking for an experienced AWS Data Engineer to join our dynamic team. The ideal candidate will have over 9 years of experience and strong expertise in AWS services, especially AWS RDS, data formats like JSON, CSV, XML, and hands-on experience in QlikSense and SQL.Key Responsibilities :- Design, develop, and optimize data pipelines...

  • AWS Data Engineer

    8 hours ago

    Bangalore/Chennai/Hyderabad/Pune, India MNR Solutions Full time

    Job Description : We are looking for an experienced AWS Data Engineer to join our dynamic team. The ideal candidate will have over 9 years of experience and strong expertise in AWS services, especially AWS RDS, data formats like JSON, CSV, XML, and hands-on experience in QlikSense and SQL.Key Responsibilities :- Design, develop, and optimize data pipelines...

  • Senior Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India Wavicle Data Solutions Full time

    We are seeking an Experienced AWS Data Architect with a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. The ideal candidate will have a deep understanding of AWS services, data engineering best practices, and cloud architecture.Key Responsibilities:Design, develop, and maintain robust ETL pipelines on...

  • Senior Cloud Data Engineer

    6 months ago

    Hyderabad/Bangalore/Chennai/Vishakhapatnam/Vizag, IN Mouri Tech Ltd Full time

    Senior Cloud Data Engineer (AWS, Python, Pyspark) Job Title : Senior AWS Cloud Data EngineerWork Location : Hyderabad/Kolhapur/Bangalore/Chennai Experience : 6 to 9 yearsRequired Skills : Python, Pyspark, AWS services such as DMS, SAM, Glue, Redshift, Lamda S3 with Apache Airflow, CI/CD, :- Minimum of 6-9 Experience with the core AWS Services and setting up...

Senior AWS Data Engineer

1 month ago

ChennaiBangaloreHyderabadPune, India ProgressX Full time

Job Title: AWS Data Engineer

Job Summary:

We are seeking a highly skilled AWS Data Engineer to join our team at ProgressX. As a key member of our data engineering team, you will be responsible for designing and building scalable, secure, and efficient data pipelines on AWS.

Key Responsibilities:

  • Design, build, and unit test applications on Spark framework on Python.
  • Build Python and PySpark-based applications based on data in both relational databases (e.g., Oracle), NoSQL databases (e.g., DynamoDB, MongoDB), and filesystems (e.g., S3, HDFS).
  • Build AWS Lambda functions on Python runtime leveraging pandas, json, boto3, requests, avro libraries.
  • Build PySpark-based data pipeline jobs on AWS Glue ETL requiring in-depth knowledge on AWS Glue Dynamic Frames and Options.
  • Build Python-based event-driven integration with Kafka Topics, leveraging Confluent Kafka libraries.
  • Design and build generic, reusable utility applications in Python.
  • Build the Python programs across Glue ETL jobs and Lambda functions.
  • Optimize performance for data access requirements by choosing the appropriate native Hadoop file formats (Avro, Parquet, ORC etc) and compression codec respectively.
  • Design and build S3 buckets, tiers, lifecycle policies, as strategic storage layer for 'Data Lake'.
  • Optimize performance of Spark applications in Hadoop using configurations around Spark Context, Spark SQL, Data Frame, and Pair RDD's.
  • Setup the Glue crawlers in order to catalog OracleDB tables, MongoDB collections, and S3 objects.
  • Configure Athena tables and SQL views based on Glue Cataloged datasets.
  • Ability to monitor, troubleshoot, and debug failures using AWS CloudWatch and Datadog.
  • Ability to solve complex data-driven scenarios and triage towards defects and production issues.
  • Participate in code release and production deployment.

Qualifications:

  • Bachelor's Degree or equivalent in computer science or related and minimum 5+ years of experience.
  • Certified on one of - Solution Architect, Data Engineer or Data Analytics Specialty by AWS.
  • Require 3+ hand-on experience on Python and PySpark programming.
  • Require 2+ hands-on experience on AWS S3, Glue ETL & Catalog, Lambda Functions, Athena & Kafka.
  • Require 1+ hands-on experience on Confluent Kafka integration.
  • Require hands-on experience working on different file formats i.e. avro, parquet, orc, json, xml.
  • Require hands-on experience on Python pandas, requests, boto3 module.
  • Require hands-on experience in writing complex SQL queries.
  • Require hands-on experience using REST API.