AWS Data Engineer

23 hours ago

Bengaluru, Karnataka, India DoctusTech Full time ₹ 8,00,000 - ₹ 24,00,000 per year

Position:

AWS Data Engineer

Experience:

5–7 years

Domain Preference:

Healthcare data engineering

About the Role

We are seeking a highly skilled AWS Data Engineer with 5–7 years of experience to join our data engineering team. The ideal candidate will design, build, and maintain robust, secure, and scalable data pipelines on AWS. Prior experience in healthcare data projects (EHR/CCD/HL7/FHIR, claims, HCC) is preferred.

You will be responsible for creating and enabling Medallion Architecture (Bronze -Silver-Gold layers), optimizing large-scale data processing, ensuring compliance (HIPAA, SOC II), and supporting downstream analytics and other initiatives.

Key Responsibilities

  1. Design, implement, and maintain ETL/ELT pipelines for ingesting data from multiple sources (APIs, EHRs, Snowflake, CSV, FHIR APIs, etc).
  2. Build and optimize data lakes and warehouses using AWS services (Glue, Athena, Redshift, S3).
  3. Implement Medallion Architecture (Bronze/Silver/Gold layers) for data standardization and enrichment.
  4. Develop on-demand and scheduled data ingestion workflows to ensure flexibility for both real-time and batch needs.
  5. Work with Airflow and PySpark to orchestrate large-scale data processing pipelines.
  6. Build observability and monitoring frameworks to track pipeline health, performance, and data quality.
  7. Implement error monitoring and notification systems (SNS, ELK, CloudWatch, Sentry) to detect and resolve issues proactively.
  8. Develop dashboards to visualize pipeline health, ingestion metrics, and error trends, enabling quick identification and resolution of issues.
  9. Ensure data quality, lineage, and integrity through automated monitoring and validation frameworks.
  10. Implement security best practices (encryption, KMS, Secrets Manager) for sensitive healthcare data.
  11. Collaborate with the product team and stakeholders to support analytics and other initiatives.
  12. Automate workflows with CI/CD pipelines.
  13. Support compliance with HIPAA, SOC II, and healthcare-specific regulatory requirements.
  14. Ensure smooth deployment of pipelines into production while maintaining separate development environments for testing, validation, and continuous improvements.

Required Skills & Qualifications

  1. 5–7 years of experience as a Data Engineer with a strong AWS background.
  2. Expertise in AWS services:
  3. Glue (ETL), Athena, Redshift, S3, Step Functions, Lambda
  4. Kafka (streaming ingestion)
  5. CloudWatch / CloudTrail / SNS (monitoring, auditing, notifications)
  6. Strong hands-on experience with PySpark and Airflow.
  7. Hands-on experience with modern file formats such as Parquet and Avro for efficient data storage and processing.
  8. Strong knowledge of data modeling (star/snowflake schema, healthcare ontologies a plus).
  9. Hands-on experience with Medallion Architecture and best practices for scalable data lakes.
  10. Experience with observability frameworks (Prometheus, Grafana, AWS CloudWatch dashboards).
  11. Proven experience with error monitoring, notifications, and self-healing pipelines.
  12. Proficiency in Python for data engineering and automation.
  13. Experience with healthcare data standards (FHIR, HL7, CCD, claims, ICD-10, CPT, HCC).
  14. Good knowledge of SQL and performance optimization in Redshift/Athena.
  15. Experience deploying and managing data pipelines across multiple environments (dev, staging, prod), with best practices for CI/CD, version control, and environment isolation.

  • AWS Sr Data Engineer

    2 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a AWS Sr Data Engineer to join our team in Bangalore or Remote, Karnātaka (IN-KA), India (IN). Position Location:OUS with...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Migrate ETL workflows from SAP BODS to AWS Glue/dbt/Talend. Develop and maintain scalable ETL pipelines in AWS. Write PySpark scripts for large-scale data processing. Optimize SQL queries and transformations for AWS PostgreSQL. Work with Cloud Engineers to ensure smooth deployment and performance tuning. Integrate data pipelines with existing Unix systems....

  • Data Engineer with Snowflake, AWS Glue, Kafka, API

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 8,00,000 - ₹ 25,00,000 per year

    6+ years of experience in Data Engineering or related role.Hands-on experience with Snowflake (data modelling, performance tuning, query optimization, Snowpipe, Time Travel, Streams & Tasks).Strong expertise in AWS Glue for ETL job development, orchestration, and optimization.Proficiency with AWS services such as S3, Lambda, Redshift, Athena, Step Functions,...

  • Aws Data Engineer

    11 hours ago

    Bengaluru, Karnataka, India Coforge Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    AWS Data EngineerJob Location: BengaluruExperience Required: 5+ YearsMandatory Skills: AWS Services, ETL, ETL Integration, CodePipeline, Jenkins, Glue, EMR, Athena, ECS, EKS, Kubernetes, CloudWatch, Prometheus, Grafana, Python, Shell, or PowerShellJob Description:We are looking for an experienced AWS Engineer with around 5-8 years of hands-on experience in...

  • AWS Architect

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role SummaryAs a AWS Architect, you will lead the design and implementation of scalable, secure, and high-performance data solutions using AWS Platform. You will define architectural standards, guide engineering teams, and collaborate with stakeholders to align data strategies with business goals. Your role will focus on leveraging AWS capabilities to build...

  • AWS Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Info Origin Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Note: Virtual Drive on 14th and 15th OctRole OverviewWe are looking for an experienced AWS Engineer with around 5-7 years of hands-on experience in designing, deploying, and managing applications on the AWS cloud platform. The ideal candidate should have strong expertise in AWS services, automation, and cloud security, along with the ability to collaborate...

  • AWS Data Engineer

    6 days ago

    Bengaluru, Karnataka, India eSkillport Management Services Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Description Job Title: AWS Data Engineer (IoT/Streaming Data) Location: Chennai or Bangalore. Experience: 3 Years Employment Type: Full-time About the Role We are seeking a highly skilled AWS Data Engineer to design, build, and optimize large-scale data pipelines and analytics platforms. The ideal candidate will have strong...

  • Data Engineer

    2 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Job Duties:Migrate ETL workflows from SAP BODS to AWS Glue/dbt/Talend. Develop and maintain scalable ETL pipelines in AWS. Write PySpark scripts for large-scale data processing. Optimize SQL queries and transformations for AWS PostgreSQL. Work with Cloud Engineers to ensure smooth deployment and performance tuning. Integrate data pipelines with existing Unix...

  • Data Engineer with AWS

    5 days ago

    Bengaluru, Karnataka, India Luxoft Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Project description We are looking for a skilled Data Engineer with strong experience in AWS and Apache Airflow to join a dynamic data team. You will be responsible for building and maintaining scalable data pipelines, orchestrating workflows, and ensuring data quality and availability across platforms. - Responsibilities- ?Data Pipeline DevelopmentDesign,...

  • AWS Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Ubique Systems Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    ** NOTICE - Immediate to 10 days MaXX** Exp yrsAWS DATA ENGINEERResponsibility of / Expectations from the Role ::AWS data engineer to build ETL data pipelines (Glue, python based EKS ingestion pipelines) and with DevOps experience to build IaC terraform, CICD gitlab pipelines.Support FTP and ALM modules implementation.Provide hands-on support for the...