AWS Data Engineer

1 day ago

Bengaluru, Karnataka, India DoctusTech Full time ₹ 8,00,000 - ₹ 24,00,000 per year

Position:

AWS Data Engineer

Experience:

5–7 years

Domain Preference:

Healthcare data engineering

About the Role

We are seeking a highly skilled AWS Data Engineer with 5–7 years of experience to join our data engineering team. The ideal candidate will design, build, and maintain robust, secure, and scalable data pipelines on AWS. Prior experience in healthcare data projects (EHR/CCD/HL7/FHIR, claims, HCC) is preferred.

You will be responsible for creating and enabling Medallion Architecture (Bronze -Silver-Gold layers), optimizing large-scale data processing, ensuring compliance (HIPAA, SOC II), and supporting downstream analytics and other initiatives.

Key Responsibilities

  1. Design, implement, and maintain ETL/ELT pipelines for ingesting data from multiple sources (APIs, EHRs, Snowflake, CSV, FHIR APIs, etc).
  2. Build and optimize data lakes and warehouses using AWS services (Glue, Athena, Redshift, S3).
  3. Implement Medallion Architecture (Bronze/Silver/Gold layers) for data standardization and enrichment.
  4. Develop on-demand and scheduled data ingestion workflows to ensure flexibility for both real-time and batch needs.
  5. Work with Airflow and PySpark to orchestrate large-scale data processing pipelines.
  6. Build observability and monitoring frameworks to track pipeline health, performance, and data quality.
  7. Implement error monitoring and notification systems (SNS, ELK, CloudWatch, Sentry) to detect and resolve issues proactively.
  8. Develop dashboards to visualize pipeline health, ingestion metrics, and error trends, enabling quick identification and resolution of issues.
  9. Ensure data quality, lineage, and integrity through automated monitoring and validation frameworks.
  10. Implement security best practices (encryption, KMS, Secrets Manager) for sensitive healthcare data.
  11. Collaborate with the product team and stakeholders to support analytics and other initiatives.
  12. Automate workflows with CI/CD pipelines.
  13. Support compliance with HIPAA, SOC II, and healthcare-specific regulatory requirements.
  14. Ensure smooth deployment of pipelines into production while maintaining separate development environments for testing, validation, and continuous improvements.

Required Skills & Qualifications

  1. 5–7 years of experience as a Data Engineer with a strong AWS background.
  2. Expertise in AWS services:
  3. Glue (ETL), Athena, Redshift, S3, Step Functions, Lambda
  4. Kafka (streaming ingestion)
  5. CloudWatch / CloudTrail / SNS (monitoring, auditing, notifications)
  6. Strong hands-on experience with PySpark and Airflow.
  7. Hands-on experience with modern file formats such as Parquet and Avro for efficient data storage and processing.
  8. Strong knowledge of data modeling (star/snowflake schema, healthcare ontologies a plus).
  9. Hands-on experience with Medallion Architecture and best practices for scalable data lakes.
  10. Experience with observability frameworks (Prometheus, Grafana, AWS CloudWatch dashboards).
  11. Proven experience with error monitoring, notifications, and self-healing pipelines.
  12. Proficiency in Python for data engineering and automation.
  13. Experience with healthcare data standards (FHIR, HL7, CCD, claims, ICD-10, CPT, HCC).
  14. Good knowledge of SQL and performance optimization in Redshift/Athena.
  15. Experience deploying and managing data pipelines across multiple environments (dev, staging, prod), with best practices for CI/CD, version control, and environment isolation.

  • AWS Sr Data Engineer

    21 hours ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a AWS Sr Data Engineer to join our team in Bangalore or Remote, Karnātaka (IN-KA), India (IN). Position Location:OUS with...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from Tata Consultancy ServicesWe are hiring AWS Data Engineer Position: AWS Data EngineerJob Location: BangaloreExperience : 10-14 yearsInterested professionals kindly apply through the link.Must Have:- Prior Principal Engineer experience, leading team best practices in design, development, and implementation, mentoring team members, and fostering...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from Tata Consultancy Services We are hiring AWS Data Engineer Position: AWS Data Engineer Job Location: Bangalore Experience : 10-14 years Interested professionals kindly apply through the link. Must Have: Prior Principal Engineer experience, leading team best practices in design, development, and implementation, mentoring team members, and...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Acme Services Full time

    We are seeking an AWS Data Engineer skilled in AWS, Python, PySpark, Glue, S3, Lambda, and other AWS data services. The role involves designing and implementing scalable data pipelines, performing ETL processes, and optimizing data workflows on AWS. Candidates must have hands-on experience with cloud-based data solutions, strong coding skills in Python, and...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Acme Services Full time

    We are seeking an AWS Data Engineer skilled in AWS, Python, PySpark, Glue, S3, Lambda, and other AWS data services. The role involves designing and implementing scalable data pipelines, performing ETL processes, and optimizing data workflows on AWS. Candidates must have hands-on experience with cloud-based data solutions, strong coding skills in Python, and...

  • Aws data engineer

    2 weeks ago

    Bengaluru, Karnataka, India Acme Services Full time

    We are seeking an AWS Data Engineer skilled in AWS, Python, Py Spark, Glue, S3, Lambda, and other AWS data services. The role involves designing and implementing scalable data pipelines, performing ETL processes, and optimizing data workflows on AWS. Candidates must have hands-on experience with cloud-based data solutions, strong coding skills in Python, and...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from Tata Consultancy ServicesWe are hiring AWS Data Engineer Position: AWS Data EngineerJob Location: BangaloreExperience : 10-14 yearsInterested professionals kindly apply through the link.Must Have:Prior Principal Engineer experience, leading team best practices in design, development, and implementation, mentoring team members, and fostering a...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Teamware Solutions Full time

    About the CompanyGreetings from Teamware Solutions a division of Quantum Leap Consulting Pvt. LtdAbout the RoleWe are hiring an AWS Data Engineer- Location: Bangalore- Work Model: Hybrid- Experience: 5-7 years- Notice Period: Immediate- 15 days- Open roles: 22Data Engineer:Data Engineer, DevOps, Data Modeller, QA, Business systems analyst, Data Analyst,...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Teamware Solutions Full time

    About the Company Greetings from Teamware Solutions a division of Quantum Leap Consulting Pvt. Ltd About the Role We are hiring an AWS Data Engineer Location: Bangalore Work Model: Hybrid Experience: 5-7 years Notice Period: Immediate- 15 days Open roles: 22 Data Engineer: Data Engineer, DevOps, Data Modeller, QA, Business systems analyst, Data...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India L&T Technology Services Full time

    AWS Data Engineer:Exp: 8-10 yrsLocation: BangaloreSkills:1) Graduate with 8-10 years of experience working as a AWS Data Engineer2) Excellent knowledge of PySpark, JavaScript, AWS - RDS, and Redshift