[Urgent] AWS Data Engineer

3 days ago

Noida India DoctusTech Full time

Job Description

Position: AWS Data Engineer

Experience: 57 years

Domain Preference: Healthcare data engineering

About the Role

We are seeking a highly skilled AWS Data Engineer with 57 years of experience to join our data engineering team. The ideal candidate will design, build, and maintain robust, secure, and scalable data pipelines on AWS. Prior experience in healthcare data projects (EHR/CCD/HL7/FHIR, claims, HCC) is preferred.

You will be responsible for creating and enabling Medallion Architecture (Bronze -Silver-Gold layers), optimizing large-scale data processing, ensuring compliance (HIPAA, SOC II), and supporting downstream analytics and other initiatives.

Key Responsibilities

- Design, implement, and maintain ETL/ELT pipelines for ingesting data from multiple sources (APIs, EHRs, Snowflake, CSV, FHIR APIs, etc).
- Build and optimize data lakes and warehouses using AWS services (Glue, Athena, Redshift, S3).
- Implement Medallion Architecture (Bronze/Silver/Gold layers) for data standardization and enrichment.
- Develop on-demand and scheduled data ingestion workflows to ensure flexibility for both real-time and batch needs.
- Work with Airflow and PySpark to orchestrate large-scale data processing pipelines.
- Build observability and monitoring frameworks to track pipeline health, performance, and data quality.
- Implement error monitoring and notification systems (SNS, ELK, CloudWatch, Sentry) to detect and resolve issues proactively.
- Develop dashboards to visualize pipeline health, ingestion metrics, and error trends, enabling quick identification and resolution of issues.
- Ensure data quality, lineage, and integrity through automated monitoring and validation frameworks.
- Implement security best practices (encryption, KMS, Secrets Manager) for sensitive healthcare data.
- Collaborate with the product team and stakeholders to support analytics and other initiatives.
- Automate workflows with CI/CD pipelines.
- Support compliance with HIPAA, SOC II, and healthcare-specific regulatory requirements.
- Ensure smooth deployment of pipelines into production while maintaining separate development environments for testing, validation, and continuous improvements.

Required Skills & Qualifications

- 57 years of experience as a Data Engineer with a strong AWS background.
- Expertise in AWS services:
- Glue (ETL), Athena, Redshift, S3, Step Functions, Lambda
- Kafka (streaming ingestion)
- CloudWatch / CloudTrail / SNS (monitoring, auditing, notifications)
- Strong hands-on experience with PySpark and Airflow.
- Hands-on experience with modern file formats such as Parquet and Avro for efficient data storage and processing.
- Strong knowledge of data modeling (star/snowflake schema, healthcare ontologies a plus).
- Hands-on experience with Medallion Architecture and best practices for scalable data lakes.
- Experience with observability frameworks (Prometheus, Grafana, AWS CloudWatch dashboards).
- Proven experience with error monitoring, notifications, and self-healing pipelines.
- Proficiency in Python for data engineering and automation.
- Experience with healthcare data standards (FHIR, HL7, CCD, claims, ICD-10, CPT, HCC).
- Good knowledge of SQL and performance optimization in Redshift/Athena.
- Experience deploying and managing data pipelines across multiple environments (dev, staging, prod), with best practices for CI/CD, version control, and environment isolation.

  • Aws Data Engineer

    1 week ago

    India Adastra Corp Full time

    As an AWS Data Engineer, you will be responsible for: Technical requirements gathering and development of functional specificationsCommunicating with technical contacts regarding incoming and outgoing data interfacesImplementing the data pipelinesDevelop and implement best practices for AWS applications: Data Integration, Analytics & ReportingCreate ETL jobs...

  • Aws data engineer

    6 days ago

    Noida, India Coforge Full time

    Role - AWS Data Engineer Skills - AWS & Py Spark Job Locations - Pune, Greater Noida & Hyderabad Experience - 6+ years We at Coforge are hiring AWS Data Engineers with the following skill-set: Design, develop, and maintain robust ETL/ELT pipelines using tools like Apache Spark, Airflow, or similar. Build and optimize data architectures (data lakes,...

  • AWS Data Engineer

    2 weeks ago

    Noida, India Coforge Full time

    Role - AWS Data Engineer Skills - AWS & PySpark Job Locations - Pune, Greater Noida & Hyderabad Experience - 6+ years We at Coforge are hiring AWS Data Engineers with the following skill-set: - Design, develop, and maintain robust ETL/ELT pipelines using tools like Apache Spark, Airflow, or similar. - Build and optimize data architectures (data lakes,...

  • AWS Data Engineer

    2 weeks ago

    Noida, India Coforge Full time

    Role - AWS Data EngineerSkills - AWS & PySparkJob Locations - Pune, Greater Noida & HyderabadExperience - 6+ yearsWe at Coforge are hiring AWS Data Engineers with the following skill-set:Design, develop, and maintain robust ETL/ELT pipelines using tools like Apache Spark, Airflow, or similar.Build and optimize data architectures (data lakes, data...

  • AWS Data Engineer

    3 weeks ago

    Noida, Uttar Pradesh, India Coforge Full time

    Role - AWS Data EngineerSkills - AWS & PySparkJob Locations - Pune, Greater Noida & HyderabadExperience - 6+ yearsWe at Coforge are hiring AWS Data Engineers with the following skill-set:- Design, develop, and maintain robust ETL/ELT pipelines using tools like Apache Spark, Airflow, or similar.- Build and optimize data architectures (data lakes, data...

  • AWS Data Engineer

    5 days ago

    Noida, India Coforge Full time

    Role - AWS Data EngineerSkills - AWS & PySparkJob Locations - Pune, Greater Noida & HyderabadExperience - 6+ yearsWe at Coforge are hiring AWS Data Engineers with the following skill-set:Design, develop, and maintain robust ETL/ELT pipelines using tools like Apache Spark, Airflow, or similar.Build and optimize data architectures (data lakes, data...

  • Aws data engineer

    6 days ago

    Noida, India Coforge Full time

    Role - AWS Data EngineerSkills - AWS & Py SparkJob Locations - Pune, Greater Noida & HyderabadExperience - 6+ years We at Coforge are hiring AWS Data Engineers with the following skill-set:Design, develop, and maintain robust ETL/ELT pipelines using tools like Apache Spark, Airflow, or similar.Build and optimize data architectures (data lakes, data...

  • AWS Data Engineer

    1 week ago

    Noida, India Coforge Full time

    Role - AWS Data EngineerSkills - AWS & PySparkJob Locations - Pune, Greater Noida & HyderabadExperience - 6+ yearsWe at Coforge are hiring AWS Data Engineers with the following skill-set:- Design, develop, and maintain robust ETL/ELT pipelines using tools like Apache Spark, Airflow, or similar.- Build and optimize data architectures (data lakes, data...

  • AWS Data Engineer

    5 days ago

    Noida, India Coforge Full time

    Role - AWS Data EngineerSkills - AWS & PySparkJob Locations - Pune, Greater Noida & HyderabadExperience - 6+ yearsWe at Coforge are hiring AWS Data Engineers with the following skill-set:- Design, develop, and maintain robust ETL/ELT pipelines using tools like Apache Spark, Airflow, or similar.- Build and optimize data architectures (data lakes, data...

  • AWS Data Engineer

    3 weeks ago

    Noida, Uttar Pradesh, India Coforge Full time

    Role - AWS Data Engineer Skills - AWS & PySpark Job Locations - Pune, Greater Noida & Hyderabad Experience - 6+ years We at Coforge are hiring AWS Data Engineers with the following skill-set: Design, develop, and maintain robust ETL/ELT pipelines using tools like Apache Spark, Airflow, or similar. Build and optimize data architectures (data lakes, data...