AWS DE- Airflow

1 month ago


chennai, India WILLWARE TECHNOLOGIES Full time

Job Description:


  • Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure.
  • Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance.
  • Collaborate with data scientists, analysts, and other stakeholders to understand requirements and translate them into technical solutions.
  • Implement best practices for data ingestion, processing, transformation, and storage.
  • Monitor pipeline performance, troubleshoot issues, and implement enhancements to improve efficiency and reliability.
  • Ensure data quality and integrity throughout the data lifecycle by implementing robust validation and error handling mechanisms.
  • Stay current with emerging technologies and industry trends in data engineering and contribute to continuous improvement initiatives.
  • Mentor junior team members and provide technical guidance and support as needed.



Qualification:


  • 5-7 years of professional experience in data engineering, with a focus on building and managing data pipelines.
  • Extensive hands-on experience with Apache Airflow, including DAG design, scheduling, and orchestration.
  • Strong proficiency in AWS services such as S3, EC2, EMR, Glue, Lambda, and Redshift.
  • Solid understanding of data warehousing concepts, ETL processes, and data modeling techniques.
  • Proficiency in programming languages such as Python, SQL, and Shell scripting.





  • Aws De- Airflow

    4 weeks ago


    Chennai, Tamil Nadu, India Willware Technologies Full time

    **Work mode: One week of onboarding at Chennai office and later Remote** - Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure. - Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance. - Collaborate with data scientists, analysts, and other stakeholders to understand...

  • Platform Engineer

    4 weeks ago


    chennai, India Mobile Programming LLC Full time

    Job Description : Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development...

  • Platform Engineer

    2 weeks ago


    Chennai, India Mobile Programming LLC Full time

    Job Description :Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development plans,...

  • AWS cloud Engineer

    4 weeks ago


    chennai, India Virtusa Full time

    AWS cloud Engineer - CREQ187082 Description Minimum 6-9 years of experience in AWS CloudBe responsible for the planning, implementation, and growth of the AWS cloud infrastructure.Implement continuous integration/continuous delivery (CI/CD) pipelines when necessary.Work alongside architecture and engineering teams to design and implement any scalable...

  • AWS Data Engineer

    2 weeks ago


    Bangalore/Pune/Chennai, India Change leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :- Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform.- This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • AWS Data Engineer

    3 weeks ago


    Bangalore/Pune/Chennai, IN Change leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :- Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform.- This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • AWS Data Engineer

    3 weeks ago


    Chennai, India CliqHR Full time

    Job Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...

  • AWS Data Engineer

    3 weeks ago


    Chennai, India CliqHR Full time

    Job Description : - Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...

  • AWS Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India CliqHR Full time

    Job Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...

  • Platform Engineer

    2 weeks ago


    Chennai/Anywhere in India/Multiple Locations Mobile Programming LLC Full time

    Job Description : Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development...

  • Platform Engineer

    3 weeks ago


    Chennai/Anywhere in India/Multiple Locations, IN Mobile Programming LLC Full time

    Job Description :Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development plans,...

  • AWS Data Engineer

    2 weeks ago


    Pune/Chennai/Mumbai/Bangalore/Delhi NCR/Hyderabad, India Change Leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform. This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • AWS Data Engineer

    3 weeks ago


    Pune/Chennai/Mumbai/Bangalore/Delhi NCR/Hyderabad, IN Change leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform. This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...


  • chennai, India Anicalls (Pty) Ltd Full time

    Candidate should have:Clear, concise communication abilities - writing, verbal, presentation - to all levels of technical and non-technical audiences with the ability to collaborateStrong experience in CICD technologies including Python, SQL, Airflow, and Git in a cloud settingUnderstanding of ELK stack is a plus for logging and reporting purposesExperience...

  • DevOps Engineer

    2 weeks ago


    Bangalore/Hyderabad/Pune/Chennai, India Growel Softech Pvt. Ltd. Full time

    Job Description:We are seeking two experienced Senior DevOps Engineers to join our team, with a strong focus on AWS DevOps, particularly with expertise in EKS and Dockers. The ideal candidates will have a deep understanding of AWS Cloud services, including but not limited to EC2 instances, ECS, and data-related applications. Familiarity with data tools such...

  • AWS Data Engineer

    4 weeks ago


    Chennai, India CliqHR Full time

    Mandatory Skills : AWS, Python, SQL, spark, Airflow, SnowflakeResponsibilities :- Create and manage cloud resources in AWS- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and...

  • AWS Data Engineer

    1 month ago


    Chennai, India CliqHR Full time

    Mandatory Skills : AWS, Python, SQL, spark, Airflow, SnowflakeResponsibilities :- Create and manage cloud resources in AWS- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and...

  • AWS Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India CliqHR Full time

    Mandatory Skills : AWS, Python, SQL, spark, Airflow, SnowflakeResponsibilities :- Create and manage cloud resources in AWS- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and...

  • AWS Data Engineer

    6 days ago


    Chennai, India LexisNexis Full time

    Software Engineer IIIDo you enjoy designing analytic solutions at an enterprise scale for global customers?Do you have experience in working on complex problems?About our TeamOur product analytics team focuses on using data to understand and enhance the user experience of our customer facing products. We gather and analyze user interaction data from various...


  • Hyderabad,Bangalore,Chennai,Vishakhapatnam,Vizag, India Mouri Tech (P) Ltd Full time

    Senior Cloud Data Engineer (AWS, Python, Pyspark) Job Title : Senior AWS Cloud Data EngineerWork Location : Hyderabad/Kolhapur/Bangalore/Chennai Experience : 6 to 9 yearsRequired Skills : Python, Pyspark, AWS services such as DMS, SAM, Glue, Redshift, Lamda S3 with Apache Airflow, CI/CD, : - Minimum of 6-9 Experience with the core AWS Services and setting...