Aws De- Airflow

6 months ago


Chennai Tamil Nadu, India Willware Technologies Full time

**Work mode: One week of onboarding at Chennai office and later Remote**
- Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure.
- Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance.
- Collaborate with data scientists, analysts, and other stakeholders to understand requirements and translate them into technical solutions.
- Implement best practices for data ingestion, processing, transformation, and storage.
- Monitor pipeline performance, troubleshoot issues, and implement enhancements to improve efficiency and reliability.
- Ensure data quality and integrity throughout the data lifecycle by implementing robust validation and error handling mechanisms.
- Stay current with emerging technologies and industry trends in data engineering and contribute to continuous improvement initiatives.
- Mentor junior team members and provide technical guidance and support as needed.

**Qualification**:

- 5-7 years of professional experience in data engineering, with a focus on building and managing data pipelines.
- Extensive hands-on experience with Apache Airflow, including DAG design, scheduling, and orchestration.
- Strong proficiency in AWS services such as S3, EC2, EMR, Glue, Lambda, and Redshift.
- Solid understanding of data warehousing concepts, ETL processes, and data modeling techniques.
- Proficiency in programming languages such as Python, SQL, and Shell scripting.

**Job Type**: Temporary
Contract length: 3-6 months

**Benefits**:

- Work from home

Schedule:

- Monday to Friday

**Experience**:

- total work: 5 years (required)
- Spark Airflow: 5 years (required)

Work Location: In person

**Speak with the employer**

+91 8667001718


  • AWS DE- Airflow

    4 weeks ago


    Chennai, India WILLWARE TECHNOLOGIES Full time

    Job Description:Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure.Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance.Collaborate with data scientists, analysts, and other stakeholders to understand requirements and translate them into technical solutions.Implement...


  • Chennai, Tamil Nadu, India Capgemini Full time

    **Job Description**: - Hands on industry experience in design and coding from scratch in AWS Glue-Pyspark with services like - S3, DynamoDB, StepFunctions etc. - Hands on industry experience in design and coding from scratch in Snowflake. - Experience in Pyspark/Snowflake 1 to 3 years with overall around 5 years of experience in building data/analytics...

  • Data Architect

    4 weeks ago


    Tamil Nadu, India Frost and Sullivan Private Limited Full time

    About the JobAt Frost & Sullivan, we are seeking a highly skilled and experienced Data Architect to join our Digital Products team. The ideal candidate will be responsible for designing, building, and maintaining our organization's data architecture.Key ResponsibilitiesDesign and implement efficient data architectures, storage solutions, and data processing...


  • Chennai, Tamil Nadu, India WILLWARE TECHNOLOGIES Full time

    Job Title: AWS Cloud Data EngineerJob SummaryWe are seeking a highly skilled AWS Cloud Data Engineer to join our team at WILLWARE TECHNOLOGIES. The successful candidate will design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure. This is an excellent opportunity for a data engineer to work on challenging projects and...

  • AWS Data Engineer

    4 weeks ago


    Chennai, Tamil Nadu, India WILLWARE TECHNOLOGIES Full time

    Job Title: AWS Data Engineer - Pipeline SpecialistAbout the Role:As a key member of our data engineering team at WILLWARE TECHNOLOGIES, you will be responsible for designing, developing, and deploying complex data pipelines using Apache Airflow on AWS infrastructure. Your expertise in data engineering, combined with your passion for innovative solutions,...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    About the RoleFulcrum Digital is a next-generation digital accelerating company providing digital transformation and technology services across various industries.We are seeking an experienced professional with a strong background in AWS cloud technologies and data engineering.The ideal candidate will be proficient in a wide range of AWS services and tools,...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    About the RoleFulcrum Digital is a cutting-edge digital transformation company providing innovative technology services to various industries. We are seeking a skilled Senior Data Engineer with expertise in AWS cloud technologies and data engineering.The ideal candidate will be proficient in a wide range of AWS services and tools, including Databricks, MWAA,...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    About Fulcrum DigitalWe are an agile and next-generation digital accelerating company providing digital transformation and technology services right from ideation to implementation. Our services have applicability across a variety of industries including banking & financial services, insurance, retail, higher education, food, healthcare, and...

  • Sr Data Engineer AWS

    4 weeks ago


    Chennai, India Fulcrum Digital Full time

    Who are we Fulcrum Digital is an agile and nextgeneration digital accelerating company providing digital transformation and technology services right from ideation to implementation. These services have applicability across a variety of industries including banking & financial services insurance retail higher education food healthcare and manufacturing. The...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    About the RoleFulcrum Digital is a leading digital transformation company that provides cutting-edge technology services. We are seeking an experienced professional with a strong background in AWS cloud technologies and data engineering.Key ResponsibilitiesDesign, develop, and maintain scalable data pipelines using AWS Databricks, AWS Lambda, AWS S3, Delta...


  • Chennai, Tamil Nadu, India Fulcrum Digital Full time

    At Fulcrum Digital, we are seeking a highly skilled professional with a strong background in AWS cloud technologies and data engineering to join our team.The ideal candidate will be proficient in a wide range of AWS services and tools, including Databricks, MWAA, Lambda, and Snowflake, among others.This role involves developing, managing, and optimizing...

  • Sr Data Engineer AWS

    3 months ago


    Chennai, India Fulcrum Digital Full time

    Who areweFulcrumDigital is an agile and nextgeneration digital accelerating companyproviding digital transformation and technology services right fromideation to implementation. These services have applicabilityacross a variety of industries including banking &financial services insurance retail higher education foodhealthcare andmanufacturing.TheRoleWeare...


  • Chennai, India Tata Consultancy Services Full time

    Role: AWS Data EngineerJOB LOCATION : Chennai, PuneEXPERIENCE REQUIREMENT : 4+Required Technical Skill:Strong Knowledge of Aws Glue/AWS REDSHIFT/SQL/ETL. Good knowledge and experience in Pyspark for forming complex Transformation logic.AWS Data Engineer,SQL, ETL, DWH , Secondary : AWS Glue , AirflowMust-HaveGood Knowledge of SQL , ETLA minimum of 3 +...


  • Chennai, India Tata Consultancy Services Full time

    Role: AWS Data EngineerJOB LOCATION : Chennai, PuneEXPERIENCE REQUIREMENT : 4+Required Technical Skill:Strong Knowledge of Aws Glue/AWS REDSHIFT/SQL/ETL. Good knowledge and experience in Pyspark for forming complex Transformation logic.AWS Data Engineer,SQL,ETL, DWH , Secondary : AWS Glue , AirflowMust-HaveGood Knowledge of SQL , ETLA minimum of 3 +...


  • Chennai, India Tata Consultancy Services Full time

    Role: AWS Data EngineerJOB LOCATION : Chennai, PuneEXPERIENCE REQUIREMENT : 4+Required Technical Skill:Strong Knowledge ofAws Glue/AWS REDSHIFT/SQL/ETL.Good knowledge and experience in Pyspark for forming complex Transformation logic.AWS Data Engineer,SQL,ETL,DWH, Secondary : AWS Glue , AirflowMust-HaveGood Knowledge of SQL , ETLA minimum of 3 + years'...

  • Sr Data Engineer AWS

    4 weeks ago


    chennai, India Fulcrum Digital Full time

    Who are weFulcrum Digital is an agile and nextgeneration digital accelerating company providing digital transformation and technology services right from ideation to implementation. These services have applicability across a variety of industries including banking & financial services insurance retail higher education food healthcare and manufacturing. The...


  • Tamil Nadu, India Frost and Sullivan (India) Private Limited Full time

    About the job :At Frost & Sullivan, we are seeking a highly skilled and experienced Data Architect to join our Digital Products team! The ideal fit for the role will be a person experienced in designing, building, and maintaining an organization's data architecture. The key focus of the position is designing and implementing data models, data...


  • Tamil Nadu, India Frost and Sullivan Private Limited Full time

    About the job :At Frost & Sullivan, we are seeking a highly skilled and experienced Data Architect to join our Digital Products team! The ideal fit for the role will be a person experienced in designing, building, and maintaining an organization's data architecture. The key focus of the position is designing and implementing data models, data integration...


  • Chennai, India Overture Rede Full time

    Job Title - AWS Data Engineer LeadLocation - Pan India Notice Period: Only Immediate Joiners.Mandatory Skills: AWS ,Glue,Athena ,Redshift, S3, EMR,Pyspark, Python,SQL.Roles & Responsibilities :Experience of AWS tools (e.g Athena, Redshift, Glue, EMR) ,Scala, Python, Spark Min 6- 10 yearsExperience of developing enterprise grade ETL/ELT data pipelines and...

  • Data Ops Engineer

    3 weeks ago


    tamil nadu, India Live Connections Full time

    Exp : 12yrs to 16yrsWe are looking for a strong IC candidate who has strong expertise in DevOps, DataOps, AWS, and Python.12+ years in data and software engineering.Deep expertise in Data Engineering, Software Engineering, Cloud Engineering, and AI (Knowledge is Mandatory).Strong understanding of DevOps, DataOps, and product management.Experience with Agile...