Aws De- Airflow

1 month ago

Chennai Tamil Nadu, India Willware Technologies Full time

**Work mode: One week of onboarding at Chennai office and later Remote**
- Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure.
- Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance.
- Collaborate with data scientists, analysts, and other stakeholders to understand requirements and translate them into technical solutions.
- Implement best practices for data ingestion, processing, transformation, and storage.
- Monitor pipeline performance, troubleshoot issues, and implement enhancements to improve efficiency and reliability.
- Ensure data quality and integrity throughout the data lifecycle by implementing robust validation and error handling mechanisms.
- Stay current with emerging technologies and industry trends in data engineering and contribute to continuous improvement initiatives.
- Mentor junior team members and provide technical guidance and support as needed.

**Qualification**:

- 5-7 years of professional experience in data engineering, with a focus on building and managing data pipelines.
- Extensive hands-on experience with Apache Airflow, including DAG design, scheduling, and orchestration.
- Strong proficiency in AWS services such as S3, EC2, EMR, Glue, Lambda, and Redshift.
- Solid understanding of data warehousing concepts, ETL processes, and data modeling techniques.
- Proficiency in programming languages such as Python, SQL, and Shell scripting.

**Job Type**: Temporary
Contract length: 3-6 months

**Benefits**:

- Work from home

Schedule:

- Monday to Friday

**Experience**:

- total work: 5 years (required)
- Spark Airflow: 5 years (required)

Work Location: In person

**Speak with the employer**

+91 8667001718

  • Senior Data Engineer

    2 weeks ago

    Chennai/Any Location, India mlesystems Full time

    Job Title: Senior Data Engineer - Airflow Location: Chennai (Remote) Company Overview: MLE Systems is a leading provider of advanced data analytics solutions, empowering businesses to harness the power of data for informed decision-making. We specialize in developing cloud-based platforms and cutting-edge technologies to drive actionable insights. We seek...

  • Senior Data Engineer

    2 weeks ago

    Chennai/Any Location, IN mlesystems Full time

    Job Title: Senior Data Engineer - Airflow Location: Chennai (Remote) Company Overview: MLE Systems is a leading provider of advanced data analytics solutions, empowering businesses to harness the power of data for informed decision-making. We specialize in developing cloud-based platforms and cutting-edge technologies to drive actionable insights. We seek a...

  • Platform Engineer

    2 months ago

    chennai, India Mobile Programming LLC Full time

    Job Description : Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development...

  • Platform Engineer

    4 weeks ago

    Chennai, India Mobile Programming LLC Full time

    Job Description :Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development plans,...

  • AWS cloud Engineer

    1 month ago

    chennai, India Virtusa Full time

    AWS cloud Engineer - CREQ187082 Description Minimum 6-9 years of experience in AWS CloudBe responsible for the planning, implementation, and growth of the AWS cloud infrastructure.Implement continuous integration/continuous delivery (CI/CD) pipelines when necessary.Work alongside architecture and engineering teams to design and implement any scalable...

  • AWS Data Engineer

    4 weeks ago

    Bangalore/Pune/Chennai, India Change leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :- Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform.- This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • AWS Data Engineer

    1 month ago

    Bangalore/Pune/Chennai, IN Change leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :- Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform.- This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • AWS Data Engineer

    5 days ago

    Chennai, India Ilink Systems Pvt Ltd Full time

    Job Description :- AWS Data Engineer with min of 5 to 7 years of experience.- Collaborate with business analysts to understand and gather requirements for existing or new ETL pipelines.- Connect with stakeholders daily to discuss project progress and updates.- Work within an Agile process to deliver projects in a timely and efficient manner.- Design and...

  • AWS Data Engineer

    1 month ago

    Chennai, India CliqHR Full time

    Job Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...

  • AWS Data Engineer

    1 month ago

    Chennai, India CliqHR Full time

    Job Description : - Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...

  • AWS Data Engineer

    1 month ago

    Chennai, Tamil Nadu, India CliqHR Full time

    Job Description :- Must have minimum 5+ years of experience as AWS Data EngineerNice to have : AWS Certified Data Analytics Specialty certification- Experience in designing and implementing data pipelines using AWS Glue and orchestration using Airflow for batch, streaming (Kafka/Kinesis) and API ingestion patterns- Work closely with DevOps engineer to...

  • AWS Data Engineer

    5 days ago

    Chennai, India CliqHR Full time

    Job Description : - AWS, Python, SQL, Spark (with and with snowflake and databricks) AWS DE- AWS, Python, SQL, Spark- Bachelor's degree in Computer Science, Information Technology, or a related field.- 4-8 years of professional experience in data engineering or a related role.- Strong expertise in AWS services (e.g., S3, Redshift, Glue, Lambda).-...

  • AWS Data Engineer

    6 days ago

    chennai, India CliqHR Full time

    Job Description : - AWS, Python, SQL, Spark (with and with snowflake and databricks) AWS DE- AWS, Python, SQL, Spark- Bachelor's degree in Computer Science, Information Technology, or a related field.- 4-8 years of professional experience in data engineering or a related role.- Strong expertise in AWS services (e.g., S3, Redshift, Glue, Lambda).- Proficient...

  • AWS Data Engineer

    6 days ago

    Chennai, India CliqHR Full time

    Job Description :- AWS, Python, SQL, Spark (with and with snowflake and databricks) AWS DE- AWS, Python, SQL, Spark- Bachelor's degree in Computer Science, Information Technology, or a related field.- 4-8 years of professional experience in data engineering or a related role.- Strong expertise in AWS services (e.g., S3, Redshift, Glue, Lambda).-...

  • AWS Data Engineer

    5 hours ago

    Chennai, Tamil Nadu, India CliqHR Full time

    Job Description :- AWS, Python, SQL, Spark (with and with snowflake and databricks) AWS DE- AWS, Python, SQL, Spark- Bachelor's degree in Computer Science, Information Technology, or a related field.- 4-8 years of professional experience in data engineering or a related role.- Strong expertise in AWS services (e.g., S3, Redshift, Glue, Lambda).-...

  • Java AWS Lead Engineer

    7 days ago

    Chennai, India Virtusa Full time

    Java AWS Lead Engineer - CREQ188358 DescriptionJOB DESCRIPTIONSkill: Java AWS Lead EngineerRole / Tier: Lead Software Engineer/Tier 2Job Description:Mandatory SkillsSpring bootCore Java and microservicesAws LambdaAWS serverless conceptsBuild cloud service on top of the modern.Infrastructure as Service (IaaS) building blocks at OCI.Design and build...

  • Platform Engineer

    4 weeks ago

    Chennai/Anywhere in India/Multiple Locations Mobile Programming LLC Full time

    Job Description : Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development...

  • Platform Engineer

    1 month ago

    Chennai/Anywhere in India/Multiple Locations, IN Mobile Programming LLC Full time

    Job Description :Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development plans,...

  • AWS Data Engineer

    4 weeks ago

    Pune/Chennai/Mumbai/Bangalore/Delhi NCR/Hyderabad, India Change Leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform. This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • Senior Data Engineer( AWS Redshift)

    2 months ago

    chennai, India Anicalls (Pty) Ltd Full time

    Candidate should have:Clear, concise communication abilities - writing, verbal, presentation - to all levels of technical and non-technical audiences with the ability to collaborateStrong experience in CICD technologies including Python, SQL, Airflow, and Git in a cloud settingUnderstanding of ELK stack is a plus for logging and reporting purposesExperience...