Big Data Engineer(AWS, Python,Airflow)

2 days ago

Hyderabad, Telangana, India Zorba AI Full time

Job Description – AWS / Python / Airflow / Big Data Engineer
Role:
AWS / Python / Airflow / Big Data Engineer

Experience:
3–5 Years

Job Overview
We are seeking
two skilled AWS/Python/Airflow/Big Data Engineers
with
3–5 years of hands-on experience
to join our data engineering team. The ideal candidates will be responsible for
designing, developing, and maintaining scalable data pipelines and workflows
in a
cloud-based AWS environment
. This role requires strong expertise in
Python, Apache Airflow, AWS services, and Big Data technologies
.

Key Responsibilities

  • Design, develop, and maintain ETL and data pipelines using Apache Airflow.
  • Build and manage workflow orchestration (DAGs) for reliable data processing.

  • Implement scalable and cost-efficient solutions on AWS Cloud, leveraging:

  • S3, EC2, Lambda, EMR, Glue, Redshift

  • Write efficient, reusable, and maintainable Python code for data processing and automation.

  • Work with Big Data frameworks such as Apache Spark and Hadoop for large-scale data processing.
  • Optimize data workflows for performance, scalability, and cost efficiency.
  • Collaborate with cross-functional teams (data analysts, product, DevOps) to understand data requirements and deliver solutions.
  • Ensure data quality, reliability, security, and monitoring across all pipelines and workflows.
  • Troubleshoot and resolve data pipeline failures and performance bottlenecks.

Required Qualifications
Technical Skills

  • AWS Cloud Services: Hands-on experience with core AWS services used in data engineering.
  • Python Programming: Strong coding skills with experience using libraries such as Pandas, PySpark, and automation scripts.
  • Apache Airflow: Proven experience building and managing Airflow DAGs and workflows.
  • Big Data Technologies: Familiarity with Spark, Hadoop, or similar distributed data processing frameworks.
  • Data Modeling & ETL: Solid understanding of ETL processes, data transformations, and pipeline design.
  • Databases & SQL: Working knowledge of SQL and relational databases.

Soft Skills

  • Strong problem-solving and debugging abilities.
  • Excellent communication and collaboration skills.
  • Ability to work independently and in a fast-paced team environment.

Preferred Qualifications

  • Experience with CI/CD pipelines and DevOps practices.
  • Familiarity with containerization technologies such as Docker and Kubernetes.
  • Exposure to cloud monitoring, logging, and performance optimization tools.

Skills: python,pipelines,big data,cloud,aws,airflow

  • Big data Engineer

    2 days ago

    Hyderabad, Telangana, India Zorba AI Full time

    Role OverviewWe are looking fortwo Junior-level Big Data Engineerswith strong hands-on experience inAWS, Python, Apache Airflow, and Big Data technologies. The ideal candidates will design, build, and maintainscalable data pipelines and workflowsin a cloud-based environment, ensuring data reliability, performance, and security.Key ResponsibilitiesDesign,...

  • Airflow Data Engineer

    6 days ago

    Hyderabad, Telangana, India Virtusa Full time

    Develops and maintains scalable data pipelines to support continuing increases in data volume and complexity.Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increase data accessibility, and fostering data-driven decision making across the organization.Implement processes and systems to monitor data...

  • Big Data QA/Test Engineer

    2 days ago

    Hyderabad, Telangana, India deltaclass technology solutions Full time

    Role : Bigdata QA Test Engineer Exp : 6+Yrs Notice :Immediate to 15 Days Location:Hyderabad (Hybrid) Big Data QA/Test engineer – 3 Openings Hyderabad or Chennai (Hybrid) Big Data QA/Test Engineer AWS + Data-bricks + Big Data + Python + DevOps Big Data QA/Test Engineer AWS + Data-bricks + Big Data + Python + DevOps Big Data QA/Test Engineer AWS +...

  • Big Data QA/Test Engineer

    15 hours ago

    Hyderabad, Telangana, India deltaclass technology solutions Full time

    Currently we are looking for Bigdata QA test Engineer. Responsibilities Role : Bigdata QA Test Engineer Exp : 6+Yrs Notice :Immediate to 15 Days Location:Hyderabad (Hybrid) Big Data QA/Test engineer – 3 Openings Hyderabad or Chennai (Hybrid) Big Data QA/Test Engineer AWS + Data-bricks + Big Data + Python + DevOps Big Data QA/Test Engineer AWS +...

  • AWS Data Engineer

    2 days ago

    Hyderabad, Telangana, India Zorba AI Full time

    Job Description: AWS Data EngineerPosition OverviewWe are seeking a seasoned AWS Data Engineer with at least 3 years of experience to design and implement a robust data mart layer built on behavioral and transactional data. The ideal candidate will have a strong background in data modeling, SQL development, and ELT pipelines, with a proven ability to ensure...

  • Airflow Reliability Engineer

    2 days ago

    Hyderabad, Telangana, India Astronomer Full time

    Astronomer empowers data teams to bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data-driven applications. Trusted by more than 800 of the...

  • Python ( Airflow / DBT ) Engineer

    4 days ago

    Hyderabad, Telangana, India Luxoft Full time

    Project descriptionThis project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data accessibility, performance, and governance while enabling advanced analytics and automation.The resource will join an ongoing effort to accelerate...

  • Python (Airflow/DBT) Engineer

    2 weeks ago

    Hyderabad, Telangana, India Luxoft Full time

    Project Description:This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data accessibility, performance, and governance while enabling advanced analytics and automation.The resource will join an ongoing effort to accelerate...

  • Python + Airflow Professional

    4 days ago

    Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time

    We are looking for a skilled Python developer with expertise in Airflow to join our team at IDESLABS PRIVATE LIMITED. The ideal candidate should have 6-10 years of experience.Roles and ResponsibilityDesign, develop, and deploy scalable data pipelines using Apache Airflow.Collaborate with cross-functional teams to identify and prioritize project...

  • AWS Cloud Python Data Engineer

    4 days ago

    Hyderabad, Telangana, India Creator City Full time

    Data Engineer (L3) – Python & AWSJob Overview:We are seeking a highly skilled Data Engineer with strong expertise inAWS cloud services and Python programming. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines, ensuring data availability, quality, and performance across enterprise systems. You will...