Python/Airflow Data Engineer

5 days ago

Bengaluru, Karnataka, India Rapsys Technologies Pte. Ltd Full time ₹ 1,04,000 - ₹ 1,30,878 per year

Years of experience: 5+ Years

JD:

Responsibilities:

  • Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.

  • Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.

  • Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.

  • Optimize and fine-tune existing data pipelines / Airflow to improve efficiency, scalability, and maintainability.

  • Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and minimal downtime.

  • Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.

  • Develop and maintain documentation for data pipelines, processes, and system architecture.

  • Stay updated with the latest industry trends and best practices related to data engineering and AWS services.

Requirements:

  • Bachelor's degree in Computer Science, Engineering, or a related field.


• Proficiency in Python, PySpark and SQL for data processing and manipulation.


• Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.

  • Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda.

  • Understanding of Snowflake Data Lake is preferred.

  • Experience with optimizing and scaling data pipelines for performance and efficiency.

  • Good understanding of data modeling, ETL processes, and data warehousing concepts.

  • Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.

  • Effective communication skills and the ability to articulate technical concepts to non-technical stakeholders.

Preferred Qualifications:

  • AWS certification(s) related to data engineering or big data.

  • Experience working with big data technologies like Snowflake, Spark, Hadoop, or related frameworks.

  • Familiarity with other data orchestration tools in addition to Apache Airflow.

  • Knowledge of version control systems like Bitbucket, Git.

Job Type: Contractual / Temporary

Contract length: 12 months

Application Question(s):

  • Are you currently serving notice period? if no, what is your official notice period?
  • Are you open for 1 Year renewable of Contract Role?

Experience:

  • Python: 5 years (Required)
  • Snowflake: 4 years (Required)
  • Apache Airflow: 5 years (Required)

  • Python/Airflow Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Rapsys Tech Solutions Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    • Proficiency in Python, PySpark and SQL for data processing and manipulation.• Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.• Understanding of Snowflake Data Lake is preferred.

  • Snowflake, Python, Airflow Engineer

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Req ID 322031NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Snowflake Python Airflow Engineer to join our team in Bangalore Karnxc4x81taka IN-KA India IN Job Duties Team...

  • Data Engineer with Airflow

    5 days ago

    Bengaluru, Karnataka, India Carnation Infotech Full time US$ 1,50,000 - US$ 2,00,000 per year

    Job Title: Data Engineer Airflow & Microsoft FabricPosition Summary We are seeking a highly experienced Senior Data Engineer to support a short-term Azure Data Fabric implementation project. The ideal candidate will have strong expertise in Apache Airflow and hands-on experience with Azure Data Fabric, with a proven track record of designing and deploying...

  • Airflow

    3 weeks ago

    Bengaluru, Karnataka, India LTIMindtree Full time

    Airflow Senior data engineer Airflow, Python, Snowflake Loc: Pune, Bengaluru, Chennai CTC: Max 19 lpa Thanks & Regards, Indumati Nakate

  • Airflow

    3 weeks ago

    Bengaluru, Karnataka, India LTIMindtree Full time

    AirflowSenior data engineerAirflow, Python, SnowflakeLoc: Pune, Bengaluru, ChennaiCTC: Max 19 lpaThanks & Regards,Indumati Nakateindumati.nakate@alphacom.in

  • Airflow Developer

    4 days ago

    Bengaluru, Karnataka, India ExaTech Inc Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Airflow Developer in Noida, Bangalore , PuneWe are seeking a skilledAirflow Developerto lead the migration of ETL workflows from Talend to Apache Airflow. The ideal candidate will have hands-on experience in designing, developing, and deploying data pipelines using Airflow, and a solid understanding of Talend ETL processes. This role involves analysing...

  • Python Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are currently seeking a Python Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN).Job Duties:As a Senior Python Engineer, you will be a member of the C3 Data Warehouse team with a focus on building our next-gen data platform used for sourcing and storing data from different technology systems across the firm into a centralized data...

  • Python Engineer

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA North America Full time

    Job DescriptionReq ID: 330195NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Python Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN).Job Duties: As a Senior Python...

  • Data Engineer

    3 days ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Data Engineer - Senior Engineer/ManagerExperience Range: yearsLocation: Bengaluru, IndiaPrimary Skills Required:Position 1(Required Exp Range years): Java, Scala, Spark, Python, Kafka, Airflow, SQL, Databricks, Snowflake, Hive, AWS, Oracle Exadata, SAP Data ServicesPosition 2(Required Exp Range years): Java, Scala, Spark, Python, Kafka, Airflow, SQL,...

  • ETL Migration SME – Airflow to Talend

    5 days ago

    Bengaluru, Karnataka, India Aiprus Software Private Limited Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: ETL Migration SME – Airflow to TalendLocation: Bangalore, IndiaExperience: 8–10 YearsJob Type: Full-TimeDepartment: Data Engineering / IT TransformationRole OverviewWe are looking for a seasoned ETL Migration SME to lead the migration of enterprise data workflows from Apache Airflow to Talend. The candidate should have strong hands-on...