Apache Airflow

2 weeks ago

Bengaluru, Karnataka, India People Prime Worldwide Full time

About Company :

They balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide customers from what's now to what's next by unlocking the value of their data and applications to solve their digital challenges, achieving outcomes that benefit both business and society.

About Client:

Our client is a global digital solutions and technology consulting company headquartered in Mumbai, India. The company generates annual revenue of over $4.29 billion (₹35,517 crore), reflecting a 4.4% year-over-year growth in USD terms. It has a workforce of around 86,000 professionals operating in more than 40 countries and serves a global client base of over 700 organizations.

Our client operates across several major industry sectors, including Banking, Financial Services & Insurance (BFSI), Technology, Media & Telecommunications (TMT), Healthcare & Life Sciences, and Manufacturing & Consumer. In the past year, the company achieved a net profit of $553.4 million (₹4,584.6 crore), marking a 1.4% increase from the previous year. It also recorded a strong order inflow of $5.6 billion, up 15.7% year-over-year, highlighting growing demand across its service lines.

Key focus areas include Digital Transformation, Enterprise AI, Data & Analytics, and Product Engineering—reflecting its strategic commitment to driving innovation and value for clients across industries.

Job Title: Apache Airflow Senior Python Tech Lead

Location:

Experience Level: 8+ years

Position Overview

We are seeking an Apache Airflow Senior Python Tech Lead with strong expertise in Python and hands-on experience in Azure cloud technologies. This role will focus on migrating processes from existing 3rd Party RPA modules to Apache Airflow, ensuring seamless orchestration and automation of workflows. The ideal candidate will have deep technical proficiency, problem-solving ability, and a strong understanding of workflow automation—preferably with exposure to North America insurance industry processes.

Key Responsibilities

Workflow Migration & Development

  • Design, develop, and implement workflows in Apache Airflow to replace current 3rd Party RPA modules.
  • Lead migration of processes such as Auto, Package, Work Order Processing, and Policy Renewals for CI, Major Accounts, and Middle Market LOBs.

Automation & Scripting

  • Build and optimize Python scripts to enable automation and integration within Airflow pipelines.
  • Perform unit testing, troubleshooting, and performance tuning for workflows and scripts.

Cloud Integration & Deployment

  • Leverage Azure services (Azure Data Factory, Azure DevOps, Azure Storage) for deployment, monitoring, and scaling of Airflow workflows.

Collaboration & Documentation

  • Collaborate with cross-functional teams to understand existing processes, dependencies, and business objectives.
  • Document workflows, configurations, and technical details to maintain comprehensive project records.

Leadership & Mentoring

  • Mentor junior developers and share best practices for Apache Airflow and Python development.

Required Technical Skills

  • Strong expertise in Apache Airflow for workflow orchestration.
  • Advanced Python programming skills for scripting, data transformation, and automation.
  • Hands-on experience with Azure cloud services (Data Factory, DevOps, Storage, etc.).
  • Proven track record in migrating and automating processes from legacy or RPA systems.
  • Strong analytical, problem-solving, and debugging skills.

  • High Salary Apache Airflow Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Maneva Consulting Private Limited Full time

    Job DescriptionGreetings from ManevaJob DescriptionJob Title-Apache Airflow EngineerExperience- 3-10 daysLocation-BangaloreNotice-Immediate to 15 daysRequirements:-As a Senior Engineer with a focus on Managed Airflow Platform (MAP) support engineering.- Evangelize and cultivate adoption of Global Platforms, open-source software and agile principles within...

  • GCP Data Engineer

    6 days ago

    Bengaluru, Karnataka, India Fractal Full time

    We are seeking an experienced GCP Data Engineer with 6+ years of experience to design, develop, and manage cloud-based data solutions on Google Cloud Platform (GCP). The ideal candidate will have expertise in BigQuery, Dataflow, Pub/Sub, Cloud Composer (Apache Airflow), and Terraform, along with strong experience in ETL/ELT pipelines, data modeling, and...

  • Cloud Data Processing Specialist

    1 day ago

    Bengaluru, Karnataka, India beBeeEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Optimize Your Data Engineering SkillsAchieve success as a skilled data engineer by mastering GCP tools and technologies.The ideal candidate will have expertise in data processing frameworks like Apache Beam (Data Flow), proficiency in programming languages such as Python and Java, and strong problem-solving abilities.They should also understand DevOps tools,...

  • Senior Data Scientist Role

    4 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 40,00,000 - ₹ 50,00,000

    Data Engineer PositionAs a data engineer, you will be responsible for designing and building large-scale data processing systems. Your work will involve using programming languages such as Python to develop data pipelines and scripts that process and transform data.We are seeking a skilled data engineer to join our team in designing and implementing data...

  • snowflake Cloud Database Administrator

    2 weeks ago

    Bengaluru, Karnataka, India Leading automobile GCC Full time

    Cloud Platform Expertise: Familiarity with leading cloud platforms such as Amazon Web Services (AWS), Microsoft Azure, and Snowflake is essential. Understanding their database offerings, such as AWS RDS, Azure SQL Database is crucial for effective management Database Administration Fundamentals: A solid grasp of DBA principles, including schema design,...

  • snowflake Cloud Database Administrator

    2 weeks ago

    Bengaluru, Karnataka, India Leading automobile GCC Full time

    Cloud Platform Expertise: Familiarity with leading cloud platforms such as Amazon Web Services (AWS), Microsoft Azure, and Snowflake is essential. Understanding their database offerings, such as AWS RDS, Azure SQL Database is crucial for effective management Database Administration Fundamentals: A solid grasp of DBA principles, including schema design,...

  • Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India BNP Paribas Full time

    Responsibilities Direct Responsibilities Design and develop real-time and batch data pipelines using tools like Apache NiFi, Apache Kafka, and Apache Airflow. Implement and maintain ETL/ELT processes, ensuring data quality and integrity across various data sources. Integrate and manage data flows across diverse systems using NiFi and Kafka topics. Monitor...

  • Data Engineer

    4 days ago

    Bengaluru, Karnataka, India BNP Paribas Full time

    ResponsibilitiesDirect ResponsibilitiesDesign and develop real-time and batch data pipelines using tools like Apache NiFi, Apache Kafka, and Apache Airflow.Implement and maintain ETL/ELT processes, ensuring data quality and integrity across various data sources.Integrate and manage data flows across diverse systems using NiFi and Kafka topics.Monitor and...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India BNP Paribas Full time

    Responsibilities Direct Responsibilities Design and develop real-time and batch data pipelines using tools like Apache NiFi, Apache Kafka, and Apache Airflow. Implement and maintain ETL/ELT processes, ensuring data quality and integrity across various data sources. Integrate and manage data flows across diverse systems using NiFi and Kafka topics. Monitor...

  • Chief Data Integration Specialist

    1 day ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewAs a Data Engineer, you will play a pivotal role in designing and developing real-time and batch data pipelines. Your expertise will be instrumental in integrating diverse systems and ensuring seamless data flow.The ideal candidate will possess a strong background in Apache NiFi for data flow management, Kafka for streaming data, and Apache...