ETL Developers with Experience in Airflow

2 days ago

Bengaluru, Karnataka, India Value Health Inc Full time

Job Title: ETL Developers

Job Location: Coimbatore

Type: WFO

Job description

Key Responsibilities:

1. ETL Design and Development:

Design and develop efficient, scalable SSIS packages to extract, transform, and load data between systems.

Translate business requirements into technical ETL solutions using data flow and control flow logic.

Develop reusable ETL components that support modular, configuration-driven architecture.

2. Data Integration and Transformation:

Integrate data from multiple heterogeneous sources: SQL Server, flat files, APIs, Excel, etc.

Implement business rules and data transformations such as cleansing, standardization, enrichment, and deduplication.

Manage incremental loads, full loads, and slowly changing dimensions (SCD) as required.

3. SQL and Database Development:

Write complex T-SQL queries, stored procedures, and functions to support data transformations and staging logic.

Perform joins, unions, aggregations, filtering, and windowing operations effectively for data preparation.

Ensure referential integrity and proper indexing for performance.

4. Performance Tuning:

Optimize SSIS packages by tuning buffer sizes, using parallelism, and minimizing unnecessary transformations.

Tune SQL queries and monitor execution plans for efficient data movement and transformation.

Implement efficient data loads for high-volume environments.

5. Error Handling and Logging:

Develop error-handling mechanisms and event logging in SSIS using Event Handlers and custom logging frameworks.

Implement restartability, checkpoints, and failure notifications in workflows.

6. Testing and Quality Assurance:

Conduct unit and integration testing of ETL pipelines.

Validate data outputs against source and business rules.

Support QA teams in user acceptance testing (UAT) and defect resolution.

7. Deployment and Scheduling:

Package, deploy, and version SSIS solutions across development, test, and production environments.

Schedule ETL jobs using SQL Server Agent or enterprise job schedulers (e.g., Control-M, Tidal).

Monitor and troubleshoot job failures and performance issues.

8. Documentation and Maintenance:

Maintain documentation for ETL designs, data flow diagrams, transformation logic, and job schedules.

Update job dependencies and maintain audit trails for data pipelines.

9. Collaboration and Communication:

Collaborate with data architects, business analysts, and reporting teams to understand data needs.

Provide technical support and feedback during requirements analysis and post- deployment support.

Participate in sprint planning, status reporting, and technical reviews.

10. Compliance and Best Practices:

Ensure ETL processes comply with data governance, security, and privacy regulations (HIPAA, GDPR, etc.).

Follow team coding standards, naming conventions, and deployment protocols.

Required Skills s Experience:

  • 4–8 years of hands-on experience with ETL development using SSIS.
  • Strong SQL Server and T-SQL skills.
  • Solid understanding of data warehousing concepts and best practices.
  • Experience with flat files, Excel, APIs, or other common data sources.
  • Familiarity with job scheduling and monitoring (e.g., SQL Agent).
  • Strong analytical and troubleshooting skills.
  • Ability to work independently and meet deadlines.

Preferred Skills:

  • Exposure to Azure Data Factory or cloud-based ETL tools.
  • Experience with Power BI or other reporting platforms.
  • Experience in healthcare, finance, or regulated domains is a plus.
  • Knowledge of version control tools like Git or Azure DevOps.

  • Airflow Developer

    4 days ago

    Bengaluru, Karnataka, India ExaTech Inc Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Airflow Developer in Noida, Bangalore , PuneWe are seeking a skilledAirflow Developerto lead the migration of ETL workflows from Talend to Apache Airflow. The ideal candidate will have hands-on experience in designing, developing, and deploying data pipelines using Airflow, and a solid understanding of Talend ETL processes. This role involves analysing...

  • ETL Migration SME – Airflow to Talend

    4 days ago

    Bengaluru, Karnataka, India Aiprus Software Private Limited Full time US$ 1,25,000 - US$ 1,75,000 per year

    Job Title: ETL Migration SME – Airflow to TalendLocation: Bangalore, IndiaExperience: 8–10 YearsJob Type: Full-TimeDepartment: Data Engineering / IT TransformationRole OverviewWe are looking for a seasoned ETL Migration SME to lead the migration of enterprise data workflows from Apache Airflow to Talend. The candidate should have strong hands-on...

  • Urgent ETL Developer

    3 days ago

    Bengaluru, Karnataka, India Eteam Full time

    Job DescriptionKey Responsibilities:1. ETL Pipeline Development:- Design, develop, and maintainscalable ETL processesto extract, transform, and load data from various structured and unstructured sources intoGCP-based data warehouses (BigQuery, Cloud SQL, Cloud Storage, etc.).- Develop efficient SQL queries and scripts to supportdata transformation,...

  • Data Engineer with Airflow

    4 days ago

    Bengaluru, Karnataka, India Carnation Infotech Full time US$ 1,50,000 - US$ 2,00,000 per year

    Job Title: Data Engineer Airflow & Microsoft FabricPosition Summary We are seeking a highly experienced Senior Data Engineer to support a short-term Azure Data Fabric implementation project. The ideal candidate will have strong expertise in Apache Airflow and hands-on experience with Azure Data Fabric, with a proven track record of designing and deploying...

  • Senior ETL Migration Specialist

    5 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior ETL Migration SpecialistWe are seeking an accomplished ETL migration specialist to spearhead the transition of enterprise data workflows from Apache Airflow to Talend. The ideal candidate will possess hands-on expertise in both platforms and a proven track record of delivering successful ETL modernization projects.Key Responsibilities:Leverage your...

  • Senior ETL Developer

    3 days ago

    Bengaluru, Karnataka, India APPSIKA SOFT TECH PRIVATE LIMITED Full time

    Position: ETL Engineer (Fivetran/Matillion ETL/Qliksense)Location: Gachibowli TelanganaEmployment Type: Full-time / ContractExperience: 3–7 years Required SkillsHands-on experience with Fivetran (connectors, transformations, monitoring).Strong working knowledge of Matillion ETL (jobs, orchestration, transformation, performance tuning).Proficiency in SQL...

  • Senior Python ETL Developer/Lead

    4 days ago

    Bengaluru, Karnataka, India ALIQAN Technologies Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Title: Senior Python ETL Developer/LeadJob Type: 6 Months + Extendable Experience : 5 to 6 yearsJob Description:We are seeking an experienced Senior ETL Developer/Lead to join our team. The ideal candidate will have a strong background in developing ETL processes using Python and open-source tools/libraries, including PySpark and Apache Airflow. This...

  • ETL Migration Subject Matter Expert

    5 days ago

    Bengaluru, Karnataka, India Aiprus Software Private Limited Full time

    Role Overview :We are looking for a seasoned ETL Migration SME to lead the migration of enterprise data workflows from Apache Airflow to Talend. The candidate should have strong hands-on experience in both platforms and a proven track record of delivering successful ETL modernization projects.Key Responsibilities :- Lead and manage the migration of ETL...

  • ETL Development

    4 days ago

    Bengaluru, Karnataka, India Standard Chartered Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Key Responsibilities 8 to 10 years Experience of working in data analytical ETL Design & Development projects Expertise in tools like Informatica, Talend, SSIS, DataStage, and custom Python-based ETL frameworks Must know Programming skills like ETL using Hadoop, Python, SQL, Shell scripting, PySpark, Scoop, Scala SQL Expertise in writing complex queries,...

  • Apache Airflow

    4 weeks ago

    Bengaluru, Karnataka, India People Prime Worldwide Full time

    About Company :They balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide customers from what's now to what's next by unlocking the value of their data and applications to solve their digital challenges, achieving outcomes that benefit both business and society.About...