Senior Data Pipeline Developer

4 days ago

Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 20,00,000 - ₹ 35,00,000
Senior Data Pipeline Developer

About the Role:

We are seeking an experienced Senior Data Pipeline Developer to join our team. As a key member of our data engineering team, you will be responsible for designing and developing comprehensive test cases for Azure Data Factory ETL pipelines using Python and Pytest.

Key Responsibilities:

  • Design and develop manual and automated test cases for Azure Data Factory ETL pipelines using Python and Pytest.
  • Validate data transformations, data mappings, and data integrity through SQL queries and data profiling.
  • Automate functional, regression, and performance tests of ETL workflows to ensure pipeline reliability.
  • Collaborate with data engineers, analysts, and developers to understand ETL and POWER BI specifications and data flow requirements.
  • Document test plans, test cases, defects, and provide timely feedback on issues.
  • Maintain Test automation frameworks, leveraging Pytest fixtures and libraries such as to validate Azure data factory activities.

Requirements:

Technical Skills:

  • Cloud: Microsoft Azure Data Factory
  • CI/CD Platforms: Azure DevOps, GitHub
  • Testing Framework: Pytest
  • Languages: Python, SQL
  • Reporting: Power BI, Tableau
  • Version Control: Git

Personal Qualities:

  • Strong understanding of data engineering concepts and principles.
  • Excellent problem-solving skills and attention to detail.
  • Ability to work effectively in a collaborative environment.
  • Effective communication and documentation skills.

This is an excellent opportunity to join a dynamic team and contribute to the development of innovative data solutions. If you are passionate about data engineering and have a strong background in testing and automation, we encourage you to apply for this exciting role.

  • Senior Data Pipeline Developer

    4 days ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 19,80,000 - ₹ 25,20,000

    Key ResponsibilitiesWe are seeking a skilled professional to design, develop and maintain data pipelines that ingest data from various sources into our data warehouse.Data Ingestion: The ideal candidate will design and implement data pipelines using AWS services such as Lambda, DMS and Glue.Data Transformation: They must have expertise in SQL scripting and...

  • Chief Data Pipeline Architect

    2 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe are seeking an experienced professional to lead our data engineering team. As a key member, you will be responsible for designing, developing, and optimizing large-scale data pipelines using Apache Spark, AWS Glue, and Azure Data Factory.Key Responsibilities:Design, develop, and optimize ETL/ELT data pipelines using Apache...

  • AI Pipeline Developer

    6 days ago

    Thoothukudi, Tamil Nadu, India beBeeArtificialIntelligence Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior AI Specialist PositionExcelgig is hiring an AI Specialist with expertise in custom model development and end-to-end pipeline creation.Key Responsibilities:Design and develop custom machine learning and deep learning models for business-specific problems (NLP, computer vision, predictive modeling, recommendation systems, etc.).Build and maintain...

  • Expert Data Pipeline Architect

    4 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team.The successful candidate will have expertise in designing, implementing and optimizing data pipelines using Python and cloud-based technologies.Key responsibilities include:Designing and implementing Extract, Transform, Load (ETL) processes to move data from diverse...

  • Data Pipeline Architect

    6 days ago

    Thoothukudi, Tamil Nadu, India beBeeInformatician Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Profile:">Lead Informatica Developer">We are seeking a highly skilled and experienced Informatica ETL Developer with expertise in designing, developing, testing, and maintaining large-scale ETL applications. The ideal candidate will have a strong background in data pipeline development and take ownership in delivering scalable solutions that meet...

  • Lead Data Pipeline Tester

    3 days ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time US$ 1,00,000 - US$ 1,45,000

    Job Opportunity: Data Pipeline TesterJob Summary:We are looking for a seasoned professional to lead the development and execution of testing strategies for data pipeline projects. The ideal candidate will have expertise in testing ETL workflows, SQL Data Warehouses, and BI reports.Key Responsibilities:Develop and execute testing strategies for data pipelines...

  • ETL Data Pipelines Specialist

    3 days ago

    Thoothukudi, Tamil Nadu, India beBeeInformatica Full time ₹ 18,00,000 - ₹ 20,00,000

    Job Title:">A seasoned professional is required to lead the design, development, and maintenance of data pipelines using Informatica IICS.The ideal candidate will have experience integrating IICS solutions with cloud-based data platforms such as Google Cloud Platform (GCP) and Google BigQuery (GBQ).Pipeline optimization for scalability and performance is...

  • High-Performance Data Pipeline Architect

    6 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Big Data Engineer PositionWe are seeking a skilled Big Data Engineer to design, build, and maintain high-performance data pipelines that process massive amounts of data. This role is critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.Key Responsibilities:Design and implement...

  • Senior Data Engineering Expert

    2 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Unlock Your Career Potential as a Senior Data EngineerAbout the Role:We are seeking an experienced and skilled GCP Data Engineer to join our team. In this role, you will design, develop, and maintain data pipelines and data warehouse solutions on Google Cloud Platform (GCP). You will play a critical role in ensuring the seamless flow of data across systems,...

  • Senior Data Engineering Specialist

    4 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe are seeking a highly skilled Senior Data Engineer with expertise in ETL pipeline design, data modeling, and data infrastructure.We are looking for a detail-oriented professional with strong experience in designing and implementing efficient ETL/ELT pipelines and data flows.This role is ideal for someone who thrives in a fast-paced engineering...