Lead Data Pipeline Specialist

2 days ago

Bareilly, Uttar Pradesh, India beBeeTester Full time ₹ 15,00,000 - ₹ 25,00,000

About this position :

We are seeking a highly skilled ETL and Power BI Tester to join our team. As an experienced professional, you will be responsible for ensuring the accuracy, integrity, and performance of ETL Pipelines deployed into Azure cloud environments.

The role involves developing and automating comprehensive test cases to validate data flows, transformations, and system behaviors using python scripting and Pytest framework, integrated into CI/CD pipelines for automated quality assurance.

Key Responsibilities:

1. Design and develop manual and automated test cases for Azure Data Factory ETL pipelines using python and pytest.

2. Validate data transformations, data mappings, and data integrity through SQL queries and data profiling.

3. Automate functional, regression, and perfomance tests of ETL workflows to ensure pipeline reliability.

4. Collaborate with data engineers, analysts, and developers to understand ETL and POWER BI specifications and data flow requirements.

5. Document Test plans, Test Cases, defects, and provide timely feedback on issues.

6. Maintain Test automation frameworks, leveraging Pytest fixtures and libraries such as to validate Azure data factory activities.

Required Skills and Qualifications:

Cloud: Microsoft Azure Data Factory

CI/CD Platforms: Azure Devops, Github

Testing Framework: Pytest

Languages: Python, SQL

Reporting: Power BI, Tableau

Version Control: Git

  • Senior Data Pipeline Specialist

    1 week ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time US$ 40,800 - US$ 70,200

    Data Engineering Position We are seeking an experienced data engineering professional to join our team and contribute to the development of large-scale data pipelines using Python, PySpark, and TypeScript. This is a long-term contract opportunity that offers the chance to work remotely and be part of a dynamic team. Key Responsibilities: Develop and...

  • Lead Data Strategist

    6 days ago

    Bareilly, Uttar Pradesh, India beBeeDataSpecialist Full time US$ 1,75,000 - US$ 2,25,000

    Data SpecialistWe are seeking a skilled Data Specialist to lead the modernization of our clients' data infrastructures, architecture, and pipelines.Key Responsibilities:Develop cloud-first strategies for core business data initiativesProvide thought leadership on technical mattersDesign and implement modernized end-to-end data strategies, including...

  • High-Performance Data Pipeline Specialist

    3 days ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Databricks Data Engineer Job DescriptionWe are seeking a highly skilled Senior Databricks Data Engineer to design, develop, and maintain scalable and high-performance data pipelines using Databricks and PySpark.The ideal candidate will have strong hands-on experience in Databricks and AWS, with a solid understanding of PySpark, CI/CD pipelines, and...

  • Senior Python Data Pipeline Specialist

    2 days ago

    Bareilly, Uttar Pradesh, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title:Freelance Data Engineer – Python\Data Engineering Expertise RequiredThreatXIntel is a cyber security company that protects businesses and organizations from cyber threats.We offer customized and affordable solutions tailored to meet the specific needs of our clients, ensuring that businesses of all sizes have access to high-quality cyber security...

  • Data Integration Specialist

    1 day ago

    Bareilly, Uttar Pradesh, India beBeeAzure Full time ₹ 8,00,000 - ₹ 15,00,000

    Job OpportunityAbout Azure Data Factory SpecialistsAzure data factory specialists are sought after by organizations to ensure smooth, scalable, and secure data flows across multiple sources and destinations. The ideal specialist will have strong hands-on expertise in Azure ADF pipelines, data orchestration, and transformations.Key ResponsibilitiesDesign,...

  • Cloud Data Ingestion Specialist

    4 days ago

    Bareilly, Uttar Pradesh, India beBeeDataIngestion Full time ₹ 15,00,000 - ₹ 20,00,000

    Cloud Data Ingestion SpecialistWe are seeking a highly skilled Cloud Data Ingestion Specialist to design, develop and optimize data ingestion pipelines for integrating multiple sources into Databricks.The ideal candidate will have expertise in CI/CD and Kubernetes, with hands-on experience in data pipelines, automation and infrastructure management. They...

  • Data Engineering Specialist

    4 days ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Engineering SpecialistWe are seeking a seasoned Data Engineer with extensive experience in designing, developing, and maintaining data pipelines and ETL processes on Databricks. The ideal candidate will have a strong background in big data ecosystems, data modeling, and ETL development.

  • Strategic Data Engineering Lead

    4 days ago

    Bareilly, Uttar Pradesh, India beBeeEngineering Full time ₹ 25,00,000 - ₹ 30,00,000

    Job DescriptionWe are seeking an experienced Engineering Manager to lead a team of talented data engineers and design scalable data pipelines and analytics platforms.The ideal candidate will have 8+ years of experience in software/data engineering with at least 3 years in a leadership or managerial role.

  • Data Extraction Specialist

    3 days ago

    Bareilly, Uttar Pradesh, India beBeeScraping Full time ₹ 10,00,000 - ₹ 16,00,000

    Job Title: Data Extraction SpecialistWe are seeking a skilled professional with expertise in web scraping and automation to join our team. As a Senior Python Developer, you will design, develop, and optimize large-scale data extraction solutions using Scrapy.The ideal candidate will have strong knowledge of Python, including experience with libraries such as...

  • Cloud Data Engineering Specialist

    2 days ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Job Title: Cloud Data Engineering SpecialistWe are seeking a skilled Cloud Data Engineering Specialist to join our team. As a key member of our data engineering team, you will collaborate with cross-functional teams to design and implement scalable data pipelines that empower business decision-making through advanced analytics.Key...