Data Pipeline Specialist
4 days ago
As a specialist in ETL and Power BI testing, you will be responsible for ensuring the accuracy, integrity, and performance of data pipelines deployed into cloud environments.
The role involves developing and automating comprehensive test cases to validate data flows, transformations, and system behaviors using python scripting and Pytest framework, integrated into Continuous Integration/Continuous Deployment pipelines for Automated quality assurance.
- Design and develop manual and automated test cases for data pipelines using python and pytest.
- Validate data transformations, data mappings, and data integrity through SQL queries and data profiling.
- Automate functional, regression, and performance tests of workflows to ensure pipeline reliability.
- Collaborate with data engineers, analysts, and developers to understand data specifications and flow requirements.
- Document test plans, test cases, defects, and provide timely feedback on issues.
- Maintain test automation frameworks, leveraging Pytest fixtures and libraries to validate data factory activities.
Required Skills:
- Cloud: Microsoft Azure Data Factory
- CI/CD Platforms: Azure Devops, Github
- Testing Framework: Pytest
- Languages: Python, SQL
- Reporting: Power BI, Tableau
- Version Control: Git
-
Data Pipelines Specialist
19 hours ago
Rajahmundry, Andhra Pradesh, India beBeeDataPipelines Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Job Title: Data Pipelines SpecialistAbout the RoleWe are seeking a highly skilled data pipelines specialist to join our team. You will work on designing and building scalable, efficient data pipelines using various technologies including IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.Key Responsibilities:Design and develop data pipelines that...
-
Big Data Pipeline Specialist
1 day ago
Rajahmundry, Andhra Pradesh, India beBeeDataPipeline Full time ₹ 20,00,000 - ₹ 25,00,000Job Opportunity: Data Pipeline SpecialistOur company seeks a skilled and experienced professional to build and maintain large-scale data pipelines using AWS services.Key Responsibilities:Design, develop, and implement production-level data pipelines that handle structured and unstructured data at terabyte scale.Own the end-to-end ETL/ELT processes, ensuring...
-
Data Pipeline Architect
4 days ago
Rajahmundry, Andhra Pradesh, India beBeeDataPipeline Full time ₹ 20,00,000 - ₹ 25,00,000Key Role: Data Pipeline ArchitectWe are seeking a skilled professional to design and build robust data pipelines that efficiently curate and ingest data.Main Responsibilities:Design, implement, and maintain large-scale data pipelines for seamless data workflows.Collaborate with cross-functional teams to ensure data quality, security, and...
-
Data Pipelines Architect
2 days ago
Rajahmundry, Andhra Pradesh, India beBeeMachineLearning Full time ₹ 1,50,00,000 - ₹ 2,00,00,000Job Title:Data Pipelines ArchitectJob DescriptionWe're evolving into an AI-native platform where intelligent agents predict churn, upsell opportunities and automate member engagement.The right candidate will design and build robust data pipelines to transform raw events into usable features.He/she will also develop and train machine learning models using...
-
Senior Data Engineer(Lead)
3 days ago
Rajahmundry, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000Lead Data Architect RoleWe are seeking a highly skilled Lead Data Architect to join our team.The ideal candidate will have expertise in designing and developing scalable data pipelines using Apache Spark, AWS Glue, and Azure Data Factory.Strong collaboration skills are essential for working with cross-functional teams to build data lake and data warehouse...
-
Senior Data Pipeline Developer
4 days ago
Rajahmundry, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Data Engineering RoleWe are seeking a skilled Data Engineer to design, develop and maintain high-quality data pipelines. The ideal candidate will have experience in designing, building and scaling batch and streaming data pipelines using PySpark.
-
Key Data Pipeline Architect
6 days ago
Rajahmundry, Andhra Pradesh, India beBeeDataEngineer Full time US$ 1,20,000 - US$ 1,80,000**Job Title: Data Engineer**">This role presents an exciting opportunity to contribute to the development and maintenance of scalable data pipelines that drive business decisions.">We are seeking a highly experienced professional with a strong background in Python, Spark, and PySpark. Your expertise in designing and optimizing data pipelines, managing large...
-
Databricks Data Engineer
6 days ago
Rajahmundry, Andhra Pradesh, India beBeeDataEngineer Full time US$ 1,20,000 - US$ 1,50,000Job DescriptionWe are seeking an experienced Data Engineer to design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks.Data Ingestion Pipeline Design and Development:Cicd pipeline Implementation:Kubernetes Application Management:Data Integrity and Security:Collaboration with Data Engineers:Required Skills &...
-
Data Architect with Pipeline Expertise
1 week ago
Rajahmundry, Andhra Pradesh, India beBeeInformatica Full time ₹ 18,00,000 - ₹ 25,00,000Job Title: Data Engineer with Informatica DEIWe are seeking an experienced data engineer to design, develop and maintain large-scale data pipelines using Informatica DEI.Key Responsibilities:Design, develop and maintain large-scale data pipelines using Informatica DEI.Collaborate with cross-functional teams to identify business requirements and design data...
-
Data Engineering Specialist
4 days ago
Rajahmundry, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 80,00,000 - ₹ 1,50,00,000Job Title: Data Engineering SpecialistThis is an exciting opportunity to join our dynamic data team as a Data Engineering Specialist. We are seeking an experienced professional to design, build, and maintain robust data pipelines and ETL/ELT processes.About the Role:We are looking for a highly skilled Data Engineer who can work collaboratively with our...
