
Data Pipeline Engineer
5 days ago
This role involves ensuring the accuracy, integrity, and performance of data pipelines deployed into cloud environments.
The Senior Data Pipeline Engineer will be responsible for designing and developing comprehensive test cases to validate data transformations, mappings, and system behaviors using Python scripting and Pytest framework, integrated into CI/CD pipelines for Automated quality assurance.
Roles and Responsibilities:
- Design and develop and execute manual and automated test cases for data pipelines using Python and Pytest.
- Validate the data transformations, data mappings, and data integrity through SQL queries and data profiling.
- Automate functional, regression, and performance tests of data workflows to ensure pipeline reliability.
- Collaborate with data engineers, analysts, and developers to understand data flow requirements.
- Document Test plans, Test Cases, defects, and provide timely feedback on issues.
- Maintain Test automation frameworks, leveraging Pytest fixtures and libraries such as to validate data activities.
Required Skills:
- Cloud: Microsoft Azure
- CI/CD Platforms: Azure Devops, Github
- Testing Framework: Pytest
- Languages: Python, SQL
- Reporting: Power BI, Tableau
- Version Control: Git
Benefits: A chance to work on challenging projects, opportunities for professional growth, and a collaborative work environment.
Others: We are looking for a highly motivated individual who is passionate about technology and willing to learn and adapt quickly.
-
Chief Data Pipeline Architect
5 days ago
Nashik, Maharashtra, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,00,000Key Data Engineering Roles and ResponsibilitiesWe are seeking a highly skilled data engineer to join our team. The ideal candidate will be responsible for designing, developing, and maintaining large-scale data pipelines using Python and AWS technologies.The data engineer will work closely with cross-functional teams to develop and implement efficient ETL...
-
Data Pipeline Specialist
5 days ago
Nashik, Maharashtra, India beBeeDeveloper Full time ₹ 15,00,000 - ₹ 20,00,000ETL Developer PositionWe are seeking a skilled ETL developer to join our data engineering team.Job Description:The successful candidate will design, develop and maintain ETL pipelines using IBM Data Stage (CP4 D) and AWS Glue/Lambda for ingestion from various sources such as flat files, APIs, Oracle, DB2, etc.Key Responsibilities include:Designing scalable...
-
Expert Data Pipeline Specialist
7 days ago
Nashik, Maharashtra, India beBeeDatabricksEngineer Full time ₹ 1,20,00,000 - ₹ 2,50,00,000Job Title: Databricks EngineerWe are seeking a skilled Databricks Engineer to join our team.A successful candidate will have:7+ years of experience in data engineering, with a strong focus on designing and developing data pipelines and ETL processes on Databricks.Proven subject matter expertise in Databricks (including Databricks SQL, Delta Lake, and...
-
Principal Data Pipeline Developer
19 hours ago
Nashik, Maharashtra, India beBeeData Full time ₹ 12,00,000 - ₹ 21,00,000About the RoleThis position involves designing, developing, and maintaining data pipelines and infrastructure.Data engineers collaborate with data scientists, analysts, and other stakeholders to ensure data is accurate, timely, and accessible.
-
Senior Data Pipeline Specialist
6 days ago
Nashik, Maharashtra, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Data Engineer Job OpportunityDeloitte presents a unique chance to work with cutting-edge leaders and organizations shaping the future of the region and beyond.As a skilled Data Engineer, you will play a critical role in supporting analytics, reporting, and data science initiatives by delivering high-quality, reliable, and performant data systems.Key...
-
Data Pipeline Developer
13 hours ago
Nashik, Maharashtra, India beBeeDataPipelineDeveloper Full time ₹ 18,00,000 - ₹ 25,00,000Job Title: Data Pipeline DeveloperKey Responsibilities:Design and develop scalable data pipelines using cloud-based services with a solid understanding of data patterns, Python, and SQL.Orchestrate data workflows ensuring seamless integration and automation across data services.Utilize cloud infrastructure components for building and managing data lakes,...
-
Data Pipeline Specialist
6 days ago
Nashik, Maharashtra, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000Job DescriptionWe are seeking a skilled professional to design and implement robust data pipelines using Python and SQL.Main Responsibilities:Design, implement, and manage continuous integration/continuous deployment (CI/CD) pipelines using GitHub Actions or equivalent tools.Configure and maintain GitHub branch protection rules and workflows to ensure smooth...
-
Chief Data Pipeline Architect
13 hours ago
Nashik, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000Key Data Pipelines ExpertiseTo succeed in this challenging role, you will need to have a deep understanding of designing and developing scalable data pipelines that power business insights. Main Responsibilities:Develop ETL/ELT pipelines using Snowflake Tasks, Streams, and Stored Procedures to ensure seamless data integration.Create optimized data models...
-
Cloud Based Data Pipeline Architect
6 days ago
Nashik, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000Job Title:Lead Data EngineerWe are seeking a skilled Lead Data Engineer to join our team. In this role, you will be responsible for designing, developing, and implementing large-scale data pipelines using Azure services such as Data Factory, Databricks, and Data Lake.Key Responsibilities:Develop complex data pipelines utilizing PySpark and SQL.Implement...
-
High Performance Data Pipeline Specialist
6 days ago
Nashik, Maharashtra, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer with strong hands-on experience in Databricks and AWS to join our dynamic data engineering team.Main Responsibilities:Designing, developing, and maintaining scalable and high-performance data pipelines using Databricks and PySparkImplementing and managing data solutions on AWS,...