Lead Data Pipeline Developer

3 days ago

Gulbarga, Karnataka, India beBeeDataGovernance Full time ₹ 20,00,000 - ₹ 25,00,000

Job Role:

Azure Data Engineer - Data Pipelines & Governance

About the Position

We are seeking a skilled Azure Data Engineer to develop, optimize, and maintain automated data pipelines in support of our data governance and analytics initiatives.

Main Responsibilities:
  • Pipeline Development - Design, build, and deploy robust ETL/ELT pipelines in Databricks (PySpark, SQL, Delta Lake) for ingestion, transformation, and quality checks.
  • Data Quality Capture - Implement profiling logic to capture issue-level metadata to support drill-down into specific records and enable targeted remediation.
  • Governance Metrics Automation - Develop data pipelines to generate metrics for dashboards covering data quality, lineage, job monitoring, access, permissions, query cost, usage, consumption, retention, policy enforcement, sensitive data mapping, and governance KPIs.
  • Mission-Critical Integration - Automate asset onboarding, metadata enrichment, classification tagging, and lineage extraction for integration into governance reporting.
  • Data Retention & Compliance - Implement logic for retention tracking and policy compliance monitoring.
  • Performance Monitoring - Build pipelines to track job performance, SLA adherence, and query costs for optimization.
  • Metadata Optimization - Maintain curated Delta tables for governance metrics.
  • Troubleshooting - Monitor pipeline execution, optimize performance, and resolve issues quickly.
  • Collaboration - Work closely with engineers, QA, and reporting teams to validate metrics and resolve data quality issues.

  • Lead Data Pipeline Architect

    3 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 1,20,00,000 - ₹ 1,60,00,000

    About the RoleWe are seeking an experienced Data Engineer to lead our data pipeline architecture and collaborate with data scientists and analysts.

  • Data Pipeline Specialist

    3 days ago

    Gulbarga, Karnataka, India beBeeValidator Full time ₹ 20,00,000 - ₹ 25,00,000

    Job OverviewA Data Pipeline Validator is responsible for ensuring the integrity and quality of data pipelines. This role requires a strong understanding of data warehouse testing methodologies and ETL concepts.Responsibilities:Develop and execute test plans to validate data quality and integrityCollaborate with cross-functional teams to identify and...

  • Data Engineer

    2 days ago

    Gulbarga, Karnataka, India beBeeDatastage Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: ETL Lead with IBM DataStageAbout the RoleWe are seeking a motivated and talented ETL developer to join our data engineering team. As an ETL lead, you will work on building scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.Key Responsibilities:Design, develop, and maintain ETL pipelines using...

  • Scalable Data Pipeline Expert

    3 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data Engineer RoleWe are seeking a seasoned Senior Data Engineer to spearhead the design, development, and maintenance of scalable data pipelines using PySpark. This is an exceptional opportunity for someone with strong expertise in PySpark and hands-on experience in building robust data pipelines.

  • Senior Data Pipeline Architect

    5 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Unlock the Power of Data EngineeringWe're seeking an accomplished Data Engineer to champion data-driven initiatives and deliver high-quality solutions.This pivotal role involves designing, building, and optimizing scalable pipelines using Spark, PySpark, and SQL, collaborating with data scientists and business stakeholders to drive business value.Design and...

  • Senior Data Pipeline Specialist

    3 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 1,60,00,000

    Key Roles and Responsibilities:Data Engineers are required to design, develop, and maintain ETL/ELT data pipelines. They must be able to ingest data from various sources into our data warehouse.This includes databases and flat files. The ideal candidate will have experience with AWS Cloud services, including Lambda, DMS, and Glue.A solid understanding of...

  • Financial Data Pipeline Specialist

    2 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title:Data Engineer for Financial and Crypto DomainsDescription:We are seeking a skilled data engineer to design, build, and maintain robust data pipelines for high-volume financial data. This role involves collaborating with cross-functional teams to provide clean, structured, and scalable datasets.Design end-to-end data pipelines for stocks, crypto,...

  • Chief Data Pipeline Architect

    5 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a candidate who is comfortable working on shifts that align with our global operations.Data Engineering OpportunitiesDevelop and maintain data pipelines using advanced frameworks to ensure seamless data flow.Critical Data TransformationsImplement high-touch data transformations for integration to meet client needs.Data Ingestion and...

  • Expert Cloud Data Pipeline Engineer

    2 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Cloud Data Integration SpecialistWe are seeking an experienced professional to support our data integration and ETL needs.The ideal candidate will have strong hands-on expertise in designing, building, and maintaining scalable and secure data pipelines across multiple sources and destinations.Design and Build Pipelines: Create efficient Azure Data Factory...

  • Senior Data Pipelining Expert

    3 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 60,00,000 - ₹ 1,00,00,000

    About this RoleWe're seeking a Data Transformation Specialist to build robust pipelines that transform large, messy data into high-quality structured data.