Senior Data Engineer

6 days ago

Dombivli, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

Job Summary

">

We are seeking a skilled Data Architect to lead the design and development of automated data pipelines supporting enterprise data governance and analytics initiatives. This role will focus on building production-ready workflows for ingestion, transformation, quality checks, lineage capture, access auditing, cost usage analysis, retention tracking, and metadata integration using Azure Databricks, Azure Data Lake, and Microsoft Purview.

">

About the Role

">

The successful candidate will be responsible for designing, building, and deploying robust ETL/ELT pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform, and curate governance and operational metadata from multiple sources landed in Databricks.

">
  • Pipeline Development: Design, build, and deploy robust ETL/ELT pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform, and curate governance and operational metadata from multiple sources landed in Databricks.
  • Granular Data Quality Capture: Implement profiling logic to capture issue-level metadata (source table, column, timestamp, severity, rule type) to support drill-down from dashboards into specific records and enable targeted remediation.
  • Governance Metrics Automation: Develop data pipelines to generate metrics for dashboards covering data quality, lineage, job monitoring, access & permissions, query cost, usage & consumption, retention & lifecycle, policy enforcement, sensitive data mapping, and governance KPIs.
  • Microsoft Purview Integration: Automate asset onboarding, metadata enrichment, classification tagging, and lineage extraction for integration into governance reporting.
  • Data Retention & Policy Enforcement: Implement logic for retention tracking and policy compliance monitoring (masking, RLS, exceptions).
  • Job & Query Monitoring: Build pipelines to track job performance, SLA adherence, and query costs for cost and performance optimization.
  • Metadata Storage & Optimization: Maintain curated Delta tables for governance metrics structured for efficient dashboard consumption.
  • Testing & Troubleshooting: Monitor pipeline execution, optimize performance, and resolve issues quickly.
  • Collaboration: Work closely with the lead engineer, QA, and reporting teams to validate metrics and resolve data quality issues.
  • Security & Compliance: Ensure all pipelines meet organizational governance, privacy, and security standards.

  • Senior Data Engineer Position

    1 day ago

    Dombivli, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 19,00,000

    Senior Data Engineer OpportunityWe are seeking a skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data systems.Key Responsibilities:Data Engineering: Design and develop high-performance data processing pipelines, data warehouses, and data...

  • Senior Data Engineer

    7 days ago

    Dombivli, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Overview">As a senior data engineering professional, you will be responsible for designing and developing scalable, secure, and high-performance data solutions on Databricks.">Key Responsibilities">Design, develop, and maintain large-scale data pipelines and ETL processes to support business growth and decision-making.Lead the architecture and...

  • Strategic Data Engineering Professional

    5 days ago

    Dombivli, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data Engineer Role OverviewWe are looking for a seasoned and proficient Senior Python Data Engineer to join our data engineering team. As a pivotal member of our team, you will play a crucial role in designing, implementing, and optimizing data pipelines.Key ResponsibilitiesDesign and Development: Design, develop, and maintain scalable and efficient...

  • Senior Data Engineering Leader

    22 hours ago

    Dombivli, Maharashtra, India beBeeSnowflake Full time ₹ 30,00,000 - ₹ 40,00,000

    Job OverviewWe are seeking an experienced Senior Snowflake Developer and Infrastructure Integration Expert to lead the implementation of full lifecycle projects.The ideal candidate will thrive at the intersection of data engineering, infrastructure, and DevOps, with a strong background in cloud integration, data security, and compliance.Key...

  • Data Engineering Leader

    3 days ago

    Dombivli, Maharashtra, India beBeeEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title:Data Engineering LeaderExperience:10-14 yearsSummary:As a Data Engineering Leader, you will lead a team of Senior Engineers and propose architecture for data warehouse design and development using AWS, Spark, Python, SQL, Databricks, and dbt.Architect and implement workflow management systems using Airflow.Design and develop cloud-native...

  • Senior Data Architect

    24 hours ago

    Dombivli, Maharashtra, India beBeeDataArchitect Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Senior Data ArchitectJob SummaryWe are seeking an experienced and highly skilled Senior Data Architect to join our team. The ideal candidate will have a strong background in data analysis, engineering, and solution design.Key ResponsibilitiesThe Senior Data Architect will be responsible for:Designing and implementing large-scale data architectures...

  • Senior Data Engineer

    6 days ago

    Dombivli, Maharashtra, India beBeeData Full time US$ 1,60,000 - US$ 1,90,000

    Job DescriptionOur organization is seeking an expert in platform engineering to lead our data observability efforts.The successful candidate will be responsible for establishing and advancing observability, monitoring, and reliability practices across our Enterprise Data & Analytics (EDAA) Platforms landscape.This role ensures end-to-end visibility into...

  • Highly Skilled Data Engineering Leader

    5 days ago

    Dombivli, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer Job DescriptionWe are seeking a highly skilled Senior Data Engineer to join our team. This role involves designing and implementing modern data warehouse solutions using Snowflake and Azure Data Factory.The successful candidate will work closely with our DW/BI leads to understand new ETL pipeline development requirements and collaborate...

  • Senior Data Engineering Specialist

    5 days ago

    Dombivli, Maharashtra, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Technical Lead - Data EngineerWe are seeking an experienced Technical Lead to drive our data engineering efforts. As a key member of our team, you will be responsible for designing and implementing data warehousing solutions that meet the needs of our business.Responsibilities:Lead the design and implementation of data warehousing solutionsDevelop and...

  • Senior Data Specialist Position

    7 days ago

    Dombivli, Maharashtra, India beBeeData Full time ₹ 9,00,000 - ₹ 12,00,000

    Senior Data Specialist  We are looking for an experienced Senior Data Specialist to join our team. The successful candidate will be responsible for designing, building and optimizing data pipelines and ETL workflows.  Key Responsibilities include:Designing, building and optimizing data pipelines and ETL workflowsManaging and processing large datasets...