Senior Data Engineer Position

2 days ago

Jodhpur, Rajasthan, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

Data Engineering Opportunity

We are seeking a skilled data engineer to design, build and maintain automated data pipelines that support business growth and analytics initiatives. This role will focus on building production-ready workflows for ingestion, transformation, quality checks, lineage capture, access auditing, cost usage analysis, retention tracking and metadata integration using Azure Databricks, Azure Data Lake and Microsoft Purview.

About the Role

The ideal candidate will have strong experience in data engineering with a focus on Azure and Databricks. They will be responsible for designing, building and deploying robust ETL/ELT pipelines to ingest, transform and curate governance and operational metadata from multiple sources landed in Databricks.

Key Responsibilities

  • Create and deploy complex data pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform and curate governance and operational metadata from multiple sources landed in Databricks.
  • Implement profiling logic to capture issue-level metadata (source table, column, timestamp, severity, rule type) to support drill-down from dashboards into specific records and enable targeted remediation.
  • Develop data pipelines to generate metrics for dashboards covering data quality, lineage, job monitoring, access & permissions, query cost, usage & consumption, retention & lifecycle, policy enforcement, sensitive data mapping, and governance KPIs.
  • Automate asset onboarding, metadata enrichment, classification tagging and lineage extraction for integration into governance reporting.
  • Implement logic for retention tracking and policy compliance monitoring (masking, RLS, exceptions).
  • Build pipelines to track job performance, SLA adherence, and query costs for cost and performance optimization.
  • Maintain curated Delta tables for governance metrics, structured for efficient dashboard consumption.
  • Monitor pipeline execution, optimize performance, and resolve issues quickly.
  • Evaluate all pipelines meet organizational governance, privacy, and security standards.

Requirements

  • Bachelor's degree in Computer Science, Engineering, Information Systems or related field.
  • 4+ years of hands-on data engineering experience, with Azure Databricks and Azure Data Lake.
  • Proficiency in PySpark, SQL, and ETL/ELT pipeline design.
  • Demonstrated experience building granular data quality checks and integrating governance logic into pipelines.
  • Working knowledge of Microsoft Purview for metadata management, lineage capture, and classification.
  • Experience with Azure Data Factory or equivalent orchestration tools.
  • Understanding of data modeling, metadata structures, and data cataloging concepts.
  • Strong debugging, performance tuning, and problem-solving skills.
  • Ability to document pipeline logic and collaborate with cross-functional teams.

PREFERRED QUALIFICATIONS:

  • Microsoft certification in Azure Data Engineering.
  • Experience in governance-heavy or regulated environments (e.g., finance, healthcare, hospitality).
  • Exposure to Power BI or other BI tools as a data source consumer.
  • Familiarity with DevOps/CI-CD for data pipelines in Azure.
  • Experience integrating both cloud and on-premises data sources into Azure.

  • Azure Data Engineer Senior Test Manager Position

    10 hours ago

    Jodhpur, Rajasthan, India beBeeTest Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Test Lead/Manager Job Overview:This is a full-time remote position for a senior test professional in Azure Data Engineering. The successful candidate will oversee the entire testing lifecycle, develop strategic plans for testing, coordinate with various teams, manage test execution, and ensure the quality of Azure Data solutions.

  • Data Engineering Leader

    8 hours ago

    Jodhpur, Rajasthan, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe are seeking a highly skilled data engineer to design and implement scalable, high-performance data pipelines. This role involves building and optimizing ETL/ELT workflows, managing big data environments, and ensuring seamless data availability for analytics, AI/ML, and business intelligence use cases.The ideal candidate will...

  • Senior Big Data Engineer

    1 day ago

    Jodhpur, Rajasthan, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Data Engineering PositionWe are seeking an experienced Data Engineer to join our team. As a key member of our data engineering team, you will play a critical role in designing, developing, and maintaining our data pipelines.

  • Senior Data Engineering Specialist

    2 days ago

    Jodhpur, Rajasthan, India beBeeDataEngineering Full time ₹ 60,00,000 - ₹ 90,00,000

    Senior Data Engineering SpecialistWe are seeking an experienced Data Engineering professional to lead the design, development, and maintenance of large-scale data processing systems.The ideal candidate will have a strong background in Palantir Foundry, Pyspark, and Typescript, with a proven track record of delivering high-quality solutions.A collaborative...

  • Data Engineering Leadership Position

    3 days ago

    Jodhpur, Rajasthan, India beBeeLeadership Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Lead Data EngineerWe are seeking a seasoned data engineer to lead our team of talented professionals in designing and delivering scalable data pipelines and analytics platforms.The ideal candidate will have deep expertise in Big Data platforms, Microsoft Azure, and Databricks, with 8+ years of experience in software/data engineering and at least 3 years in a...

  • Senior Data Engineer Manager

    1 day ago

    Jodhpur, Rajasthan, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Technical Lead - Data Engineer PositionWe are seeking an experienced Technical Lead to oversee the development and implementation of our data engineering projects. This role will be responsible for leading a team of engineers and ensuring that our data solutions meet the highest standards of quality and reliability.Must Have Skills:8+ years of experience in...

  • Cloud Data Systems Engineer

    24 hours ago

    Jodhpur, Rajasthan, India beBeeDataSystems Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title:Senior Cloud Data Systems EngineerJob Summary:Our team seeks a Senior Cloud Data Systems Engineer to design, develop, and maintain large-scale data systems. This role requires a deep understanding of cloud technologies, such as Apache Ray, AWS, and GCP.Key Responsibilities:Operate, monitor, and troubleshoot all aspects of production and...

  • Data Engineer

    2 weeks ago

    Jodhpur, Rajasthan, India Firstsource Full time

    Key Skills and Responsibilities:We are seeking a Senior Data Engineer with a strong background in cloud-native data engineering, primarily on Microsoft Azure, and familiarity with AWS and GCP. The ideal candidate will have deep expertise in building scalable data pipelines, implementing enterprise-grade data governance, security, and AI-powered engineering...

  • AWS Cloud Data Engineering Lead

    1 day ago

    Jodhpur, Rajasthan, India beBeeDataEngineering Full time ₹ 25,00,000 - ₹ 40,00,000

    We are seeking a seasoned professional to lead our data engineering efforts on AWS cloud infrastructure.As a Senior AWS Data Engineer, you will design, develop, and deploy end-to-end data pipelines using Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift. Your expertise in Apache Spark and SQL will enable us to deliver actionable insights and drive...

  • Chief Data Engineer

    2 days ago

    Jodhpur, Rajasthan, India beBeeDatabaseDeveloper Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Senior Database DeveloperJob SummaryWe are seeking a seasoned database developer to spearhead Microsoft-based database engineering projects.The ideal candidate should possess expertise in Transact-SQL development, data warehousing, and ETL processes.Key Responsibilities:Design and develop databases using Microsoft tools and technologies.Implement...