Data Engineering Specialist

6 hours ago

Belgaum, Karnataka, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

Job Title: Data Engineering Specialist

Job Description:

We are seeking an experienced Data Engineering Specialist to develop, optimize and maintain automated data pipelines supporting business intelligence initiatives. This role will focus on building production-ready workflows for data ingestion, transformation, quality checks and metadata integration, primarily using Azure Databricks, Azure Data Lake and Microsoft Purview.

The ideal candidate will have hands-on experience with PySpark, SQL and ETL/ELT pipeline design, as well as a working knowledge of Microsoft Purview for metadata management and lineage capture.

Key Responsibilities:

  • Design, build and deploy robust data pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform and curate governance and operational metadata from multiple sources landed in Databricks.
  • Implement profiling logic to capture issue-level metadata (source table, column, timestamp, severity, rule type) to support drill-down from dashboards into specific records and enable targeted remediation.
  • Develop data pipelines to generate metrics for dashboards covering data quality, lineage, job monitoring, access & permissions, query cost, usage & consumption, retention & lifecycle, policy enforcement, sensitive data mapping and governance KPIs.
  • Automate asset onboarding, metadata enrichment, classification tagging and lineage extraction for integration into governance reporting.
  • Implement logic for retention tracking and policy compliance monitoring (masking, RLS, exceptions).
  • Build pipelines to track job performance, SLA adherence and query costs for cost and performance optimization.
  • Maintain curated Delta tables for governance metrics, structured for efficient dashboard consumption.
  • Monitor pipeline execution, optimize performance and resolve issues quickly.
  • Collaborate closely with the lead engineer, QA and reporting teams to validate metrics and resolve data quality issues.
  • Ensure all pipelines meet organizational governance, privacy and security standards.
Required Skills and Qualifications:

To be successful in this role, you will need:

  • Bachelor's degree in Computer Science, Engineering, Information Systems or related field
  • 4+ years of hands-on data engineering experience, with Azure Databricks and Azure Data Lake
  • Proficiency in PySpark, SQL and ETL/ELT pipeline design
  • Demonstrated experience building granular data quality checks and integrating governance logic into pipelines
  • Working knowledge of Microsoft Purview for metadata management, lineage capture and classification
  • Experience with Azure Data Factory or equivalent orchestration tools
  • Understanding of data modeling, metadata structures and data cataloging concepts
  • Strong debugging, performance tuning and problem-solving skills
  • Ability to document pipeline logic and collaborate with cross-functional teams
Benefits:

This role offers a competitive salary and benefits package, as well as opportunities for professional growth and development.

Others:

Please note that we are an equal opportunities employer and welcome applications from all qualified candidates.

  • Data Engineering Specialist

    2 days ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 40,00,000 - ₹ 60,00,000

    Data Engineering ExpertOur organization is seeking a skilled Data Engineer to join our team and support data-driven initiatives. As a key member of the team, you will play a vital role in building scalable pipelines, working with streaming data, and collaborating with data scientists and business stakeholders to deliver high-quality solutions.The role...

  • Data Engineering Specialist

    2 days ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    As a seasoned Data Engineer, you will play a pivotal role in designing and implementing cutting-edge data pipelines that drive business growth.About the RoleThis is an exceptional opportunity for a highly skilled professional with expertise in ETL development, data engineering, and cloud-based data services to join our team.We are seeking someone who can...

  • Data Insights Specialist

    1 day ago

    Belgaum, Karnataka, India beBeeData Full time ₹ 10,00,000 - ₹ 20,00,000

    ">Data Insights Specialist">We are seeking a detail-oriented Data Insights Specialist to join our team.

  • Advanced Data Engineering Specialist

    4 hours ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time US$ 38,400 - US$ 62,400

    We are seeking an experienced Data Engineer to bring their expertise to a greenfield project initiative.Key Responsibilities:Briefly summarise the key responsibilities of the role.Requirements:At least 5-8 years of experience as a Data Engineer with strong knowledge of the following technologies:A well-rounded background in data engineering, including...

  • Machine Learning Engineer

    3 hours ago

    Belgaum, Karnataka, India beBeeExpertise Full time ₹ 18,00,000 - ₹ 25,00,000

    Machine Learning Engineer - Infrastructure SpecialistWe are seeking a seasoned engineer with 7+ years of experience in managing and maintaining machine learning infrastructure to drive the success of our team.Main Responsibilities:Design, implement, and maintain scalable and efficient machine learning pipelines using programming languages such as Python and...

  • Data Insights Specialist

    3 days ago

    Belgaum, Karnataka, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Data Insights SpecialistAbout the Opportunity:Lingaro is seeking an experienced professional to join our team. As a Data Insights Specialist, you will be responsible for extracting valuable insights from data, developing predictive models, and implementing AI-powered solutions.Key Responsibilities:Extract insights from data using Exploratory Data Analysis,...

  • Expert Data Architect

    4 hours ago

    Belgaum, Karnataka, India beBeeData Full time ₹ 13,20,000 - ₹ 2,01,60,000

    Data Architect SpecialistWe are seeking an experienced Data Engineer to join our organization. The ideal candidate should have strong expertise in building and automating data pipelines, managing multi-environment deployments, and working with modern DevOps/configuration management tools.

  • Data Engineer

    3 weeks ago

    Belgaum, Karnataka, India Petrofac Full time

    Petrofac is a leading international service provider to the energy industry, with a diverse client portfolio including many of the world's leading energy companies.We design, build, manage, and maintain infrastructure for our clients. We recruit, reward, and develop our people based on merit, regardless of race, nationality, religion, gender, age, sexual...

  • Cloud Data Engineer and Platform Specialist

    3 days ago

    Belgaum, Karnataka, India beBeeData Full time US$ 1,50,000 - US$ 1,80,000

    Job Role OverviewWe are seeking a seasoned Infrastructure and Data Engineering Expert to lead full lifecycle implementations, including secure platform setup, cloud integration, and high-performance data pipeline development.This role is ideal for those who thrive at the intersection of data engineering, infrastructure, and DevOps, and want to be part of...

  • Data Intelligence Specialist

    2 days ago

    Belgaum, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    About the RoleWe are seeking a highly skilled and relentlessly curious Data Engineer to join our team as a Marketing Platform Specialist. This role is ideal for someone who can dedicate themselves to extracting every possible byte of data from marketing platforms, uncovering hidden API endpoints and undocumented features.The MissionYour focus will be on...