Principal Data Specialist

7 days ago

Tiruchi, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000
Job Summary

We are seeking a highly skilled Data Engineer to develop and maintain automated data pipelines that support business intelligence and analytics initiatives.

  • Pipeline Development: Design, build, and deploy robust ETL/ELT pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform, and curate governance and operational metadata from multiple sources landed in Databricks.
  • Granular Data Quality Capture: Implement profiling logic to capture issue-level metadata (source table, column, timestamp, severity, rule type) to support drill-down from dashboards into specific records and enable targeted remediation.
  • Governance Metrics Automation: Develop data pipelines to generate metrics for dashboards covering data quality, lineage, job monitoring, access & permissions, query cost, usage & consumption, retention & lifecycle, policy enforcement, sensitive data mapping, and governance KPIs.
  • Microsoft Purview Integration: Automate asset onboarding, metadata enrichment, classification tagging, and lineage extraction for integration into governance reporting.
  • Data Retention & Policy Enforcement: Implement logic for retention tracking and policy compliance monitoring (masking, RLS, exceptions).
  • Job & Query Monitoring: Build pipelines to track job performance, SLA adherence, and query costs for cost and performance optimization.
  • Metadata Storage & Optimization: Maintain curated Delta tables for governance metrics, structured for efficient dashboard consumption.
  • Testing & Troubleshooting: Monitor pipeline execution, optimize performance, and resolve issues quickly.
  • Collaboration: Work closely with the lead engineer, QA, and reporting teams to validate metrics and resolve data quality issues.
  • Security & Compliance: Ensure all pipelines meet organizational governance, privacy, and security standards.

Requirements

  • Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
  • 4+ years of hands-on data engineering experience, with Azure Databricks and Azure Data Lake.
  • Proficiency in PySpark, SQL, and ETL/ELT pipeline design.
  • Demonstrated experience building granular data quality checks and integrating governance logic into pipelines.
  • Working knowledge of Microsoft Purview for metadata management, lineage capture, and classification.
  • Experience with Azure Data Factory or equivalent orchestration tools.
  • Understanding of data modeling, metadata structures, and data cataloging concepts.
  • Strong debugging, performance tuning, and problem-solving skills.
  • Ability to document pipeline logic and collaborate with cross-functional teams.

  • Principal Design Verification Specialist

    6 days ago

    Tiruchi, Tamil Nadu, India beBeeVerification Full time US$ 8,00,000 - US$ 12,00,000

    Job Title: Principal Design Verification SpecialistWe are seeking a highly skilled Design Verification Engineer to join our team. As a key member of our design verification team, you will be responsible for verifying the functionality and performance of digital designs.Responsibilities:Verify the functionality and performance of digital designs.Collaborate...

  • Data Transformation Specialist

    2 weeks ago

    Tiruchi, Tamil Nadu, India beBeeTransformation Full time ₹ 3,00,000

    Data Transformation SpecialistJob Title: Data Transformation SpecialistJob Overview:As a data transformation specialist, you will be responsible for converting and transforming data into a standardized format.Main Responsibilities:Converting data from various formats into XML using data integration technologies.Ensuring that the converted data is accurate...

  • Data Analysis Specialist

    6 days ago

    Tiruchi, Tamil Nadu, India beBeedata Full time ₹ 7,50,000 - ₹ 15,00,000

    Data Analysis SpecialistWe are seeking a skilled Data Analysis Specialist to join our team.

  • Data Engineering Specialist

    1 day ago

    Tiruchi, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,20,00,000 - ₹ 1,50,00,000

    Job Title:Data Engineering SpecialistA highly skilled Data Engineering Specialist is sought after to lead the design and implementation of scalable data pipelines using Microsoft Fabric, Azure Synapse, and other cutting-edge technologies.

  • Data Transformation Specialist

    1 week ago

    Tiruchi, Tamil Nadu, India beBeeDataTransformation Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Transformation SpecialistWe are seeking a highly skilled professional to join our team as a Data Transformation Specialist. As a key member of our organization, you will be responsible for designing and implementing data transformation processes that drive business value.

  • Data Insights Specialist

    2 days ago

    Tiruchi, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Unlock Your Potential as a Data Insights SpecialistAbout the RoleThis is a full-time remote position for a Data Insights Specialist responsible for analyzing complex data sets, generating actionable insights, and reporting findings to stakeholders. Key tasks include:Data collection and cleaningData modeling and statistical analysisCreating visualizations to...

  • Data Insights Specialist

    2 days ago

    Tiruchi, Tamil Nadu, India beBeeDataInsights Full time ₹ 15,00,000 - ₹ 20,00,000

    **Job Title:** Data Insights Specialist"> ">About the Role:">We are seeking a skilled and analytical professional to join our data team in production technology. As a Data Insights Specialist, you will be responsible for transforming raw data into meaningful insights that inform strategic business decisions across the organization.">This role requires...

  • Data Engineering Specialist

    2 days ago

    Tiruchi, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,20,00,000 - ₹ 1,80,00,000

    Job Title: Data Engineering SpecialistWe are seeking a skilled Data Engineering Specialist to join our team. As a key member of our digital solutions group, you will play a critical role in designing and building robust data pipelines for processing channel activity data from multiple sources.The ideal candidate will possess strong knowledge of cloud...

  • Cloud Data Specialist

    2 days ago

    Tiruchi, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    We're seeking a skilled cloud data specialist to design, build and maintain scalable data pipelines and ETL solutions in the cloud.Key Responsibilities:Design and implement scalable data pipelines using PySpark and Python.Collaborate with data analysts, data scientists, and engineering teams to support data-driven initiatives.Work closely with...

  • Data Conversion Specialist

    6 days ago

    Tiruchi, Tamil Nadu, India beBeeDataConversion Full time ₹ 3,00,000

    Job Title: Data Conversion SpecialistWe are seeking a highly skilled Data Conversion Specialist to join our team. The ideal candidate will have experience in converting data from various formats into XML.Key Responsibilities:Converting data from various formats into XMLUsing XML technologies for transformation and integrationEnsuring that the converted data...