Data Quality Engineer

5 days ago

Bengaluru, Karnataka, India Western Digital Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Description :

Role : Data Quality Engineer, Enterprise Data Platform

EXP : 10 Years ( Data Quality & Data governance) , AWS , SQL & Python , Cloud, Power BI

Location : Bengaluru, India

Business Function : Data Analytics

Work Location : Bangalore Hallmark Office (IBS)- LOC_WDT_IBS

Job Description :

About the Role :

We are seeking a skilled and forward-thinking Data Quality Engineer to advance the data trust, governance, and certification framework for our enterprise Data Lakehouse platform built on Databricks, Apache Iceberg, AWS (Glue, Glue Catalog, SageMaker Studio), Dremio, Atlan, and Power BI.

This role is critical in ensuring that data across Bronze (raw), Silver (curated), and Gold (business-ready) layers is certified, discoverable, and AI/BI-ready.

You will design data quality pipelines, semantic layers, and governance workflows, enabling both Power BI dashboards and Conversational Analytics leveraging LLMs (Large Language Models).

Your work will ensure that all 9 dimensions of data quality (accuracy, completeness, consistency, timeliness, validity, uniqueness, integrity, conformity, reliability) are continuously met, so both humans and AI systems can trust and use the data effectively.

ESSENTIAL DUTIES AND RESPONSIBILITIES :

Data Quality & Reliability :

- Build and maintain automated validation frameworks across Bronze ? Silver ? Gold pipelines.

- Develop tests for schema drift, anomalies, reconciliation, timeliness, and referential integrity.

- Integrate validation into Databricks (Delta Lake, Delta Live Tables, Unity Catalog) and Iceberg-based pipelines.

Data Certification & Governance :

- Define data certification workflows ensuring only trusted data is promoted for BI/AI consumption.

- Leverage Atlan and AWS Glue Catalog for metadata management, lineage, glossary, and access control.

- Utilize Icebergs schema evolution & time travel to ensure reproducibility and auditability.

Semantic Layer & Business Consumption :

- Build a governed semantic layer on gold data to support BI and AI-driven consumption.

- Enable Power BI dashboards and self-service reporting with certified KPIs and metrics.

- Partner with data stewards to align semantic models with business glossaries in Atlan.

Conversational Analytics & LLM Enablement :

- Prepare and certify datasets that fuel conversational analytics experiences.

- Collaborate with AI/ML teams to integrate LLM-based query interfaces (e.g., natural language to SQL) with Dremio, Databricks SQL, and Power BI.

- Ensure LLM responses are grounded on high-quality, certified datasets, reducing hallucinations and maintaining trust.

ML Readiness & SageMaker Studio :

- Provide certified, feature-ready datasets for ML training and inference in SageMaker Studio.

- Collaborate with ML engineers to ensure input data meets all 9 quality dimensions.

- Establish monitoring for data drift and model reliability.

Holistic Data Quality Dimensions :

- Continuously enforce all 9 dimensions of data quality

- Accuracy, Completeness, Consistency, Timeliness, Validity, Uniqueness, Integrity, Conformity, Reliability.

Qualifications :

Required :

- 5 to 10 years of experience in data engineering, data quality, or data governance roles.

- Strong skills in Python, PySpark, and SQL.

- Hands-on with Databricks (Delta Lake, Unity Catalog, Delta Live Tables) and Apache Iceberg.

- Expertise in AWS data stack (S3, Glue ETL, Glue Catalog, Athena, EMR, Redshift, SageMaker Studio).

- Experience with Power BI semantic modeling, DAX, and dataset certification.

- Familiarity with Dremio or query engines (Trino, Presto).

- Knowledge of Atlan or equivalent catalog/governance tools.

- Experience with data quality testing frameworks (Great Expectations, Deequ, Soda).

Preferred :

- Exposure to Conversational Analytics platforms or LLM-powered BI (e.g., natural language query over Lakehouse/Power BI).

- Experience integrating LLM pipelines (LangChain, OpenAI, AWS Bedrock, etc.) with enterprise data.

- Familiarity with data observability tools (Monte Carlo, Bigeye, DataDogs, Grafana).

- Knowledge of data compliance frameworks (GDPR, CCPA, HIPAA).

- Cloud certifications : AWS Data Analytics Specialty, Databricks Certified Data Engineer

  • Data Quality Engineer

    7 days ago

    Bengaluru, Karnataka, India Wenger & Watson Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Our Ai/ML engineering team ensures innovative, immersive webexperiences that delight our clients and their customers. We are always striving to balance thedemanding nature of working on cutting-edge technologies with the real-world demands ofhigh performance, high security, and accessibility. Working in collaboration with our multi-disciplinary engineering,...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...

  • Sr. Data Engineer Data Science

    7 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Req ID: 342869NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sr. Data Engineer Data Science to join our team in Bangalore, Karnātaka (IN-KA), India (IN). ...

  • Data Engineer

    7 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Collaborate with customers and stakeholders during the shift to gather requirements, clarify data needs, and resolve issues in real time. Implement data quality checks and validation routines to ensure integrity and consistency. Work with data architects and analysts to understand data models and business logic. Participate in code reviews and testing to...

  • Data Quality Engineer

    2 days ago

    Bengaluru, Karnataka, India Weekday AI Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    This role is for one of the Weekday's clientsSalary range: Rs Rs ie INR 20-25 LPA)Min Experience: 4 yearsLocation: BengaluruJobType: full-timeThe Data Quality Engineer ensures that data across the organization's platforms is accurate, consistent, well-documented, and reliable. This role focuses on implementing frameworks and processes to maintain high data...

  • Data Quality Engineer

    2 days ago

    Bengaluru, Karnataka, India Weekday AI Full time ₹ 20,00,000 - ₹ 25,00,000

    This role is for one of the Weekday's clientsSalary range: Rs Rs ie INR 20-25 LPA)Min Experience: 4 yearsLocation: BengaluruJobType: full-timeThe Data Quality Engineer ensures that data across the organization's platforms is accurate, consistent, well-documented, and reliable. This role focuses on implementing frameworks and processes to maintain high data...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Req ID: 343998NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - Sr ETL Developer (Architect) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). "Job...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 4,00,000 - ₹ 8,00,000 per year

    Req ID: 344005NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - ETL Developer to join our team in Bangalore, Karnātaka (IN-KA), India (IN). "Job Duties: Design,...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 8,00,000 - ₹ 16,00,000 per year

    Build scalable data pipelines from structured and unstructured sources Design ontologies that model business entities, relationships, and process flows Collaborate with domain SMEs and ML engineers to structure agent inputs and outputs Implement transformation, validation, and enrichment logic across diverse datasets Maintain and optimize cloud data...