Chief Data Engineer

6 days ago

Vizag, Andhra Pradesh, India beBeeDataArchitect Full time ₹ 15,00,000 - ₹ 20,00,000

Job Summary:

We are seeking a seasoned Data Architect to join our team and lead the design, development, and implementation of scalable data pipelines that support business growth and analytics initiatives. The ideal candidate will have expertise in building production-ready workflows for ingestion, transformation, quality checks, lineage capture, access auditing, cost usage analysis, retention tracking, and metadata integration using Azure Databricks, Azure Data Lake, and Microsoft Purview.

Key Responsibilities:
  • Pipeline Development: Design, build, and deploy robust ETL/ELT pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform, and curate governance and operational metadata from multiple sources landed in Databricks.
  • Granular Data Quality Capture: Implement profiling logic to capture issue-level metadata (source table, column, timestamp, severity, rule type) to support drill-down from dashboards into specific records and enable targeted remediation.
  • Governance Metrics Automation: Develop data pipelines to generate metrics for dashboards covering data quality, lineage, job monitoring, access & permissions, query cost, usage & consumption, retention & lifecycle, policy enforcement, sensitive data mapping, and governance KPIs.
  • Microsoft Purview Integration: Automate asset onboarding, metadata enrichment, classification tagging, and lineage extraction for integration into governance reporting.
  • Data Retention & Policy Enforcement: Implement logic for retention tracking and policy compliance monitoring (masking, RLS, exceptions).
  • Job & Query Monitoring: Build pipelines to track job performance, SLA adherence, and query costs for cost and performance optimization.
  • Metadata Storage & Optimization: Maintain curated Delta tables for governance metrics, structured for efficient dashboard consumption.
  • Testing & Troubleshooting: Monitor pipeline execution, optimize performance, and resolve issues quickly.
  • Collaboration: Work closely with the lead engineer, QA, and reporting teams to validate metrics and resolve data quality issues.
  • Security & Compliance: Ensure all pipelines meet organizational governance, privacy, and security standards.
Requirements:
  • Expertise in Azure Databricks, Azure Data Lake, and Microsoft Purview.
  • Proficiency in PySpark, SQL, and Delta Lake.
  • Strong understanding of data governance, data quality, and metadata management.
  • Excellent communication and collaboration skills.

  • Chief Data Architect

    6 days ago

    Vizag, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Chief Data EngineerWe are seeking an experienced Chief Data Engineer to lead our data engineering team and develop a robust, scalable, and secure data ecosystem.Design, build, and maintain large-scale data architectures using cutting-edge technologies such as cloud platforms, big data frameworks, and real-time data streaming.This role demands deep technical...

  • Chief Data Strategist

    20 hours ago

    Vizag, Andhra Pradesh, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Chief Data StrategistKey Responsibilities:Develop and implement advanced data analytics solutions utilizing machine learning and deep learning techniques.Requirements:At least 8 years of experience in ML/DL with a technical background.Familiarity with Python and at least one of TensorFlow, Keras, or PyTorch.Proficiency in NLP libraries such as...

  • Chief Data Engineer

    20 hours ago

    Vizag, Andhra Pradesh, India beBeeDataArchitect Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job TitleAs a Data Architect, you will be responsible for designing and developing scalable data pipelines and ETL processes. This role involves collaborating with Machine Learning Engineers to integrate data solutions into machine learning models and workflows.

  • Strategic Data Manager

    3 days ago

    Vizag, Andhra Pradesh, India beBeeDataDriven Full time ₹ 18,00,000 - ₹ 25,00,000

    Chief Manager Job DescriptionThe role of Chief Manager involves analyzing large datasets to identify trends and patterns. This requires the use of statistical methods to validate findings and ensure they are reliable. The Chief Manager must also perform exploratory data analysis (EDA) to understand the data's structure and characteristics.Key...

  • Chief Data Engineer

    20 hours ago

    Vizag, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 2,00,00,000

    We are seeking a highly skilled Data Engineering professional to join our team.Job Description:As a key member of our data engineering team, you will be responsible for developing and maintaining scalable data processing systems using Azure Functions and ADF. You will also build ELT pipelines with ADF / Azure Data Services, implement transformations using...

  • Chief Data Engineering Specialist

    6 days ago

    Vizag, Andhra Pradesh, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Senior Databricks Data Engineer Position We are seeking an experienced Senior Databricks Data Engineer to join our team and play a key role in designing, developing, and maintaining scalable and high-performance data pipelines using Databricks and PySpark. Key Responsibilities: Design and develop efficient data pipelines that cater to the organization's data...

  • Chief Data Platform Engineer

    7 days ago

    Vizag, Andhra Pradesh, India beBeeDataArchitect Full time ₹ 18,00,000 - ₹ 25,50,000

    Job Overview">We're looking for an experienced Data Architect to join our team. As a key member of our data and AI organization, you will be responsible for designing and implementing scalable, secure, and efficient data platforms that meet client requirements.">Our ideal candidate has a strong background in data engineering, including data warehousing, ETL,...

  • Chief Data Engineer

    7 days ago

    Vizag, Andhra Pradesh, India beBeeDataStrategist Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Data Strategist PositionWe seek a senior data strategist to lead our data architecture, design and implement scalable data pipelines, and collaborate with stakeholders to achieve business objectives.The ideal candidate will be experienced in designing and deploying production-grade data warehouses, have expertise in data governance frameworks and data...

  • Lead Hospitality Engineer

    20 hours ago

    Vizag, Andhra Pradesh, India beBeeEngineering Full time ₹ 80,00,000 - ₹ 1,20,00,000

    Job SummaryThe Chief Engineer oversees day-to-day engineering operations in hospitality settings, ensuring technical systems function smoothly, regulatory compliance is met, and team performance is optimized.This role contributes to guest satisfaction and operational excellence through strategic planning, training, and disciplined execution of engineering...

  • Chief Data Engineering Specialist

    3 days ago

    Vizag, Andhra Pradesh, India beBeeDataModeling Full time ₹ 60,00,000 - ₹ 1,00,00,000

    Key Data Modeling PositionOur team is seeking a highly skilled and experienced Data Modeler to join our advisory team.Design, develop, and optimize scalable data pipelines using Azure Data Factory, Databricks, and Kafka.Architect and implement robust data warehousing solutions using Snowflake and other cloud-native technologies.Collaborate with business...