Lead Data Solutions Architect

4 days ago

Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000
Job Overview:

We are seeking a skilled Data Engineer to develop and maintain automated data pipelines supporting business intelligence initiatives. This role focuses on building production-ready workflows for data ingestion, transformation, quality checks, lineage capture, access auditing, cost analysis, retention tracking, and metadata integration.

Key Responsibilities:

  1. Pipeline Development: Design, build, and deploy robust ETL/ELT pipelines using Azure Databricks, Azure Data Lake, and Microsoft Purview to ingest, transform, and curate governance and operational metadata.
  2. Granular Data Quality Capture: Implement profiling logic to capture issue-level metadata to support drill-down from dashboards into specific records and enable targeted remediation.
  3. Governance Metrics Automation: Develop data pipelines to generate metrics for dashboards covering data quality, lineage, job monitoring, access & permissions, query cost, usage & consumption, retention & lifecycle, policy enforcement, sensitive data mapping, and governance KPIs.
  4. Microsoft Purview Integration: Automate asset onboarding, metadata enrichment, classification tagging, and lineage extraction for integration into governance reporting.
  5. Data Retention & Policy Enforcement: Implement logic for retention tracking and policy compliance monitoring (masking, RLS, exceptions).
  6. Job & Query Monitoring: Build pipelines to track job performance, SLA adherence, and query costs for cost and performance optimization.
  7. Metadata Storage & Optimization: Maintain curated Delta tables for governance metrics, structured for efficient dashboard consumption.
  8. Testing & Troubleshooting: Monitor pipeline execution, optimize performance, and resolve issues quickly.
  9. Collaboration: Work closely with the engineering, QA, and reporting teams to validate metrics and resolve data quality issues.
  10. Security & Compliance: Ensure all pipelines meet organizational governance, privacy, and security standards.

Requirements:

  • Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
  • 4+ years of hands-on data engineering experience, with Azure Databricks and Azure Data Lake.
  • Proficiency in PySpark, SQL, and ETL/ELT pipeline design.
  • Demonstrated experience building granular data quality checks and integrating governance logic into pipelines.
  • Working knowledge of Microsoft Purview for metadata management, lineage capture, and classification.
  • Experience with Azure Data Factory or equivalent orchestration tools.
  • Understanding of data modeling, metadata structures, and data cataloging concepts.
  • Strong debugging, performance tuning, and problem-solving skills.
  • Ability to document pipeline logic and collaborate with cross-functional teams.

Preferred Qualifications:

  • Microsoft certification in Azure Data Engineering.
  • Experience in governance-heavy or regulated environments (e.g., finance, healthcare).
  • Exposure to Power BI or other BI tools as a data source consumer.
  • Familiarity with DevOps/CI-CD for data pipelines in Azure.
  • Experience integrating both cloud and on-premises data sources into Azure.

  • Senior Data Solutions Architect

    4 days ago

    Allahabad, Uttar Pradesh, India beBeeDataArchitect Full time ₹ 35,00,000 - ₹ 40,00,000

    Job DescriptionWe are seeking an experienced Technical Architect to lead our data engagements on GCP Cloud.The ideal candidate will have more than 8 years of experience in technical, solutioning, and analytical roles, as well as 5+ years of experience in building and managing data lakes, data warehouses, data integration, data migration, and business...

  • Lead Cloud Solutions Architect

    3 days ago

    Allahabad, Uttar Pradesh, India beBeeCloudVisionary Full time ₹ 20,00,000 - ₹ 25,00,000

    Cloud Visionary Job SummaryThe Cloud Architect will design, deploy and maintain scalable cloud applications using leading cloud practices and architectural approaches. This includes consulting on integration strategies, enterprise architectures, platforms and application infrastructure.Key ResponsibilitiesCollaborate with clients to design cloud solutions...

  • Data Solutions Architect

    1 day ago

    Allahabad, Uttar Pradesh, India beBeeFabric Full time ₹ 18,00,000 - ₹ 25,00,000

    Design Analytics PlatformsIn a rapidly evolving data landscape, we're seeking a skilled Fabric Architect to craft innovative analytics solutions in the Microsoft ecosystem. This role is an opportunity to shape the future of data-driven insights and empower teams to deliver high-quality results.Key Responsibilities:Develop scalable Lakehouse, Data Warehouse,...

  • Cloud Solution Architect

    5 days ago

    Allahabad, Uttar Pradesh, India beBeeSolution Full time US$ 1,50,000 - US$ 2,00,000

    Job Title :: Cloud Solutions ArchitectLocation :: RemoteDomain experience :: Electrification, AI/MLExperience :: Minimum 10 years relevant experienceJob Description:We are seeking a cloud solutions architect to lead the design and implementation of scalable, high-performance Salesforce-based solutions. The ideal candidate will bring deep experience in...

  • Chief Data Architect

    1 day ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time US$ 1,08,300 - US$ 1,64,441

    Senior Data Engineer PositionThe Mastermind Group is seeking a seasoned professional to lead the development and implementation of intelligent data infrastructure. This role requires designing robust ETL pipelines, building scalable business intelligence solutions, and driving data-powered outcomes that enhance client value across telecom operations and IT...

  • Customer Data Solutions Architect

    3 days ago

    Allahabad, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job TitleWe are seeking a seasoned Customer Data Solutions Architect to design, implement and support cutting-edge data integration systems.Design, configure and implement customer data solutions in real-time.Build and customize pages, business entities, field groups, hierarchies and workflows for optimal data management.Configure and execute data quality...

  • Leading Data Architect

    3 days ago

    Allahabad, Uttar Pradesh, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title:Data EngineerJob Description: Overview:We are seeking a skilled Data Engineer to join our team. The ideal candidate will have a strong understanding of data modeling, ETL/ELT processes, and data governance.Key Responsibilities:Design and oversee the architecture of Microsoft Fabric solutions.Lead a team of data engineers, BI developers, and...

  • Innovative Data Solutions Expert

    6 days ago

    Allahabad, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,01,00,000

    Data Solutions ArchitectJob Overview:We are seeking a highly experienced Data Solution Architect to drive the design and delivery of cutting-edge data and analytics solutions. This pivotal role requires a deep understanding of modern data platforms, cloud architectures, and end-to-end solution delivery in complex environments.Responsibilities:Collaborate...

  • Senior Data Architect

    4 weeks ago

    Allahabad, Uttar Pradesh, India ValueLabs Full time

    Dear Aspirants,We at ValueLabs have an immediate requirement for Data Architect. Below is the Job scope for the same.Role: Data ArchitectPreferable immediate to 15 days joinersLocation: Hyderabad/RemoteRoles and Responsibilities:Overall 8+ years of IT Experience and also couple of years as Azure Data ArchitectStrong expertise in all aspects of Azure services...

  • Cloud Solutions Architect

    3 days ago

    Allahabad, Uttar Pradesh, India beBeeCloudSolutions Full time ₹ 20,00,000 - ₹ 25,00,000

    Cloud Solutions Architect RoleWe are seeking a skilled Cloud Solutions Architect to lead technical onboarding, design cloud-native solutions and deliver professional services.Key Responsibilities:Deliver workshops, demos and training to customers.Collaborate with product and engineering teams for customer experience improvement and feature feedback.Provide...