Senior Data Engineer

6 days ago

Solapur, Maharashtra, India beBeeData Full time ₹ 12,00,000 - ₹ 15,00,000

Job Overview

We are seeking a skilled professional to develop, optimize and maintain automated data pipelines supporting data governance and analytics initiatives.

Main Responsibilities
  • Pipeline Development: Design, build and deploy robust ETL/ELT pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform and curate governance and operational metadata from multiple sources landed in Databricks.
  • Granular Data Quality Capture: Implement profiling logic to capture issue-level metadata (source table, column, timestamp, severity, rule type) to support drill-down from dashboards into specific records and enable targeted remediation.
  • Governance Metrics Automation: Develop data pipelines to generate metrics for dashboards covering data quality, lineage, job monitoring, access & permissions, query cost, usage & consumption, retention & lifecycle, policy enforcement, sensitive data mapping, and governance KPIs.
  • Microsoft Purview Integration: Automate asset onboarding, metadata enrichment, classification tagging, and lineage extraction for integration into governance reporting.
  • Data Retention & Policy Enforcement: Implement logic for retention tracking and policy compliance monitoring (masking, RLS, exceptions).
  • Job & Query Monitoring: Build pipelines to track job performance, SLA adherence, and query costs for cost and performance optimization.
  • Metadata Storage & Optimization: Maintain curated Delta tables for governance metrics, structured for efficient dashboard consumption.
  • Testing & Troubleshooting: Monitor pipeline execution, optimize performance, and resolve issues quickly.
  • Collaboration: Work closely with the lead engineer, QA, and reporting teams to validate metrics and resolve data quality issues.
  • Security & Compliance: Ensure all pipelines meet organizational governance, privacy, and security standards.

About the Role: This is a critical position that requires strong technical skills, attention to detail, and excellent problem-solving abilities. The ideal candidate will have experience working with data engineering tools and technologies, as well as strong communication and collaboration skills.

Benefits: In addition to a competitive salary and benefits package, we offer opportunities for growth and development, a dynamic work environment, and a team of talented professionals who are passionate about data engineering.

  • Senior Data Engineer

    6 days ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Senior Data EngineerAbout the RoleWe are seeking a highly skilled and experienced Senior Data Engineer to drive the technical depth, accuracy, and instructional value of our training content.This role is responsible for defining and validating learning content aligned to industry-recognized certification paths, providing technical guidance on data...

  • Senior Data Engineer Specialist

    5 days ago

    Solapur, Maharashtra, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,25,00,000

    Job Title: Senior Data Engineering SpecialistOverviewWe are seeking a highly skilled and experienced Data Engineer to join our team. As a Senior Data Engineer, you will play a key role in designing, developing, and maintaining our data infrastructure.Key Responsibilities:Design and Develop Scalable Data Pipelines: Utilize PySpark to create efficient and...

  • Cloud Architect

    6 days ago

    Solapur, Maharashtra, India beBeeData Full time ₹ 18,00,000 - ₹ 32,00,000

    Senior Data Engineer RoleWe are seeking an experienced Senior Data Engineer to join our team. As a key member of the organization, you will be responsible for designing, implementing, and maintaining large-scale data systems.

  • Chief Data Engineering Specialist

    5 days ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Data Engineer OpportunityWe are seeking a skilled Senior Data Engineer to help design and implement an application for a large retailer.The ideal candidate will have hands-on coding expertise in Python and Pyspark, with strong experience in Telemetry, Big Data, and backend engineering.Domain experience in IoT systems, HVAC/refrigeration equipment, and...

  • Senior Data Engineering Project Coordinator

    1 day ago

    Solapur, Maharashtra, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Overview:We are seeking an experienced Senior Data Engineering Project Coordinator to join our team. As a key member of our Agile development team, you will play a pivotal role in ensuring the successful implementation of data engineering projects.About the Role:This is a full-time position with a 5-day work-from-office schedule. The shift timings are...

  • Data Analyst

    1 week ago

    Solapur, Maharashtra, India Quant-data Full time

    We're Hiring: Data Analyst (Credit Analytics | Remote | Full-Time) Leverage financial and credit data to power AI-driven lending solutionsWe are seeking a skilled Data Analyst (Credit Analytics) with 4+ years of experience to join our AI-driven lending platform team. In this role, you will analyze financial, customer, and transaction datasets, generate...

  • Senior Data Engineering Leadership Position

    3 days ago

    Solapur, Maharashtra, India beBeeDataEngineering Full time US$ 1,50,000 - US$ 2,00,000

    **Senior Data Engineering Leadership Role**This position requires a highly skilled engineer to lead the development and operation of production-grade data pipelines.The successful candidate will be responsible for providing technical direction, designing and building reliable batch and change-driven pipelines, and handling schema evolution and failures...

  • Strategic Data Engineering Leadership

    1 day ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 19,00,000

    Senior Data EngineerWe are seeking an experienced Senior Data Engineer to join our organization. As a key member of the team, you will be responsible for designing, developing, and maintaining large-scale data processing systems.The ideal candidate will have a strong background in Python programming and experience working with cloud-based technologies such...

  • Senior Data Engineer T500-18489

    3 weeks ago

    Solapur, Maharashtra, India ANSR Full time

    ANSR is hiring for one of its client.About Albertsons Companies India:At Albertsons Companies India, we're not just pushing the boundaries of technology and retail innovation, we're cultivating a space where ideas flourish and careers thrive. Our workplace in India is a vital extension of the Albertsons Companies Inc. workforce and important to the next...

  • Master Data Engineer

    4 days ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer Job DescriptionAt our organization, we are seeking a Senior Data Engineer to lead the design, development, and operation of scalable data pipelines that support real-time clinical workflows. The ideal candidate will have deep expertise in building event-driven systems and integrating healthcare data with EHR/EMR systems such as...