Databricks Data Engineering Lead

3 hours ago

Mumbai, Maharashtra, India Crisil Full time ₹ 8,00,000 - ₹ 24,00,000 per year

Position Overview:
The
Databricks Data Engineering Lead
role is ideal a highly skilled Databricks Data Engineer who will architect and lead the implementation of scalable, high-performance data pipelines and platforms using the Databricks Lakehouse ecosystem. The role involves managing a team of data engineers, establishing best practices, and collaborating with cross-functional stakeholders to unlock advanced analytics, AI/ML, and real-time decision-making capabilities.

Key Responsibilities:

  • Lead the design and development of modern data pipelines, data lakes, and lakehouse architectures using Databricks and Apache Spark.
  • Manage and mentor a team of data engineers, providing technical leadership and fostering a culture of excellence.
  • Architect scalable ETL/ELT workflows to process structured and unstructured data from various sources (cloud, on-prem, streaming).
  • Build and maintain Delta Lake tables and optimize performance for analytics, machine learning, and BI use cases.
  • Collaborate with data scientists, analysts, and business teams to deliver high-quality, trusted, and timely data products.
  • Ensure best practices in data quality, governance, lineage, and security, including the use of Unity Catalog and access controls.
  • Integrate Databricks with cloud platforms (AWS, Azure, or GCP) and data tools (Snowflake, Kafka, Tableau, Power BI, etc.).
  • Implement CI/CD pipelines for data workflows using tools such as GitHub, Azure DevOps, or Jenkins.
  • Stay current with Databricks innovations and provide recommendations on platform strategy and architecture improvements

Qualifications:

  • Education: Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.

  • Experience:

  • 7+ years of experience in data engineering, including 3+ years working with Databricks and Apache Spark.

  • Proven leadership experience in managing and mentoring data engineering teams.

  • Skills:

  • Proficiency in PySpark, SQL, and experience with Delta Lake, Databricks Workflows, and MLflow.

  • Strong understanding of data modeling, distributed computing, and performance tuning.
  • Familiarity with one or more major cloud platforms (Azure, AWS, GCP) and cloud-native services.
  • Experience implementing data governance and security in large-scale environments.
  • Experience with real-time data processing using Structured Streaming or Kafka.
  • Knowledge of data privacy, security frameworks, and compliance standards (e.g., PCIDSS, GDPR).

  • Exposure to machine learning pipelines, notebooks, and ML Ops practices.

  • Certifications :

  • Databricks Certified Data Engineer or equivalent certification.

  • Data Engineer- databricks

    2 weeks ago

    Mumbai, Maharashtra, India Qualintia Solutions Full time ₹ 8,00,000 - ₹ 25,00,000 per year

    Role- Data Engineer- DatabricksExp- 8+ years,Job Type- ContractualLocation- MumbaiWork Mode- WFO• Learning & Certification Opportunities: Enhance your professional growth.• Comprehensive Medical Coverage and Life Insurance: For your well-being.• Flexible Work Environment: Enjoy a 5-day work week.• Collaborative Culture: Be part of a fun, innovative...

  • Senior Data Engineer- Databricks

    2 weeks ago

    Mumbai, Maharashtra, India Koantek Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking an experienced Databricks Full Stack Data Engineer with 5–6 years of industryexperience. The ideal candidate will have a proven track record of working on live projects,preferably within the manufacturing or energy sectors. He/she will play a key role in developingand maintaining scalable data solutions using Databricks and related...

  • Data Engineer

    2 weeks ago

    Mumbai, Maharashtra, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...

  • Databricks

    3 days ago

    Mumbai, Maharashtra, India Talent Worx Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Every day, your work will make an impact that matters, while you thrive in a dynamic culture of inclusion, collaboration, and high performance. As the undisputed leader in professional services, Our Client is where you will find unrivaled opportunities to succeed and realize your full potential.We are seeking highly skilled Databricks Data Engineers to join...

  • Databricks Technical Lead

    1 day ago

    Navi Mumbai, Maharashtra, India ZIDEA Full time ₹ 4,50,000 - ₹ 6,00,000 per year

    Role: Technical Lead — Databricks (Data Platform Operations)Location:Ghansoli, Navi MumbaiMode:General shift (weekend/night support only when required)Experience:12–15 yearsImmediate joiners preferred.About the Role:We're looking for a seasoned Technical Lead who can guide a Databricks operations team, drive stability across the platform, and take charge...

  • Databricks Architect

    5 days ago

    Mumbai, Maharashtra, India Jio Full time US$ 1,20,000 - US$ 2,40,000 per year

    Title: Senior Databricks ArchitectLocation: Mumbai, Gurgaon, HyderabadAbout the RoleWe are seeking a Senior Databricks Architect to lead the design and implementation of scalable, secure, and cost-optimized data platforms on Azure using Databricks. You will own end-to-end architecture for data ingestion, transformation, governance, and analytics, and mentor...

  • Databricks Architect

    2 hours ago

    Mumbai, Maharashtra, India Jio Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the RoleWe are seeking a Senior Databricks Architect to lead the design and implementation of scalable, secure, and cost-optimized data platforms on Azure using Databricks. You will own end-to-end architecture for data ingestion, transformation, governance, and analytics, and mentor engineering teams to deliver high-quality data solutions.Key...

  • Data Engineer(Azure Databricks)

    36 minutes ago

    Navi Mumbai, Maharashtra, India Persistent Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    About Position:As a Data Engineer, you will design and manage robust data pipelines using Azure Databricks, Spark, and Kafka. You'll process large-scale datasets and implement CDC pipelines to power analytics platforms. Your role includes modeling data using the Medallion Architecture and ensuring data quality and governance. Collaboration with analytics...

  • Senior Databricks Architect

    5 days ago

    Mumbai, Maharashtra, India Jio Platforms Limited (JPL) Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Work Locations: Mumbai OR Hyderabad OR GurgaonAbout the Role:We are seeking a Senior Databricks Architect to lead the design and implementation of scalable, secure, and cost-optimized data platforms on Azure using Databricks. You will own end-to-end architecture for data ingestion, transformation, governance, and analytics, and mentor engineering teams to...

  • Senior Databricks Architect

    5 days ago

    Mumbai, Maharashtra, India Jio Full time ₹ 15,00,000 - ₹ 30,00,000 per year

    About the RoleWe are seeking a Senior Databricks Architect to lead the design and implementation of scalable, secure, and cost-optimized data platforms on Azure using Databricks. You will own end-to-end architecture for data ingestion, transformation, governance, and analytics, and mentor engineering teams to deliver high-quality data solutions.Key...