Databricks Data Engineering Lead

1 day ago

Mumbai, Maharashtra, India Crisil Full time ₹ 8,00,000 - ₹ 24,00,000 per year

Position Overview:
The
Databricks Data Engineering Lead
role is ideal a highly skilled Databricks Data Engineer who will architect and lead the implementation of scalable, high-performance data pipelines and platforms using the Databricks Lakehouse ecosystem. The role involves managing a team of data engineers, establishing best practices, and collaborating with cross-functional stakeholders to unlock advanced analytics, AI/ML, and real-time decision-making capabilities.

Key Responsibilities:

  • Lead the design and development of modern data pipelines, data lakes, and lakehouse architectures using Databricks and Apache Spark.
  • Manage and mentor a team of data engineers, providing technical leadership and fostering a culture of excellence.
  • Architect scalable ETL/ELT workflows to process structured and unstructured data from various sources (cloud, on-prem, streaming).
  • Build and maintain Delta Lake tables and optimize performance for analytics, machine learning, and BI use cases.
  • Collaborate with data scientists, analysts, and business teams to deliver high-quality, trusted, and timely data products.
  • Ensure best practices in data quality, governance, lineage, and security, including the use of Unity Catalog and access controls.
  • Integrate Databricks with cloud platforms (AWS, Azure, or GCP) and data tools (Snowflake, Kafka, Tableau, Power BI, etc.).
  • Implement CI/CD pipelines for data workflows using tools such as GitHub, Azure DevOps, or Jenkins.
  • Stay current with Databricks innovations and provide recommendations on platform strategy and architecture improvements

Qualifications:

  • Education: Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.

  • Experience:

  • 7+ years of experience in data engineering, including 3+ years working with Databricks and Apache Spark.

  • Proven leadership experience in managing and mentoring data engineering teams.

  • Skills:

  • Proficiency in PySpark, SQL, and experience with Delta Lake, Databricks Workflows, and MLflow.

  • Strong understanding of data modeling, distributed computing, and performance tuning.
  • Familiarity with one or more major cloud platforms (Azure, AWS, GCP) and cloud-native services.
  • Experience implementing data governance and security in large-scale environments.
  • Experience with real-time data processing using Structured Streaming or Kafka.
  • Knowledge of data privacy, security frameworks, and compliance standards (e.g., PCIDSS, GDPR).

  • Exposure to machine learning pipelines, notebooks, and ML Ops practices.

  • Certifications :

  • Databricks Certified Data Engineer or equivalent certification.

  • Delivery Solutions Architect

    2 weeks ago

    Mumbai, Maharashtra, India Databricks Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Delivery Solutions Architect CSQ426R270 At Databricks, we are on a mission to empower our customers to solve the world's toughest data problems by utilizing the Databricks Data Intelligence Platform. As a Delivery Solutions Architect (DSA), you will play an important role during this journey. You will collaborate with our sales and field engineering...

  • Data Engineer

    6 hours ago

    Mumbai, Maharashtra, India, Maharashtra Nebula Tech Solutions Full time

    Data Engineer – Databricks Expert Remote (India-based) | Full-time | Immediate Joiners PreferredAre you passionate about building scalable data pipelines and optimizing performance with cutting-edge tools like Databricks and Apache Spark? We're looking for a seasoned Data Engineer to join our team and work on high-impact data products for a leading global...

  • Data Engineering Lead

    2 weeks ago

    Mumbai, Maharashtra, India CRISIL Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Databricks Data Engineering Lead Department MI&A Job Description Position Overview: The Databricks Data Engineering Lead role is ideal a highly skilled Databricks Data Engineer who will architect and lead the implementation of scalable, high-performance data pipelines and platforms using the Databricks Lakehouse ecosystem. The role involves...

  • Senior Data Engineer- Databricks

    2 weeks ago

    Mumbai, Maharashtra, India Koantek Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking an experienced Databricks Full Stack Data Engineer with 5–6 years of industryexperience. The ideal candidate will have a proven track record of working on live projects,preferably within the manufacturing or energy sectors. He/she will play a key role in developingand maintaining scalable data solutions using Databricks and related...

  • Databricks Engineer

    7 days ago

    Mumbai, Maharashtra, India Gretia Tech Services Private Limited Full time ₹ 80,00,000 - ₹ 2,00,00,000 per year

    Description :Key Skills & Experience : years of hands-on experience with Databricks, including job optimization and performance tuning. years in Data Engineering, with a solid foundation in data pipeline design, development, and implementation. - Expertise in PySpark for large-scale data processing and transformations. - Proven experience in ETL design...

  • Databricks Architect

    5 days ago

    Mumbai, Maharashtra, India Jio Full time ₹ 15,00,000 - ₹ 30,00,000 per year

    Title: Senior Databricks ArchitectLocation: Mumbai, Gurgaon, HyderabadAbout the RoleWe are seeking a Senior Databricks Architect to lead the design and implementation of scalable, secure, and cost-optimized data platforms on Azure using Databricks. You will own end-to-end architecture for data ingestion, transformation, governance, and analytics, and mentor...

  • Aws Databricks

    2 weeks ago

    Mumbai, Maharashtra, India People Prime Worldwide Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    As a Data Engineer with 5+ years of hands-on experience in data engineering/ETL using Databricks on AWS/Azure cloud infrastructure and functions, your key responsibilities will include: - Performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. - Implementing...

  • Data Engineer(Azure Databricks)

    1 day ago

    Navi Mumbai, Maharashtra, India Persistent Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    About Position:As a Data Engineer, you will design and manage robust data pipelines using Azure Databricks, Spark, and Kafka. You'll process large-scale datasets and implement CDC pipelines to power analytics platforms. Your role includes modeling data using the Medallion Architecture and ensuring data quality and governance. Collaboration with analytics...

  • Lead Data Engineer

    1 week ago

    Mumbai, Maharashtra, India M&G Global Services Private Limited Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are M&G Global Services Private Limited (formerly known as 10FA India Private Limited, and prior to that Prudential Global Services Private Limited). We are a fully owned subsidiary of the M&G plc group of companies, operating as a Global Capability Centre providing a range of value adding services to the Group since 2003. At M&G our purpose isto give...

  • Senior Databricks Architect

    7 days ago

    Mumbai, Maharashtra, India Jio Platforms Limited (JPL) Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Work Locations: Mumbai OR Hyderabad OR GurgaonAbout the Role:We are seeking a Senior Databricks Architect to lead the design and implementation of scalable, secure, and cost-optimized data platforms on Azure using Databricks. You will own end-to-end architecture for data ingestion, transformation, governance, and analytics, and mentor engineering teams to...