Data Engineering Lead

3 days ago

Mumbai, Maharashtra, India CRISIL Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Databricks Data Engineering Lead

Department

MI&A

Job Description

Position Overview:

The Databricks Data Engineering Lead role is ideal a highly skilled Databricks Data Engineer who will architect and lead the implementation of scalable, high-performance data pipelines and platforms using the Databricks Lakehouse ecosystem. The role involves managing a team of data engineers, establishing best practices, and collaborating with cross-functional stakeholders to unlock advanced analytics, AI/ML, and real-time decision-making capabilities.

Key Responsibilities:

  • Lead the design and development of modern data pipelines, data lakes, and lakehouse architectures using Databricks and Apache Spark.
  • Manage and mentor a team of data engineers, providing technical leadership and fostering a culture of excellence.
  • Architect scalable ETL/ELT workflows to process structured and unstructured data from various sources (cloud, on-prem, streaming).
  • Build and maintain Delta Lake tables and optimize performance for analytics, machine learning, and BI use cases.
  • Collaborate with data scientists, analysts, and business teams to deliver high-quality, trusted, and timely data products.
  • Ensure best practices in data quality, governance, lineage, and security, including the use of Unity Catalog and access controls.
  • Integrate Databricks with cloud platforms (AWS, Azure, or GCP) and data tools (Snowflake, Kafka, Tableau, Power BI, etc.).
  • Implement CI/CD pipelines for data workflows using tools such as GitHub, Azure DevOps, or Jenkins.
  • Stay current with Databricks innovations and provide recommendations on platform strategy and architecture improvements

Qualifications:

  • Education: Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.
  • Experience:
    • 7 years of experience in data engineering, including 3 years working with Databricks and Apache Spark.
    • Proven leadership experience in managing and mentoring data engineering teams.
  • Skills:
    • Proficiency in PySpark, SQL, and experience with Delta Lake, Databricks Workflows, and MLflow.
    • Strong understanding of data modeling, distributed computing, and performance tuning.
    • Familiarity with one or more major cloud platforms (Azure, AWS, GCP) and cloud-native services.
    • Experience implementing data governance and security in large-scale environments.
    • Experience with real-time data processing using Structured Streaming or Kafka.
    • Knowledge of data privacy, security frameworks, and compliance standards (e.g., PCIDSS, GDPR).
    • Exposure to machine learning pipelines, notebooks, and ML Ops practices.
  • Certifications :
    • Databricks Certified Data Engineer or equivalent certification.
Open Positions

1

Mandatory Skills

Proficiency in PySpark & SQL, Delta Lake, Databricks Workflows,

Education Qualification

Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.

Experience

7 to 10 years

  • Lead Data Engineer

    1 week ago

    Mumbai, Maharashtra, India Bradford Consultants Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    The Data Engineering Lead will serve as the solution architect responsible for designing, documenting, implementing and optimizing one or more data platforms including the above-mentioned Data lake. The role will specifically focus on the correct design of the data engineering for setting up a large data warehouse/data lakeRequired Skills & Experience :12+...

  • Lead Data Engineer

    59 minutes ago

    Mumbai, Maharashtra, India M&G Global Services Private Limited Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are M&G Global Services Private Limited (formerly known as 10FA India Private Limited, and prior to that Prudential Global Services Private Limited). We are a fully owned subsidiary of the M&G plc group of companies, operating as a Global Capability Centre providing a range of value adding services to the Group since 2003. At M&G our purpose isto give...

  • Data Engineer

    7 days ago

    Mumbai, Maharashtra, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...

  • Data Engineer

    6 days ago

    Mumbai, Maharashtra, India NTT DATA Global Delivery Services Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    FID Federal Risk Applications Data Engineer Req ID: 341951 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a FID Federal Risk Applications Data Engineer to join our team in...

  • Lead Data Engineer

    2 weeks ago

    Mumbai, Maharashtra, India Jio Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    JIO PLATFORMS LTDDATA ENGINEEROverview of the Company: Jio Platforms Ltd. is a revolutionary Indian multinational tech company, often referred to as India's biggest startup, headquartered in Mumbai. Launched in 2019, it's the powerhouse behind Jio, India's largest mobile network with over 400 million users. But Jio Platforms is more than just telecom. It's a...

  • Data Engineer

    2 days ago

    Mumbai, Maharashtra, India NTT DATA Global Delivery Services Ltd Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Data Engineer Req ID: 340021 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Mumbai, Mahārāshtra (IN-MH), India (IN). Job Duties: Key...

  • Senior Lead Cementing Engineer

    4 days ago

    Mumbai, Maharashtra, India Leader Engineering Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Senior Lead Cementing EngineerDeepwater Oil & Gas Projects – IndiaLeader Engineering | On behalf of a Major International OperatorJob DetailsJob Type -Rotational (Offshore)Job Duration -Through 2030Schedule -As per project schedule – Call-Out BasisOverviewLeader Engineering is seeking an experienced Senior Lead Cementing Engineer to support deepwater...

  • Engineering Lead

    2 days ago

    Mumbai, Maharashtra, India Ness Digital Engineering Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title – Engineering Lead (.Net)Experience – 10 yearsRoles & ResponsibilityStrong Communication skills & High level of ownership as directly interacting with client business, end-users & PM.Collaborate with other professionals to determine functional and non-functional requirements for new software or applications.Ability to lead by example,...

  • Lead Data Engineer 6+ Years

    1 week ago

    Mumbai, Maharashtra, India CodesmoTech Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Company DescriptionCodesmoTech is a leader in digital transformation, helping global corporations and startups achieve agility and scale. We specialize in AI, automation, and cloud computing, delivering unparalleled user experiences, design, and functionality. Our services include Digital Transformation, Data and AI, Cloud Solutions, Modern Software...

  • Microsoft Cloud Data Engineering Lead

    7 days ago

    Mumbai, Maharashtra, India Crisil Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position Overview:The Microsoft Cloud Data Engineering Lead role is ideal for an experienced Microsoft Cloud Data Engineer who will architect, build, and optimize data platforms using Microsoft Azure technologies. The role requires the candidate to have deep technical expertise in Azure data services, strong leadership capabilities, and a passion for...