Data Engineering Lead

17 hours ago

Mumbai, Maharashtra, India CRISIL Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Databricks Data Engineering Lead

Department

MI&A

Job Description

Position Overview:

The Databricks Data Engineering Lead role is ideal a highly skilled Databricks Data Engineer who will architect and lead the implementation of scalable, high-performance data pipelines and platforms using the Databricks Lakehouse ecosystem. The role involves managing a team of data engineers, establishing best practices, and collaborating with cross-functional stakeholders to unlock advanced analytics, AI/ML, and real-time decision-making capabilities.

Key Responsibilities:

  • Lead the design and development of modern data pipelines, data lakes, and lakehouse architectures using Databricks and Apache Spark.
  • Manage and mentor a team of data engineers, providing technical leadership and fostering a culture of excellence.
  • Architect scalable ETL/ELT workflows to process structured and unstructured data from various sources (cloud, on-prem, streaming).
  • Build and maintain Delta Lake tables and optimize performance for analytics, machine learning, and BI use cases.
  • Collaborate with data scientists, analysts, and business teams to deliver high-quality, trusted, and timely data products.
  • Ensure best practices in data quality, governance, lineage, and security, including the use of Unity Catalog and access controls.
  • Integrate Databricks with cloud platforms (AWS, Azure, or GCP) and data tools (Snowflake, Kafka, Tableau, Power BI, etc.).
  • Implement CI/CD pipelines for data workflows using tools such as GitHub, Azure DevOps, or Jenkins.
  • Stay current with Databricks innovations and provide recommendations on platform strategy and architecture improvements

Qualifications:

  • Education: Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.
  • Experience:
    • 7 years of experience in data engineering, including 3 years working with Databricks and Apache Spark.
    • Proven leadership experience in managing and mentoring data engineering teams.
  • Skills:
    • Proficiency in PySpark, SQL, and experience with Delta Lake, Databricks Workflows, and MLflow.
    • Strong understanding of data modeling, distributed computing, and performance tuning.
    • Familiarity with one or more major cloud platforms (Azure, AWS, GCP) and cloud-native services.
    • Experience implementing data governance and security in large-scale environments.
    • Experience with real-time data processing using Structured Streaming or Kafka.
    • Knowledge of data privacy, security frameworks, and compliance standards (e.g., PCIDSS, GDPR).
    • Exposure to machine learning pipelines, notebooks, and ML Ops practices.
  • Certifications :
    • Databricks Certified Data Engineer or equivalent certification.
Open Positions

1

Mandatory Skills

Proficiency in PySpark & SQL, Delta Lake, Databricks Workflows,

Education Qualification

Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.

Experience

7 to 10 years

  • Lead Engineer

    1 week ago

    Mumbai, Maharashtra, India NTT DATA, Inc. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAAs a Networking...

  • Lead Data Engineer

    5 days ago

    Mumbai, Maharashtra, India Bradford Consultants Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    The Data Engineering Lead will serve as the solution architect responsible for designing, documenting, implementing and optimizing one or more data platforms including the above-mentioned Data lake. The role will specifically focus on the correct design of the data engineering for setting up a large data warehouse/data lakeRequired Skills & Experience :12+...

  • Electrical Engineering Innovation Lead

    1 week ago

    Navi Mumbai, Maharashtra, India BDx Data Centers Full time US$ 90,000 - US$ 1,20,000 per year

    Key DeliverablesTo lead the development of all aspects of research in Electrical Power Systems Engineering specifically in the space of software defined power ie managing stranded power in a DC.Collaborate with Design and Engineering (D&E) core teams to develop, implement, and manage the DC electrical designs for new development, infrastructure upgrades and...

  • Data Engineer

    3 days ago

    Mumbai, Maharashtra, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...

  • Lead Data Engineer

    1 week ago

    Mumbai, Maharashtra, India Jio Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    JIO PLATFORMS LTDDATA ENGINEEROverview of the Company: Jio Platforms Ltd. is a revolutionary Indian multinational tech company, often referred to as India's biggest startup, headquartered in Mumbai. Launched in 2019, it's the powerhouse behind Jio, India's largest mobile network with over 400 million users. But Jio Platforms is more than just telecom. It's a...

  • Data Engineer

    3 days ago

    Mumbai, Maharashtra, India NTT DATA Global Delivery Services Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    FID Federal Risk Applications Data Engineer Req ID: 341951 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a FID Federal Risk Applications Data Engineer to join our team in...

  • Lead Process Engineer

    1 week ago

    Mumbai, Maharashtra, India ZEE Engineering Group Full time US$ 90,000 - US$ 1,20,000 per year

    Company DescriptionZEE Engineering Group is a privately-owned Engineering Consultancy specializing in the Oil and Gas industry since 1986. Based in Malaysia with additional offices in India and Indonesia, ZEE provides personalized, cost-effective solutions through a team of dedicated engineering specialists. The company upholds ISO 9001 certification...

  • Lead Data Engineer

    1 week ago

    Navi Mumbai, Maharashtra, India JobTravia Pvt. Ltd. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title Lead Data EngineerEducation Bachelor's degree in engineering (CS, IT, EXTC or equivalent)Experience: 6-8 yearsIn depth understanding of spark framework and experience workingwith Spark SQL and Pyspark.Expert in Python programming language and common python librariesStrong analytical experience with database in writing complex queries,query...

  • FID Federal Risk Applications Data Engineer

    2 weeks ago

    Mumbai, Maharashtra, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Req ID: 341951NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a FID Federal Risk Applications Data Engineer to join our team in Mumbai, Mahārāshtra (IN-MH), India (IN). Job Duties:...

  • Lead Data Engineer 6+ Years

    5 days ago

    Mumbai, Maharashtra, India CodesmoTech Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Company DescriptionCodesmoTech is a leader in digital transformation, helping global corporations and startups achieve agility and scale. We specialize in AI, automation, and cloud computing, delivering unparalleled user experiences, design, and functionality. Our services include Digital Transformation, Data and AI, Cloud Solutions, Modern Software...