Databricks Data Engineering Lead

11 hours ago

Mumbai, Maharashtra, India Crisil Full time ₹ 8,00,000 - ₹ 24,00,000 per year

Position Overview:
The
Databricks Data Engineering Lead
role is ideal a highly skilled Databricks Data Engineer who will architect and lead the implementation of scalable, high-performance data pipelines and platforms using the Databricks Lakehouse ecosystem. The role involves managing a team of data engineers, establishing best practices, and collaborating with cross-functional stakeholders to unlock advanced analytics, AI/ML, and real-time decision-making capabilities.

Key Responsibilities:

  • Lead the design and development of modern data pipelines, data lakes, and lakehouse architectures using Databricks and Apache Spark.
  • Manage and mentor a team of data engineers, providing technical leadership and fostering a culture of excellence.
  • Architect scalable ETL/ELT workflows to process structured and unstructured data from various sources (cloud, on-prem, streaming).
  • Build and maintain Delta Lake tables and optimize performance for analytics, machine learning, and BI use cases.
  • Collaborate with data scientists, analysts, and business teams to deliver high-quality, trusted, and timely data products.
  • Ensure best practices in data quality, governance, lineage, and security, including the use of Unity Catalog and access controls.
  • Integrate Databricks with cloud platforms (AWS, Azure, or GCP) and data tools (Snowflake, Kafka, Tableau, Power BI, etc.).
  • Implement CI/CD pipelines for data workflows using tools such as GitHub, Azure DevOps, or Jenkins.
  • Stay current with Databricks innovations and provide recommendations on platform strategy and architecture improvements

Qualifications:

  • Education: Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.

  • Experience:

  • 7+ years of experience in data engineering, including 3+ years working with Databricks and Apache Spark.

  • Proven leadership experience in managing and mentoring data engineering teams.

  • Skills:

  • Proficiency in PySpark, SQL, and experience with Delta Lake, Databricks Workflows, and MLflow.

  • Strong understanding of data modeling, distributed computing, and performance tuning.
  • Familiarity with one or more major cloud platforms (Azure, AWS, GCP) and cloud-native services.
  • Experience implementing data governance and security in large-scale environments.
  • Experience with real-time data processing using Structured Streaming or Kafka.
  • Knowledge of data privacy, security frameworks, and compliance standards (e.g., PCIDSS, GDPR).

  • Exposure to machine learning pipelines, notebooks, and ML Ops practices.

  • Certifications :

  • Databricks Certified Data Engineer or equivalent certification.

  • Field Engineering Manager

    1 week ago

    Mumbai, Maharashtra, India Databricks Full time ₹ 20,00,000 - ₹ 60,00,000 per year

    Manager, Field Engineering FEQ326R99 As a Manager, Field Engineering (Data & AI) (Strategic Accounts, Data & AI) will lead a team of Solutions Architects (Data & AI) focusing on large enterprise & strategic customers in India. You will help lead our Global Accounts team in India expansion, you will build and lead a technical pre-sales team. You will...

  • Manager, Field Engineering

    10 hours ago

    Mumbai, Maharashtra, India Databricks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    FEQ326R99As a Manager, Field Engineering (Data & AI)  (Strategic Accounts, Data & AI) will lead a team of Solutions Architects (Data & AI) focusing on large enterprise & strategic customers in India. You will help lead our Global Accounts team in India expansion, you will build and lead a technical pre-sales team. You will guide and get involved to enhance...

  • Databricks Data Engineer

    2 weeks ago

    Mumbai, Maharashtra, India Crisil Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    Position Overview:The Databricks Data Engineer role is ideal for a skilled and motivated Databricks Data Engineer to design, build, and optimize data pipelines and analytics solutions on the Databricks Lakehouse Platform. This role is ideal for candidates who have experience with big data technologies and a passion for building robust data systems.Key...

  • Delivery Solutions Architect

    1 week ago

    Mumbai, Maharashtra, India Databricks Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Delivery Solutions Architect CSQ426R270 At Databricks, we are on a mission to empower our customers to solve the world's toughest data problems by utilizing the Databricks Data Intelligence Platform. As a Delivery Solutions Architect (DSA), you will play an important role during this journey. You will collaborate with our sales and field engineering...

  • Data Engineer

    3 weeks ago

    Mumbai, Maharashtra, India, Maharashtra Nebula Tech Solutions Full time

    Data Engineer – Databricks Expert Remote (India-based) | Full-time | Immediate Joiners PreferredAre you passionate about building scalable data pipelines and optimizing performance with cutting-edge tools like Databricks and Apache Spark? We're looking for a seasoned Data Engineer to join our team and work on high-impact data products for a leading global...

  • Data Engineer- databricks

    2 weeks ago

    Mumbai, Maharashtra, India Qualintia Solutions Full time ₹ 8,00,000 - ₹ 25,00,000 per year

    Role- Data Engineer- DatabricksExp- 8+ years,Job Type- ContractualLocation- MumbaiWork Mode- WFO• Learning & Certification Opportunities: Enhance your professional growth.• Comprehensive Medical Coverage and Life Insurance: For your well-being.• Flexible Work Environment: Enjoy a 5-day work week.• Collaborative Culture: Be part of a fun, innovative...

  • Data Engineering Lead

    2 weeks ago

    Mumbai, Maharashtra, India CRISIL Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Databricks Data Engineering Lead Department MI&A Job Description Position Overview: The Databricks Data Engineering Lead role is ideal a highly skilled Databricks Data Engineer who will architect and lead the implementation of scalable, high-performance data pipelines and platforms using the Databricks Lakehouse ecosystem. The role involves...

  • Resident Solutions Architect

    7 hours ago

    Mumbai, Maharashtra, India Databricks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    CSQ426R257We are seeking an experienced Resident Solution Architect (RSA) to join our Professional Services team and work directly with strategic customers on their data and AI transformation initiatives using the Databricks platform. As an RSA, you will serve as a trusted technical advisor and hands-on expert, guiding customers to solve complex big data...

  • Senior Data Engineer- Databricks

    2 weeks ago

    Mumbai, Maharashtra, India Koantek Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking an experienced Databricks Full Stack Data Engineer with 5–6 years of industryexperience. The ideal candidate will have a proven track record of working on live projects,preferably within the manufacturing or energy sectors. He/she will play a key role in developingand maintaining scalable data solutions using Databricks and related...

  • Databricks Engineer

    2 weeks ago

    Mumbai, Maharashtra, India AVE-Promagne Business Solutions Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Responsibilities : - Design, develop, and implement data pipelines using Apache Spark and Databricks notebooks to process and transform large datasets. - Optimize data pipelines for performance, scalability, and cost-effectiveness. - Automate data pipelines and integrate them with our existing data infrastructure. - Develop and maintain unit...