Lead Databricks Engineer

7 days ago

Bengaluru, Karnataka, India DXC Technology Full time ₹ 15,00,000 - ₹ 20,00,000 per year

Job Description:

Lead Databricks Engineer / Architect

Job Location: Hyderabad / Bangalore / Chennai / Kolkata / Noida/ Gurgaon / Pune / Indore / Mumbai

Role Overview

We are seeking an experienced Lead Data Engineer / Architect with over 10 years of industry experience (5+ years in Databricks and modern data platforms) to design, implement, and lead enterprise-scale data solutions. The ideal candidate is hands-on with Databricks ecosystem, PySpark (UDFs, Pandas UDFs), dbt, and Airflow, and is capable of driving architecture decisions, leading engineering teams, and ensuring best practices in data engineering.

Key Responsibilities

  • Lead the design and implementation of data lakehouse architectures using Databricks, Delta Lake, and Delta Live Tables.
  • Define best practices for workspace setup, cluster management, repos, and job orchestration.
  • Partner with stakeholders to translate business needs into scalable data engineering solutions.
  • Data Engineering & Development
  • Develop and optimize ETL/ELT pipelines using PySpark, dbt, and Airflow.
  • Implement advanced PySpark transformations leveraging UDFs and Pandas UDFs for complex use cases.
  • Ensure data quality, lineage, and observability across all pipelines.

Required Skills & Qualifications

  • 10+ years of total IT experience, with at least 5 years of relevant data engineering experience on Databricks.
  • Strong expertise in:
  • Databricks (Workspace, Clusters, Jobs, Repos, Delta Live Tables)
  • PySpark (UDFs & Pandas UDFs)
  • dbt for data transformations
  • Apache Airflow for orchestration
  • Solid understanding of cloud data platforms (Azure/AWS/GCP).
  • Hands-on with CI/CD, version control (Git), and DevOps practices for data engineering.
  • Databricks Certified Data Engineer Associate (mandatory).
  • Proven ability to lead and mentor teams, with excellent communication and stakeholder management skills.

At DXC Technology, we believe strong connections and community are key to our success. Our work model prioritizes in-person collaboration while offering flexibility to support wellbeing, productivity, individual work styles, and life circumstances. We're committed to fostering an inclusive environment where everyone can thrive.

Recruitment fraud is a scheme in which fictitious job opportunities are offered to job seekers typically through online services, such as false websites, or through unsolicited emails claiming to be from the company. These emails may request recipients to provide personal information or to make payments as part of their illegitimate recruiting process. DXC does not make offers of employment via social media networks and DXC never asks for any money or payments from applicants at any point in the recruitment process, nor ask a job seeker to purchase IT or other equipment on our behalf. More information on employment scams is available here.

  • Director of Engineering

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    (P-1384)About The TeamAt Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs.Ingesting data into the Lakehouse is a strategic area of investment for Databricks and a key enabler for Data and AI...

  • Director of Engineering

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    (P-1384)About the TeamAt Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data...

  • Senior Staff Software Engineer

    7 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    (P-1174)We are hiring aSenior Staff Software Engineerfor Databricks' Engineering team reporting to an Engineering Leader. You will be part of the Databricks engineering organization, working with teams that develop Databricks products and features for thousands of enterprises worldwide.Key CharacteristicsAs an executive engineering individual contributor at...

  • Director of Engineering

    2 weeks ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    (P-1384)About the TeamAt Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data...

  • Senior Software Engineer

    7 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    P-1385Companies are investing billions of dollars into developing and deploying AI and the data platforms that enable it. But do they know what is happening?At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical...

  • Senior Software Engineer

    7 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    P-1346At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Staff Software Engineer

    3 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    P-1346At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Lead Databricks Engineer

    2 weeks ago

    Bengaluru, Karnataka, India DXC Technology Full time US$ 1,50,000 - US$ 2,00,000 per year

    Job Description:Lead Databricks Engineer / Architect Job Location: Hyderabad / Bangalore / Chennai / Kolkata / Noida/ Gurgaon / Pune / Indore / Mumbai Role OverviewWe are seeking an experienced Lead Data Engineer / Architect with over 10 years of industry experience (5+ years in Databricks and modern data platforms) to design, implement, and lead...

  • Senior Engineering Manager

    7 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    P-995At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Staff Software Engineer

    4 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    P- 1430At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...