Lead Databricks Engineer

2 hours ago

Bengaluru, India DXC Technology Full time

Job Description:

Lead Databricks Engineer / Architect

Job Location: Hyderabad / Bangalore / Chennai / Kolkata / Noida/ Gurgaon / Pune / Indore / Mumbai

Role Overview
We are seeking an experienced Lead Data Engineer / Architect with over 10 years of industry experience (5+ years in Databricks and modern data platforms) to design, implement, and lead enterprise-scale data solutions. The ideal candidate is hands-on with Databricks ecosystem, PySpark (UDFs, Pandas UDFs), dbt, and Airflow, and is capable of driving architecture decisions, leading engineering teams, and ensuring best practices in data engineering.
Key Responsibilities

Lead the design and implementation of data lakehouse architectures using Databricks, Delta Lake, and Delta Live Tables. Define best practices for workspace setup, cluster management, repos, and job orchestration. Partner with stakeholders to translate business needs into scalable data engineering solutions. Data Engineering & Development Develop and optimize ETL/ELT pipelines using PySpark, dbt, and Airflow. Implement advanced PySpark transformations leveraging UDFs and Pandas UDFs for complex use cases. Ensure data quality, lineage, and observability across all pipelines.


Required Skills & Qualifications

10+ years of total IT experience, with at least 5 years of relevant data engineering experience on Databricks. Strong expertise in: Databricks (Workspace, Clusters, Jobs, Repos, Delta Live Tables) PySpark (UDFs & Pandas UDFs) dbt for data transformations Apache Airflow for orchestration Solid understanding of cloud data platforms (Azure/AWS/GCP). Hands-on with CI/CD, version control (Git), and DevOps practices for data engineering. Databricks Certified Data Engineer Associate (mandatory). Proven ability to lead and mentor teams, with excellent communication and stakeholder management skills.

At DXC Technology, we believe strong connections and community are key to our success. Our work model prioritizes in-person collaboration while offering flexibility to support wellbeing, productivity, individual work styles, and life circumstances. We’re committed to fostering an inclusive environment where everyone can thrive.

  • Director of Engineering

    3 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 1,50,00,000 - ₹ 2,50,00,000 per year

    Director of Engineering (Lakeflow) (P-1384) About the Team At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. Ingesting data into the Lakehouse is a strategic area of investment for...

  • Director of Engineering

    5 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    (P-1384)About The TeamAt Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs.Ingesting data into the Lakehouse is a strategic area of investment for Databricks and a key enabler for Data and AI...

  • Senior Staff Software Engineer

    5 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 15,00,000 - ₹ 60,00,000 per year

    Job Description (P-1174) We are hiring a Senior Staff Software Engineer for Databricks' Engineering team reporting to an Engineering Leader. You will be part of the Databricks engineering organization, working with teams that develop Databricks products and features for thousands of enterprises worldwide.Key Characteristics As an executive engineering...

  • Senior Staff Software Engineer

    1 day ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Staff Software Engineer (Data Platform) (P-1174) We are hiring a Senior Staff Software Engineer for Databricks' Engineering team reporting to an Engineering Leader. You will be part of the Databricks engineering organization, working with teams that develop Databricks products and features for thousands of enterprises worldwide. Key Characteristics ...

  • Senior Staff Software Engineer

    3 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Senior Staff Software Engineer (Backend) (P-1174) We are hiring a Senior Staff Software Engineer for Databricks' Engineering team reporting to an Engineering Leader. You will be part of the Databricks engineering organization, working with teams that develop Databricks products and features for thousands of enterprises worldwide. Key Characteristics ...

  • Senior Staff Software Engineer

    5 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description (P-1174) We are hiring a Senior Staff Software Engineer for Databricks' Engineering team reporting to an Engineering Leader. You will be part of the Databricks engineering organization, working with teams that develop Databricks products and features for thousands of enterprises worldwide.Key Characteristics As an executive engineering...

  • Senior Staff Software Engineer

    3 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    (P-1174)We are hiring aSenior Staff Software Engineerfor Databricks' Engineering team reporting to an Engineering Leader. You will be part of the Databricks engineering organization, working with teams that develop Databricks products and features for thousands of enterprises worldwide.Key CharacteristicsAs an executive engineering individual contributor at...

  • Senior Staff Software Engineer

    2 weeks ago

    Bengaluru, India Databricks Full time

    (P-1174)We are hiring aSenior Staff Software Engineerfor Databricks' Engineering team reporting to an Engineering Leader. You will be part of the Databricks engineering organization, working with teams that develop Databricks products and features for thousands of enterprises worldwide. Key CharacteristicsAs an executive engineering individual contributor at...

  • Senior Software Engineer

    2 weeks ago

    Bengaluru, India Databricks Full time

    P-1385 Companies are investing billions of dollars into developing and deploying AI and the data platforms that enable it. But do they know what is happening? At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical...

  • Senior Software Engineer

    3 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    P-1385Companies are investing billions of dollars into developing and deploying AI and the data platforms that enable it. But do they know what is happening?At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical...