Lead Databricks Engineer

12 hours ago

Bengaluru, Karnataka, India DXC Technology Full time US$ 1,50,000 - US$ 2,00,000 per year

Job Description:

Lead Databricks Engineer / Architect

Job Location: Hyderabad / Bangalore / Chennai / Kolkata / Noida/ Gurgaon / Pune / Indore / Mumbai

Role Overview
We are seeking an experienced Lead Data Engineer / Architect with over 10 years of industry experience (5+ years in Databricks and modern data platforms) to design, implement, and lead enterprise-scale data solutions. The ideal candidate is hands-on with Databricks ecosystem, PySpark (UDFs, Pandas UDFs), dbt, and Airflow, and is capable of driving architecture decisions, leading engineering teams, and ensuring best practices in data engineering.
Key Responsibilities

  • Lead the design and implementation of data lakehouse architectures using Databricks, Delta Lake, and Delta Live Tables.
  • Define best practices for workspace setup, cluster management, repos, and job orchestration.
  • Partner with stakeholders to translate business needs into scalable data engineering solutions.
  • Data Engineering & Development
  • Develop and optimize ETL/ELT pipelines using PySpark, dbt, and Airflow.
  • Implement advanced PySpark transformations leveraging UDFs and Pandas UDFs for complex use cases.
  • Ensure data quality, lineage, and observability across all pipelines.

Required Skills & Qualifications

  • 10+ years of total IT experience, with at least 5 years of relevant data engineering experience on Databricks.
  • Strong expertise in:
  • Databricks (Workspace, Clusters, Jobs, Repos, Delta Live Tables)
  • PySpark (UDFs & Pandas UDFs)
  • dbt for data transformations
  • Apache Airflow for orchestration
  • Solid understanding of cloud data platforms (Azure/AWS/GCP).
  • Hands-on with CI/CD, version control (Git), and DevOps practices for data engineering.
  • Databricks Certified Data Engineer Associate (mandatory).
  • Proven ability to lead and mentor teams, with excellent communication and stakeholder management skills.

At DXC Technology, we believe strong connections and community are key to our success. Our work model prioritizes in-person collaboration while offering flexibility to support wellbeing, productivity, individual work styles, and life circumstances. We're committed to fostering an inclusive environment where everyone can thrive.

Recruitment fraud is a scheme in which fictitious job opportunities are offered to job seekers typically through online services, such as false websites, or through unsolicited emails claiming to be from the company. These emails may request recipients to provide personal information or to make payments as part of their illegitimate recruiting process. DXC does not make offers of employment via social media networks and DXC never asks for any money or payments from applicants at any point in the recruitment process, nor ask a job seeker to purchase IT or other equipment on our behalf. More information on employment scams is available here.

  • Director of Engineering

    1 week ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    (P-1384)About the TeamAt Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. Ingesting data into the Lakehouse is a strategic area of investment for Databricks and a key enabler for Data and AI...

  • Director of Engineering

    1 day ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    (P-1384)About the TeamAt Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data...

  • Sr. Solutions Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Databricks Full time

    Job DescriptionFEQ426R153As a Sr. Solutions Engineer (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our...

  • Sr. Solutions Engineer

    5 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,20,000 - US$ 2,00,000 per year

    FEQ426R153As a Sr. Solutions Engineer (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...

  • Engineering Manager

    1 week ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    P-995Companies are investing billions of dollars into developing and deploying AI and the data platforms that enable it. But do they know what is happening? At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical...

  • Partner Solutions Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Databricks Full time

    Job DescriptionFEQ125R56As a Partner Solutions Architect (PSA) for India, you will work with Databricks&apos Consulting and System Integrator (C&SI) partners, teammates, and with the technical and sales team members who work directly with our customers. You will develop &apostechnical champions&apos within our top C&SI Partners, providing enablement on...

  • Staff Software Engineer

    1 week ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    P-926At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Databricks lead

    2 days ago

    Bengaluru, Karnataka, India Recro Full time

    Designation - Databricks leadExperience - 8+ YearsKey Responsibilities• Lead the design and implementation ofscalable data pipelines and ETL processes on Databricks.• Architect and optimize big data and analyticssolutions using Spark, Delta Lake, and MLflow.• Collaborate with stakeholders to translate businessrequirementsinto technical solutions.•...

  • Databricks lead

    3 days ago

    Bengaluru, Karnataka, India Recro Full time

    Designation - Databricks leadExperience - 8+ YearsKey Responsibilities• Lead the design and implementation ofscalable data pipelines and ETL processes on Databricks.• Architect and optimize big data and analyticssolutions using Spark, Delta Lake, and MLflow.• Collaborate with stakeholders to translate businessrequirementsinto technical solutions.•...

  • Databricks Lead

    1 day ago

    Bengaluru, Karnataka, India Genisys Group Full time

    Job SummaryDatabricks Lead is a senior role and expectation is that the ideal candidate should have minimum 10+ years of development hands-on experience.Key Skills & Qualifications:Databricks Expertise: Deep experience with the Databricks platform, including Spark, SQL, and data processing capabilities.Programming Languages: Proficiency in programming...