Databricks Implementation Lead

1 day ago

Hyderabad, Telangana, India Rubis Software Solutions Pvt Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year
  • Experience needed:
    years
  • Type:
     Full-Time
  • Mode:
     WFO
  • Shift:
    IST
  • Location:
     Hyderabad, India
  • Notice Period:
     Immediate to 15 days

About the Role

We are seeking an experienced
Databricks Implementation Lead
 to spearhead end-to-end delivery of a large-scale 
data engineering and analytics project
 built on the 
Databricks Lakehouse platform
.

The ideal candidate will have deep expertise in 
data architecture
, hands-on experience with 
Databricks (SQL, PySpark, Delta Lake, MLflow)
, and the ability to lead teams through 
solution design, migration, optimization, and production rollout
.

Key Responsibilities

1. Project Leadership

  • Own the complete 
    Databricks implementation lifecycle
     from requirement gathering, architecture design, and environment setup to production deployment and handover.
  • Act as the primary 
    technical liaison
     between business stakeholders, data engineering teams, and cloud platform teams.
  • Ensure alignment with project goals, timelines, and compliance standards (
    data governance, security, cost optimization
    ).

2. Architecture & Design

  • Design scalable and secure 
    data pipelines
     leveraging 
    Databricks, Delta Lake
    , and 
    cloud-native services (Azure, AWS, or GCP)
    .
  • Define the 
    data ingestion, transformation, and consumption architecture
    , integrating multiple data sources (APIs, databases, files, streaming).
  • Establish 
    CI/CD pipelines
     for Databricks notebooks and jobs, enabling automation and version control (using Git / Azure DevOps).

3. Technical Implementation

  • Lead and mentor a team of 
    data engineers
     to develop 
    ETL/ELT pipelines
     using 
    PySpark, SQL, and Databricks workflows
    .
  • Configure clusters, jobs, and 
    Lakehouse storage
     following best practices.
  • Optimize 
    Databricks compute and query performance
     for cost and efficiency.
  • Implement 
    data quality frameworks, monitoring dashboards, and error-handling logic
    .

4. Governance & Operations

  • Set up 
    access control, role-based security, and data lineage tracking
     in 
    Unity Catalog
    .
  • Collaborate with the 
    CloudOps team
     for environment setup, monitoring, and cost governance.
  • Drive 
    documentation, training, and knowledge transfer
     to ensure sustainability and handover readiness.

Required Skills & Experience

  • 8+ years of experience in 
    data engineering or big data projects
    .
  • 3+ years of 
    hands-on Databricks experience
     (SQL, PySpark, Delta Lake, MLflow).
  • Strong experience with 
    ETL/ELT pipeline design
    , 
    data modeling
    , and 
    data warehousing
    .
  • Experience with 
    Azure Data Factory, AWS Glue
    , or similar orchestration tools.
  • Proficiency in 
    Python, SQL
    , and 
    Spark optimization techniques
    .
  • Experience implementing 
    DevOps practices (CI/CD, Git, Databricks Repos)
    .
  • Familiarity with 
    data governance frameworks, Unity Catalog, and RBAC
    .
  • Proven track record in 
    leading multi-member implementation teams
    .

Preferred Qualifications

  • Databricks Certified Data Engineer Professional
     or 
    Solution Architect certification
    .
  • Experience with 
    AI/ML workflows
     and model management using 
    MLflow
    .
  • Exposure to 
    Power BI, Tableau, or Looker
     for downstream visualization.
  • Strong communication skills and experience in 
    client-facing roles
    .

Soft Skills

  • Excellent 
    problem-solving and analytical skills
    .
  • Strong 
    leadership and mentoring
     capabilities.
  • Ability to work in a 
    fast-paced, cross-functional environment
    .
  • Clear 
    documentation and presentation
     skills.

  • Lead Databricks Engineer

    1 week ago

    Hyderabad, Telangana, India Milestone Technologies, Inc. Full time ₹ 30,00,000 - ₹ 50,00,000 per year

    Role Overview:We are looking for a seasoned and visionary Lead Databricks Engineerwith over 14 years of experience to lead the design and implementation of enterprise-grade data architecture, governance frameworks, and scalable data platforms. This role requires deep expertise in modern data technologies, big data ecosystems, and cloud-native solutions. The...

  • Databricks Engineer

    1 week ago

    Hyderabad, Telangana, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics. Use Databricks Workflows for orchestrating batch and streaming pipelines. Develop and maintain CI/CD pipelines for data applications using tools such as Azure DevOps,...

  • Lead Databricks Engineer

    1 week ago

    Hyderabad, Telangana, India Milestone Technologies, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Company OverviewMilestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over...

  • Databricks + Pyspark

    1 week ago

    Hyderabad, Telangana, India Cognizant Full time ₹ 1,00,00,000 - ₹ 3,00,00,000 per year

    Skills- Databricks+ PysparkExperience: 4 to 13 yearsLocation: AIA-PuneWe are looking for a highly skilled Data Engineer with expertise in PySpark and Databricks to design, build, and optimize scalable data pipelines for processing massive datasets.Key Responsibilities:Build & Optimize Pipelines: Develop high-throughput ETL workflows using PySpark on...

  • Databricks+spark

    1 week ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryWe are seeking a highly skilled Sr. Developer with 8 to 12 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. Experience in Sales & Marketing domains is a plus. The role involves developing and optimizing data solutions to...

  • Lead Consultant-Databricks

    2 weeks ago

    Hyderabad, Telangana, India Genpact Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    {E131AB7B-750D-4D85-AC7A-88463F2B8DE8}&file=Non-Tech-Lead Consultant- Databricks Developer .docx&action=default&mobileredirect=trueJobLead ConsultantPrimary LocationIndia-HyderabadScheduleFull-timeEducation LevelBachelor's / Graduation / EquivalentJob PostingSep 9, 2025, 1:38:46 AMUnposting DateMar 8, 2026, 6:38:46 AMMaster Skills ListDigitalJob CategoryFull...

  • DataBricks - Data Engineering

    5 days ago

    Hyderabad, Telangana, India Wipro Full time ₹ 15,00,000 - ₹ 30,00,000 per year

    Wipro Limited (NYSE: WIT, BSE: 507685, NSE: WIPRO) is a leading technology services and consulting company focused on building innovative solutions that address clients' most complex digital transformation needs. Leveraging our holistic portfolio of capabilities in consulting, design, engineering, and operations, we help clients realize their boldest...

  • Databricks with Python

    1 week ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role - Databricks with PythonExp - 5 to 9 YrsLocation - HyderabadNote - This is a scheduled drive. Only eligible candidates who receive invite from us will be allowed inside Campus.Job DescriptionExtensive expertise in designing and implementing data load processes using Azure Data Factory, Azure Databricks, Delta Lake, Azure Delta Lake Storage and...

  • Databricks Engineer

    5 days ago

    Hyderabad, Telangana, India Stanra Tech Solutions Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    We are looking for a skilled and experienced Databricks Engineer to join our team at our Hyderabad (Madhapur) office. The ideal candidate will have a strong background in data engineering, hands-on experience with Databricks, and a solid understanding of cloud platforms (AWS, Azure, or GCP). This is a great opportunity to work on cutting-edge data projects...

  • Pyspark + Databricks

    1 week ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryWe are seeking a highly skilled Sr. Developer with 6 to 10 years of experience to join our team. The ideal candidate will have expertise in Databricks SQL Databricks Workflows and PySpark. Experience in the Cards & Payments domain is a plus. This is a hybrid work model with day shifts and no travel required.ResponsibilitiesDevelop and maintain...