Databricks Engineer

3 days ago

Bengaluru Chennai Pune, India Newbie Soft Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Title: Senior Databricks Architect / Lead Databricks Engineer

Experience: 10+ Years (Minimum 4+ Years in Databricks)

Locations: Chennai | Bangalore | Pune | Hyderabad

Work Mode: Hybrid 3 Days from Office

Job Overview

We are looking for a highly experienced Databricks Architect with 10+ years of IT experience and at least 4 years of deep, hands-on Databricks expertise. The candidate will lead the design and delivery of scalable, secure, and cost-optimized data platforms on the Databricks Lakehouse ecosystem.

This role requires strong architectural ownership, technical leadership, and the ability to guide engineering teams.

Key Responsibilities

  • Lead end-to-end Databricks architecture: ingestion, transformation, governance, analytics.
  • Design scalable Lakehouse Architecture using Delta Tables, schema evolution, ACID transactions, time-travel, and interoperability.
  • Integrate Databricks with enterprise ETL / orchestration tools.
  • Build optimized pipelines with Spark (DataFrame API & Spark SQL) using Python, Scala, and SQL.

  • Utilize Databricks-native features:

  • Delta Live Tables (DLT)

  • Unity Catalog
  • Workflows

  • Lakeflow & Lakehouse Federation

  • Perform cluster configuration, performance tuning, cost optimization, security & monitoring.

  • Build real-time streaming pipelines using Structured Streaming + Autoloader.
  • Implement DevOps & automation: Terraform, GitHub Actions, Azure DevOps CI/CD.
  • Create scalable data pipeline architectures using Databricks best practices.
  • Strong expertise in data modeling (conceptual, logical, physical).
  • Collaborate closely with CXOs, architects, and global stakeholders.
  • Prepare solution proposals, architecture diagrams, technical documents, RFx responses, and presentations.
  • Mentor and guide engineering teams to deliver high-quality data solutions.

Required Skills

  • 10+ years of IT experience with 4+ years hands-on in Databricks

  • Deep expertise in:

  • Lakehouse architecture & Delta ecosystem

  • Spark DataFrame API, Spark SQL
  • Python, Scala, SQL
  • Unity Catalog, DLT, Workflows
  • Structured Streaming & Autoloader

  • CI/CD (GitHub Actions, Azure DevOps) & Terraform

  • Strong skills in cluster performance, cost governance, data security, and monitoring

  • Excellent communication & stakeholder management

  • Engineering Manager

    1 week ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    P-995At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Engineering Manager

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    P-995At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Staff Software Engineer

    2 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 2,00,000 - ₹ 4,00,000 per year

    P-375At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Engineering Manager

    2 weeks ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 15,00,000 - ₹ 30,00,000 per year

    P-995At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Engineering Manager

    1 week ago

    Bengaluru, India Databricks Full time

    P-995 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Senior Backline Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    P-1386At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Staff Software Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 25,00,000 - ₹ 50,00,000 per year

    P-1348At Databricks, we are passionate about enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data and AI infrastructure platform, so our customers can focus on the high value challenges that are central to their own missions. Our...

  • Staff Software Engineer

    2 weeks ago

    Bengaluru, India Databricks Full time

    P-1346 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Senior Software Engineer

    2 weeks ago

    Bengaluru, India Databricks Full time

    P-1346 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Staff Backline Engineer

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

    P-1376Job Overview:We are seeking a highly skilled Frontline Senior Technical Solutions Engineer with over 5 years of experience to join our Platform Support team. This role is pivotal in delivering exceptional support for our Databricks Data Intelligence platform, addressing complex technical challenges, and ensuring the seamless operation of our data...