Databricks Data Engineer

1 week ago

Mumbai Metropolitan Region, India USR SYSTEMS LLC Full time ₹ 12,00,000 - ₹ 36,00,000 per year

9 Azure DataBricks Data Engineers with Kafka Live Streaming required in Navi Mumbai. Onsite. Immediate interview and Joining.

Location: Navi Mumbai (Ghansoli)

Duration: Permanent/Full Time

Experience Required: 3-5 Years & 6-8+ Years

Job Summary:

  • We are looking for a highly skilled
    Azure Data Engineer
    with a strong background in real-time and batch data ingestion and big data processing, particularly using
    Kafka and Databricks
    .
  • The ideal candidate will have a deep understanding of
    streaming architectures

,
Medallion data models
, and
performance optimization techniques
in cloud environments.
- This role requires
hands-on technical expertise
, including
live coding during the interview process
.

Key Responsibilities

  • Design and implement
    Live streaming data pipelines

integrating
Kafka with Databricks

using
Structured Streaming
.
- Architect and maintain
Medallion Architecture
with well-defined
Bronze, Silver, and Gold layers
.
- Implement efficient ingestion using
Databricks Autoloader

for high-throughput data loads.
- Work with
large volumes of structured and unstructured data
, ensuring high availability and performance.
- Apply performance tuning techniques such as
partitioning, caching
, and
cluster resource optimization
.
- Collaborate with cross-functional teams (data scientists, analysts, business users) to build robust data solutions.
- Establish best practices for
code versioning
,
deployment automation
, and
data governance
.

Required Technical Skills:

  • Strong expertise in Azure Databricks and Spark Structured Streaming
  • 3-8 Years' experience in Data Engineering
  • Processing modes (append, update, complete)
  • Output modes (append, complete, update)
  • Checkpointing and state management
  • Experience with Kafka integration

for real-time data pipelines
- Deep understanding of
Medallion Architecture
- Proficiency with
Databricks Autoloader

and schema evolution
- Deep understanding of Unity Catalog and Foreign catalog
- Strong knowledge of
Spark SQL, Delta Lake, and DataFrames
- Expertise in
performance tuning

(query optimization, cluster configuration, caching strategies)
- Must have Data management strategies
- Excellent with Governance and Access management
- Strong with Data modelling, Data warehousing concepts, Databricks as a platform
- Solid understanding of
Window functions

Proven experience in:

  • Merge/Upsert logic
  • Implementing
    SCD Type 1 and Type 2
  • Handling
    CDC (Change Data Capture)

scenarios
- Retail/Telcom/Energy any one industry expertise
- Real time use case execution
- Data modelling

  • Resident Solutions Architect

    6 days ago

    Mumbai Metropolitan Region, India Databricks Full time ₹ 20,00,000 - ₹ 40,00,000 per year

    CSQ426R257We are seeking an experienced Resident Solution Architect (RSA) to join our Professional Services team and work directly with strategic customers on their data and AI transformation initiatives using the Databricks platform. As an RSA, you will serve as a trusted technical advisor and hands-on expert, guiding customers to solve complex big data...

  • Senior Azure Databricks Engineer

    10 hours ago

    Mumbai Metropolitan Region, India SWITS DIGITAL Private Limited Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title:Azure Databricks EngineerLocation:PAN IndiaExperience:8–10 years total, with 4+ years relevant experienceJob SummaryWe are looking for a highly skilled Azure Databricks Engineer to join our data engineering team. The ideal candidate will have hands-on expertise in building scalable data pipelines, optimizing big data workloads, and leveraging...

  • Data Engineer

    4 days ago

    Mumbai Metropolitan Region, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID:342707NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Mumbai, Mahārāshtra (IN-MH), India (IN).Job Duties: Key Responsibilities:Design and...

  • Architect : Big Data / Ml

    2 weeks ago

    Mumbai, India Databricks Full time

    As a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...

  • Solutions Architect

    12 hours ago

    Mumbai, India Databricks Full time

    **What we look for**: As a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our...

  • Solutions Architect

    4 days ago

    Mumbai, India Databricks Full time

    As a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. This is a senior role, where will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our...

  • Enterprise Solutions Architect

    2 weeks ago

    Mumbai, India Databricks Full time

    As a Pre-sales Solutions Architect (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...

  • Enterprise Solutions Architect

    1 week ago

    Mumbai, India Databricks Full time

    As a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve tangible...

  • Principal Solutions Architect

    2 weeks ago

    Mumbai, India Databricks Full time

    As a Pre-sales Solutions Architect (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...

  • Manager, Field Engineering

    7 days ago

    mumbai city, India Databricks Full time

    FEQ326R99As a Manager, Field Engineering (Data & AI)  (Strategic Accounts, Data & AI) will lead a team of Solutions Architects (Data & AI) focusing on large enterprise & strategic customers in India. You will help lead our Global Accounts team in India expansion, you will build and lead a technical pre-sales team. You will guide and get involved to enhance...