Senior Databricks Engineer

1 week ago

Pune, Maharashtra, India Newscape Consulting Full time

About the Role

We are seeking a 
Senior Databricks Engineer
 to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data ecosystems to power business insights and AI/ML initiatives.

Key Responsibilities

  • Design & Development:
  • Build and maintain scalable ETL/ELT data pipelines using 
    Apache Spark
     and 
    Databricks
    .
  • Develop 
    Delta Lake
     architectures and optimize data storage for performance and cost.
  • Automate data ingestion, transformation, and orchestration processes across multiple data sources.
  • Build different stages of medallion architecture such as Bronze, Silver and Gold layer
  • Data Architecture & Modeling:
  • Collaborate with data architects to design data lakehouse and warehouse solutions.
  • Implement 
    data quality frameworks
    , versioning, and governance standards (e.g., Unity Catalog).
  • Optimize Spark jobs, cluster configurations, and query performance.
  • Cloud & Infrastructure:
  • Work with 
    Azure Databricks or AWS
     to configure and manage cloud-based data platforms.
  • Develop and manage CI/CD pipelines for Databricks notebooks, workflows, and jobs.
  • Implement security best practices for data access, encryption, and compliance.
  • Collaboration & Leadership:
  • Partner with data scientists, analysts, and product teams to enable data-driven decisions.
  • Provide technical mentorship to junior engineers.
  • Contribute to strategic planning and architecture reviews.

Required Qualifications

  • Bachelor's or Master's degree in 
    Computer Science
    , 
    Information Systems
    , 
    Engineering
    , or related field.
  • 6+ years
     of experience in data engineering or software development.
  • 4+ years
     hands-on experience with 
    Databricks
     and 
    Apache Spark
    .
  • Strong proficiency in 
    Python
    , 
    SQL
    , and 
    PySpark
    .
  • Experience with 
    Delta Lake
    , 
    Data Lakehouse
    , and 
    Data Warehousing
     concepts.
  • Solid understanding of 
    data modeling
    , 
    ETL design
    ,
    medallion architecture
    and 
    distributed data processing
    .
  • Familiarity with 
    CI/CD tools
     (Azure DevOps, GitHub Actions, Jenkins, etc.).
  • Cloud platform expertise in 
    Azure
    , 
    AWS
    , or 
    GCP
    .

Preferred Qualifications

  • Experience with 
    machine learning pipelines
     in Databricks (MLflow).
  • Knowledge of 
    data governance
     and 
    compliance frameworks
    .
  • Experience integrating Databricks with BI tools like 
    Power BI
    , 
    Tableau
    , or 
    Looker
    .
  • Certifications such as:
  • Databricks Certified Data Engineer Associate or Professional
  • Azure Data Engineer Associate
  • AWS Big Data Specialty

Soft Skills

  • Strong problem-solving and analytical thinking.
  • Excellent communication and collaboration abilities.
  • Proactive mindset with ownership of deliverables.
  • Passion for learning emerging data technologies.

Why Join Us

  • Work with cutting-edge data technologies on large-scale projects.
  • Opportunity to shape the organization's data strategy and architecture.
  • Collaborative and innovative culture focused on continuous growth.

  • Senior Databricks Specialist

    1 day ago

    Pune, Maharashtra, India KPI Partners Full time

    Job Description: KPI Partners is seeking a highly skilled and experienced Senior Databricks Specialist to join our dynamic team. The ideal candidate will have a strong background in data engineering and analytics, with expertise in Databricks and related technologies. Key Responsibilities: - Design, develop, and maintain data pipelines using Databricks...

  • Newscape - Senior Databricks Engineer - Azure Data Services

    3 days ago

    Pune, Maharashtra, India Newscape Consulting Full time

    DescriptionWere Hiring : Senior Databricks Engineer / Azure Data Engineer / Lead Data engineer.Location : Baner, Pune (Onsite Only).Work Mode : Full-time | Office-based.Availability : Immediate Joiners / Notice Period ? 30 Summary :We are looking for a Senior Databricks Engineer / Azure Data Engineer who will be responsible for designing and implementing...

  • Azure Databricks Engineer

    2 weeks ago

    Pune, Maharashtra, India Savi Technologies Full time

    Opening: Azure Databricks EngineerLocation: Pune, Maharashtra (Onsite)Experience Range: 5-6 YearsApply at: -Required Skills:Azure Data Factory (ADF)PySparkAzure Databricks (ADB)SQLJob Description:We're looking for an experienced Azure Databricks Engineer with a background in financial crime analytics and hands-on expertise in:Databricks engineering &...

  • Databricks

    5 days ago

    Pune, Maharashtra, India Talent Worx Full time

    Every day, your work will make an impact that matters, while you thrive in a dynamic culture of inclusion, collaboration, and high performance. As the undisputed leader in professional services, Our Client is where you will find unrivaled opportunities to succeed and realize your full potential.We are seeking highly skilled Databricks Data Engineers to join...

  • Databricks Data Engineer

    3 weeks ago

    Pune, Maharashtra, India, Maharashtra Ascendion Full time

    Job Title: Senior Data EngineerExperience : 5+ years Location: Gurgaon /Pune / BangaloreSkills : Databricks , Delta Lake, PySpark , SQL Desired Skills and Experience: • 3–5 years of experience in data engineering, preferably with Azure Databricks. • Strong technical expertise in PySpark, SQL, and Delta Lake.• Familiarity with Unity Catalog, data...

  • Azure Databricks Developer

    2 weeks ago

    Pune, Maharashtra, India Newscape Consulting Full time ₹ 8,00,000 - ₹ 16,00,000 per year

    We're Hiring: Senior Databricks Engineer / Azure Data Engineer Location:Baner, Pune (Onsite Only) Work Mode:Full-time | Office-based Availability:Immediate Joiners / Notice Period ≤ 15 Days Note:Only local Pune candidates or those willing to relocate to Pune may apply.* L2 interviews will be conductedin person— apply only if you are open to face-to-face...

  • Databricks Solution Architect

    2 weeks ago

    Pune, Maharashtra, India Codvo Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Title: Databricks Solution ArchitectExperience: 10+ yearsAbout UsAt Codvo, we are committed to building scalable, future-ready data platforms that power business impact. We believe in a culture of innovation, collaboration, and growth, where engineers can experiment, learn, and thrive. Join us to be part of a team that solves complex data challenges with...

  • Azure Databricks Data Engineer

    6 days ago

    Pune, Maharashtra, India Reveille Technologies,Inc Full time

    Hiring: Azure Databricks Data Engineer (FTE | Face-to-Face Interview) Location: Pune Experience: 9+ Years Notice Period: 0–30 Days (Serving candidates preferred)We're looking for an experienced Data Engineer with strong hands-on expertise in Azure Databricks and a solid data background. Key Skills:Primary: Azure, Databricks, ADF, PySpark/PythonSecondary:...

  • Databricks Solution Architect

    2 weeks ago

    Pune, Maharashtra, India Codvo Full time ₹ 8,00,000 - ₹ 16,00,000 per year

    Job Title: Databricks Solution ArchitectExperience: 10+ yearsAbout UsAt Codvo, we are committed to building scalable, future-ready data platforms that power business impact. We believe in a culture of innovation, collaboration, and growth, where engineers can experiment, learn, and thrive. Join us to be part of a team that solves complex data challenges with...

  • Databricks Developer

    4 days ago

    Pune, Maharashtra, India Tata Technologies Full time

    Dear Candidate,We at TATA Technologies looking for an experienced candidate for Databricks. Please check the below JD for more detailsJob Title : Databricks DeveloperTotal Exp:3-6 YearsLocation: PuneNotice Period: Immediate-30 DaysRole & responsibilitiesDesign and implement Databricks-based data architectures to meet business requirements.Develop and...