Senior Databricks Engineer

2 days ago

Pune, Maharashtra, India beBeeDatabricks Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

Job Opportunity: Databricks Platform Expert

We are seeking a seasoned professional to lead the design, deployment, and management of secure, scalable, and high-performance Databricks environments for insurance data processing and analytics.

Key Responsibilities:

  • Design, deploy, and manage secure, scalable, and high-performance Databricks environments for insurance data processing and analytics.
  • Collaborate with architects, engineers, and security teams to establish infrastructure standards and ensure connectivity with legacy systems, cloud-native data sources, and third-party platforms.
  • Implement automated provisioning and configuration of Azure and Databricks resources using Terraform, supporting enterprise DevOps practices.
  • Build and maintain GitHub Actions-based automation pipelines for environment deployment, monitoring, and incident response to ensure consistency, traceability, and efficiency.
  • Monitor platform health, optimize cluster utilization, and conduct tuning to ensure reliability, scalability, and cost efficiency for mission-critical workloads.
  • Enforce enterprise and regulatory standards (RBAC, Key Vault, Managed Identities, Encryption, PDPO, GDPR, insurance compliance).
  • Lead integration of Informatica tools for metadata management, cataloguing, lineage tracking, and compliance across the platform.
  • Prepare infrastructure architecture diagrams, network topology, security configurations, and runbooks for governance, audits, and operational readiness.
  • Troubleshoot platform and infrastructure issues, perform RCA, and drive continuous improvements to enhance reliability.
  • Stay up to date with Azure, Databricks, and DevOps advancements, proactively recommending enhancements for platform efficiency and business value.

Requirements:

  • Bachelor's degree in Computer Science, IT, or related field (Master's preferred).
  • 7-14 years of experience in Azure cloud engineering, with at least 5+ years specialized in Databricks platform management.
  • Proven expertise in provisioning and managing Databricks clusters & workspaces for processing structured, semi-structured, and unstructured data in insurance/banking domains.
  • Hands-on experience integrating Azure Data Factory (ADF) and Azure Data Lake Storage Gen2 with Databricks for automated ETL/ELT pipelines.
  • Strong proficiency in Infrastructure as Code (Terraform) for automated deployments and resource management.
  • Experience deploying and integrating Informatica solutions for metadata, lineage, and governance in large-scale data ecosystems.
  • Deep knowledge of cloud security, identity & access management, encryption, and compliance frameworks in regulated industries (insurance/financial services preferred).
  • Practical expertise in GitHub Actions or similar CI/CD tools for platform automation and DevOps workflows.
  • Proven ability in troubleshooting, performance optimization, and incident management for high-availability data platforms.
  • Excellent documentation, collaboration, and stakeholder communication skills, with experience supporting both technical and business users.

  • Senior Databricks Engineer

    4 weeks ago

    Pune, Maharashtra, India Growel Softech Pvt. Ltd. Full time

    Senior Databricks EngineerDescription :We are seeking a Senior Databricks Engineer to join our dynamic team in India. The ideal candidate will have extensive experience in building and optimizing data pipelines and will play a crucial role in leveraging Databricks to drive data :- Design and implement scalable data pipelines using Databricks.- Optimize...

  • Senior Databricks Engineer

    6 days ago

    Pune, Maharashtra, India Newscape Consulting Full time

    We're Hiring: Senior Databricks Engineer / Azure Data Engineer

  • Senior Databricks Engineer

    6 days ago

    Pune, Maharashtra, India Newscape Consulting Full time

    We're Hiring: Senior Databricks Engineer / Azure Data Engineer Location: Baner, Pune Work Mode: Onsite Availability: Immediate Joiners / 15 Days NP Are you passionate about building scalable, high-performance data platforms? We're looking for a Senior Databricks Engineer / Azure Data Engineer to join our growing team and work on cutting-edge Azure Data...

  • Senior Databricks Engineer

    6 days ago

    Pune, Maharashtra, India Newscape Consulting Full time

    We're Hiring: Senior Databricks Engineer / Azure Data Engineer Location: Baner, Pune Work Mode: Onsite Availability: Immediate Joiners / 15 Days NPAre you passionate about building scalable, high-performance data platforms?We're looking for a Senior Databricks Engineer / Azure Data Engineer to join our growing team and work on cutting-edge Azure Data...

  • Senior Databricks Engineer

    2 days ago

    Pune, Maharashtra, India Newscape Consulting Full time

    We're Hiring: Senior Databricks Engineer / Azure Data Engineer Location: Baner, Pune Work Mode: Onsite Availability: Immediate Joiners / 15 Days NPAre you passionate about building scalable, high-performance data platforms? We're looking for a Senior Databricks Engineer / Azure Data Engineer to join our growing team and work on cutting-edge Azure Data...

  • Senior Databricks Engineer

    24 hours ago

    Pune, Maharashtra, India Newscape Consulting Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    We're Hiring: Senior Databricks Engineer / Azure Data EngineerLocation:Baner, PuneWork Mode:OnsiteAvailability:Immediate Joiners / 15 Days NPAre you passionate about buildingscalable, high-performance data platforms?We're looking for aSenior Databricks Engineer / Azure Data Engineerto join our growing team and work oncutting-edge Azure Data Engineering...

  • Senior Data Engineer Databricks

    2 weeks ago

    Pune, Maharashtra, India Simbus Technologies Pvt. Ltd. Full time

    At Simbus, we bring Richard Branson's saying to life: "Train people well enough so they can leave, treat them well enough so they don't want to."JOB OVERVIEW:We are seeking a highly skilled and experienced Senior Data Engineer with a strong background in Databricks to join our dynamic team. The ideal candidate will have over 4 years of experience in data...

  • Databricks Engineer

    24 hours ago

    Pune, Maharashtra, India Bitwise Solutions Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Role: DataBricks EngineerNotice Period: Immediate joiners onlyLocation: PuneWork Mode: HybridPosition Overview:We need a skilled Databricks Data Engineer with strong analytical and troubleshooting abilities. The ideal candidate should have hands-on experience in building scalable data pipelines using PySpark on cloud platforms like Azure/AWS along with a...

  • Databricks Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Ascendion Full time

    Job Title: Senior Data Engineer Experience : 3+ years Location: Gurgaon Skills: PySpark, SQL, Databricks, AWS. Role Summary: We are looking for 3–4 experienced Databricks Developers to support a fast-paced, high-impact data engineering initiative. The ideal candidates should have hands-on expertise in building scalable data pipelines using...

  • Databricks Data Engineer

    6 days ago

    Pune, Maharashtra, India Ascendion Full time

    Job Title: Senior Data EngineerExperience : 3+ yearsLocation: Gurgaon /Pune / BangaloreSkills: PySpark, SQL, Databricks, AWS.Role Summary:We are looking for 3–4 experienced Databricks Developers to support a fast-paced, high-impact data engineering initiative. The ideal candidates should have hands-on expertise in building scalable data pipelines using...