Databricks Lead

2 weeks ago

Gurgaon Tower B Floor CC, India Axtria Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Position Summary

The Technical Lead – Databricks Lakehouse Migration & Integration will play a pivotal role in enabling our pharmaceutical and life sciences clients to modernize their data platforms. This role involves leading the migration of complex clinical, regulatory, and commercial datasets into the Databricks Lakehouse environment, ensuring compliance with industry standards such as GxP, HIPAA, and GDPR. The incumbent will provide technical leadership, oversee integration strategies, and deliver scalable, secure, and compliant data solutions that support commercial operations, drug development and clinical trials.

Job Responsibilities

Migration & Integration

Lead the migration of legacy data warehouses and research data lakes into the Databricks Lakehouse platform.Architect and implement Delta Lakehouse solutions tailored to life sciences data domainsIntegrate Databricks with upstream laboratory systems, electronic data capture (EDC), ERP, CRM, and downstream analytics/BI platforms

Compliance & Governance

Ensure adherence to GxP, HIPAA, GDPR, and other regulatory requirements in data architecture and workflows.Implement robust governance, metadata management, and security frameworks using Databricks Unity Catalog.

Leadership & Collaboration

Provide technical leadership and mentorship to data engineers and consultants.Collaborate with client stakeholders including R&D, clinical operations, regulatory affairs, and commercial teams.Translate business requirements into compliant, scalable technical solutions.

Optimization & Innovation

Optimize Spark jobs, Delta tables, and streaming pipelines for large-scale clinical and commercial datasetsPromote automation, CI/CD, and DevOps practices in data engineering projectsStay current with Databricks advancements and emerging technologies in pharma data management.

Education BE/B.Tech Master of Computer Application Work Experience

Bachelor's or Master's degree in Computer Science, Information Technology, Bioinformatics, or related field.

7+ years of experience in data engineering, with at least 2–3 years in a technical lead role.

Strong hands-on expertise with Databricks (Spark, SQL, MLflow, Delta Lake).

Deep understanding of Delta Lakehouse architecture and its application to regulated data environments.

Proficiency in PySpark, SQL, and Python for distributed data processing.

Experience with cloud platforms (Azure, AWS, or GCP) and their data services.

Familiarity with ETL/ELT frameworks, data pipelines, and real-time streaming technologies (e.g., Kafka).

Demonstrated experience working with life sciences/pharma datasets (clinical trials, regulatory submissions, pharmacovigilance, commercial analytics).

Behavioural Competencies Teamwork & Leadership Motivation to Learn and Grow Ownership Cultural Fit Talent Management Technical Competencies Problem Solving Lifescience Knowledge Communication Project Management Databricks PySpark Delivery Management- BIM/ Cloud Info Management Skills

  • Manager - BIM

    2 weeks ago

    Gurgaon, Tower B ( Floor ), CC, India Axtria Full time ₹ 6,00,000 - ₹ 15,00,000 per year

    Position Summary Highly skilled GenAI Application Leads with 8 to 15 years of total experience who can lead the design, development, testing, and deployment of Generative AI–based applications focused on Data and Analytics in Life Sciences domain. The ideal candidate will have a strong background in Python, RAG, knowledge graphs, Gen AI/LLM frameworks...

  • Manager - BIM

    2 weeks ago

    Gurgaon, Tower B ( Floor ), CC, India Axtria Full time ₹ 15,00,000 - ₹ 30,00,000 per year

    Position Summary Highly skilled GenAI Application Leads with 8 to 15 years of total experience who can lead the design, development, testing, and deployment of Generative AI–based applications focused on Data and Analytics in Life Sciences domain. The ideal candidate will have a strong background in Python, RAG, knowledge graphs, Gen AI/LLM frameworks...

  • Lead Azure Data Engineer

    5 days ago

    Gurgaon, India Incedo Inc. Full time

    Role: Lead Azure Data EngineerLocation: GurugramExperience: 8+ YearsWork Model: ODC ( 5 days work from office)Technical SkillsAzure Analytics (Databricks, Data Factory)AI/ML (Azure ML, Cognitive Services)Programming (Python/Java/Scala, SQL)DevOps & CI/CD (Azure DevOps, Docker, Kubernetes)Problem-Solving AbilityCommunication SkillsCollaboration with Offshore...

  • Lead Azure Data Engineer

    1 day ago

    Gurgaon, India Incedo Inc. Full time

    Role: Lead Azure Data Engineer Location: Gurugram Experience: 8+ Years Work Model: ODC ( 5 days work from office) Technical Skills Azure Analytics (Databricks, Data Factory) AI/ML (Azure ML, Cognitive Services) Programming (Python/Java/Scala, SQL) DevOps & CI/CD (Azure DevOps, Docker, Kubernetes) Problem-Solving Ability Communication Skills Collaboration...

  • Azure DataOps Lead

    1 week ago

    Gurgaon, Haryana, India Rackspace Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Job Title: Azure DataOps Lead Department: Managed Services – DataOps Location: Remote (India) Reports To: Data Engineering Manager – DataOps Role Overview The Azure DataOpsLeadwillbe responsible for leading the operational delivery, automation, and governance of Azure-based data platforms for enterprise clients. This role involves managing L1/L2/L3...

  • Azure DataOps Lead

    1 week ago

    Gurgaon, Haryana, India Rackspace Technology Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Job Title: Azure DataOps LeadDepartment: Managed Services – DataOps Location: Remote (India) Reports To: Data Engineering Manager – DataOpsRole OverviewThe Azure DataOpsLeadwillbe responsible for leading the operational delivery, automation, and governance of Azure-based data platforms for enterprise clients. This role involves managing L1/L2/L3...

  • Technical Lead

    4 days ago

    Gurgaon, Haryana, India Insight Full time

    Requisition Number: 103064Technical Lead- DataLocation: The role will be a hybrid position located in India.Insight at a Glance14,000+ engaged teammates globally20 on Fortune's World's Best Workplaces list$9.2 billion in revenueReceived 35+ industry and partner awards in the past year$1.4M+ total charitable contributions in 2023 by Insight globallyNow is the...

  • Delivery Head @Chennai/90L

    3 weeks ago

    Gurgaon Division, India Live Connections Full time

    Note: Currently Should be working in the Retail/Ecommerce domain Role: The ideal candidate will have progressed from a datawarehousing/big data engineering background into an architect role (Snowflake, Databricks) and now drives end‑to‑end delivery leadership. This role requires strong expertise in pricing and product catalogue platforms, combined with...

  • Data engineer

    3 weeks ago

    Gurgaon, India GMG Full time

    What we do:GMG is a global well-being company retailing, distributing and manufacturing a portfolio of leading international and home-grown brands across sport, food and health sectors. Its vision is to inspire people to win in ways that make the world better. Today, GMG’s investments span four key verticals: GMG Sports, GMG Food, GMG Health, and GMG...

  • Data Engineer

    3 weeks ago

    Gurgaon, India GMG Full time

    What we do:GMG is a global well-being company retailing, distributing and manufacturing a portfolio of leading international and home-grown brands across sport, food and health sectors. Its vision is to inspire people to win in ways that make the world better. Today, GMG’s investments span four key verticals: GMG Sports, GMG Food, GMG Health, and GMG...