Azure Databricks Data Engineer

3 days ago

Bhopal, Madhya Pradesh, India SAA Consultancy Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Location: Bhopal, MP, India

Experience: 3-5 Years

Key Responsibilities

  • Design and build ETL pipelines in Azure Databricks (PySpark, Delta Lake) to load, clean, and deliver data across Bronze, Silver, and Gold layers.
  • Implement Data Lakehouse Architecture on Azure Data Lake Gen2 with partitioning, schema management, and performance optimization.
  • Develop data models (dimensional/star schema) for reporting in Synapse and Power BI.
  • Integrate Databricks with Azure services – ADF, Key Vault, Event Hub, Synapse Analytics, Purview, and Logic Apps.
  • Build and manage CI/CD pipelines in Azure DevOps (YAML, Git repos, pipelines).
  • Optimize performance through cluster tuning, caching, Z-ordering, Delta optimization, and job parallelization.
  • Ensure data security and compliance (row-level security, PII masking, GDPR/HIPAA, audit logging).
  • Collaborate with data architects and analysts to translate business needs into technical solutions.

Required Skills

  • Strong experience in Azure Databricks (Python, PySpark, SQL).
  • Proficiency with Delta Lake (ACID transactions, schema evolution, incremental loads).
  • Hands-on with Azure ecosystem – Data Factory, ADLS Gen2, Key Vault, Event Hub, Synapse.
  • Knowledge of data governance & lineage tools (Purview, Unity Catalog is a plus).
  • Strong understanding of data warehouse design and star schema.
  • Azure DevOps (YAML, Git repos, pipelines) experience.
  • Good debugging skills for performance tuning & schema drift issues.

Good to Have

  • Experience with healthcare or financial data.
  • Familiarity with FHIR, OMOP, OpenEHR (for healthcare projects).
  • Exposure to AI/ML integration using Databricks ML runtime.
  • Experience with Unity Catalog for governance across workspaces.

What You Will Deliver

  • Automated snapshot + incremental pipelines in Databricks.
  • Delta Lake architecture with partitioning, Z-ordering, and schema drift handling.
  • Metadata-driven ingestion framework (YAML configs).
  • Power BI datasets connected to Gold layer.
  • CI/CD pipelines for deployment across.

Are you ready to take the lead in building scalable data solutions with Azure Databricks? Apply now

  • Lead Azure Databricks Data Engineer

    1 week ago

    Bhopal, Madhya Pradesh, India SAA Consultancy Full time ₹ 1,04,000 - ₹ 13,08,780 per year

    Location: Bhopal, MP, IndiaExperience: 7+ yearsKey ResponsibilitiesDesign and build ETL pipelines in Azure Databricks (PySpark, Delta Lake) to load, clean, and deliver data across Bronze, Silver, and Gold layers.Implement Data Lakehouse Architecture on Azure Data Lake Gen2 with partitioning, schema management, and performance optimization.Develop data models...

  • azure data engineer

    2 weeks ago

    Bhopal, Madhya Pradesh, India Eagle in Cloud Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    WE ARE HIRINGJob Role- Azure Data EngineerExperience - 7+ YearsLocation - IndoreNotice Period - Immediate JoinersPrimary Skills: Advanced SQL Server, ETL development using SSIS, Azure Data Factory, and Databricks, Power BI, Data warehousing, Data quality and validation, General Microsoft technology stackRole DescriptionThis is a full-time on-site role...

  • Senior Data Engineer/Data Architect

    1 week ago

    Bhopal, Madhya Pradesh, India Intertec Systems Full time

    Skills : Azure Data Fabric, snowflake, synapse, Databricks, Data engineering, AWS, Data Architecture, Data Integration. We are seeking an experienced Presales Solution Architect with deep Microsoft Data Platform & AI expertise, combined with knowledge and exposure to other modern data platforms such as Snowflake, SAAL. AI, Databricks, Synapse, and BigQuery....

  • Chief Data Architect and Engineering Specialist

    1 week ago

    Bhopal, Madhya Pradesh, India beBeeDataEngineer Full time US$ 1,00,000 - US$ 1,40,000

    Senior Data Architect and EngineerThis role involves designing, implementing, and managing large-scale data architectures.Lead the design of enterprise-grade Microsoft Data Platform solutions including Azure Synapse Analytics, Azure Data Lake, Azure SQL, Cosmos DB, Azure Data Factory, Azure Stream Analytics, Azure ML, and Microsoft Purview.Architect hybrid...

  • Presales Solution Architect

    2 weeks ago

    Bhopal, Madhya Pradesh, India Intertec Systems Full time

    Skills : Solution Architecture, Data Platform Design, Cloud Platforms, Pre-sales Consulting, Microsoft Fabric, Azure Synapse Analytics, Azure Data Lake, Azure SQL,Job Description : We are seeking an experienced Presales Solution Architect with deep Microsoft Data Platform & AI expertise, combined with knowledge and exposure to other modern data platforms...

  • Data Engineering Specialist

    2 weeks ago

    Bhopal, Madhya Pradesh, India beBeeExpert Full time ₹ 12,00,000 - ₹ 20,00,000

    Analytics and Reporting ExpertWe are seeking a highly skilled Analytics and Reporting Expert to develop and maintain cutting-edge analytics and reporting applications. The ideal candidate should have extensive experience working with Python (Flask), Go, and modern reporting platforms. Strong familiarity with Azure and relational databases is essential. Key...

  • Data Extraction Engineer

    2 weeks ago

    Bhopal, Madhya Pradesh, India Relu Consultancy Full time ₹ 50,000 - ₹ 28,00,000 per year

    Relu Consultancy is seeking a Data Extraction Engineer with expertise in Python (Selenium). In this role, you will design, implement, and maintain robust data scraping solutions that drive our projects forward. This is your chance to contribute to cutting-edge initiatives while enjoying a work schedule tailored to your needs.Job Title:Data Extraction...

  • IOT engineer

    2 weeks ago

    Bhopal, Madhya Pradesh, India Weekday AI Full time ₹ 5,00,000 - ₹ 8,00,000 per year

    This role is for one of the Weekday's clientsSalary range: Rs Rs ie INR 5-8 LPA)Min Experience: 3 yearsJobType: full-time We are seeking a skilled IoT Engineer with strong expertise in developing and deploying IoT-based solutions. The ideal candidate will have hands-on experience with sensor integration, device connectivity, real-time monitoring, and...

  • IOT engineer

    2 weeks ago

    Bhopal, Madhya Pradesh, India Weekday AI Full time ₹ 5,00,000 - ₹ 8,00,000 per year

    This role is for one of the Weekday's clientsSalary range: Rs Rs ie INR 5-8 LPA)Min Experience: 3 yearsJobType: full-timeWe are seeking a skilled IoT Engineer with strong expertise in developing and deploying IoT-based solutions. The ideal candidate will have hands-on experience with sensor integration, device connectivity, real-time monitoring, and...

  • IOT engineer

    2 weeks ago

    Bhopal, Madhya Pradesh, India Weekday AI Full time ₹ 5,00,000 - ₹ 8,00,000

    This role is for one of the Weekday's clientsSalary range: Rs Rs ie INR 5-8 LPA)Min Experience: 3 yearsJobType: full-timeWe are seeking a skilled IoT Engineer with strong expertise in developing and deploying IoT-based solutions. The ideal candidate will have hands-on experience with sensor integration, device connectivity, real-time monitoring, and...