Azure Databricks Lead

15 hours ago

Mumbai, Maharashtra, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878
Job Description

We are seeking a skilled professional to lead our Azure Databricks initiative. As a key member of our team, you will be responsible for designing and developing data pipelines using Azure Databricks and PySpark/Scala.

You will work closely with our data engineers and architects to ensure efficient data movement and transformation. Additionally, you will establish data quality checks and validation rules within Azure Databricks, monitor data quality metrics, and address anomalies promptly.

Furthermore, you will leverage Azure Databricks Unity Catalog to manage metadata, tables, and views. You will integrate Databricks assets seamlessly with other Azure services and ensure proper documentation and organization of data assets.

Your expertise in Delta Lake will enable you to understand and utilize its capabilities, including ACID transactions and time travel. You will implement Delta Lake tables for reliable data storage and versioning, as well as optimize performance by leveraging Delta Lake features.

In this role, you will also profile and analyze query performance, optimize SQL queries, Spark jobs, and transformations for efficiency, and tune resource allocation to achieve optimal execution times.

As a seasoned professional, you will have managed compute resources effectively within Azure Databricks clusters, scaled clusters dynamically based on workload requirements, and monitored resource utilization and cost efficiency.

  • Azure Databricks Developer

    3 days ago

    Mumbai, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineer - Azure DatabricksOur organization seeks a skilled Data Engineer to lead the development of our data infrastructure using Azure Databricks and Pyspark.Key Responsibilities:Databricks Development:Design, develop, and deploy scalable and efficient data pipelines on Databricks.Data Engineering Expertise:Expert knowledge of Databricks, including...

  • Databricks Developer

    4 days ago

    Mumbai, Maharashtra, India Seaspan Corporation Full time

    Position Description : We are seeking a skilled Databricks Developer to join our Data Development team. Reporting to the Team Lead, Data Development, the Databricks Developer implements robust data pipelines using Apache Spark on Databricks, supports advanced data transformation, and enables scalable data products that serve enterprise analytics and...

  • Databricks Developer

    4 days ago

    Mumbai, Maharashtra, India Seaspan Corporation Full time US$ 1,20,000 - US$ 2,00,000 per year

    Seaspan teams are goal-driven and share a high-performance culture, focusing on building services offerings to become a leading asset manager. Seaspan provides many of the world's major shipping lines with alternatives to vessel ownership by offering long-term leases on large, modern containerships and pure car, truck carriers (PCTCs) combined with industry...

  • Senior Databricks Solutions Architect

    4 days ago

    Mumbai, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OverviewThis role focuses on leading and implementing data solutions using Databricks on the Azure cloud.The ideal candidate will have experience in designing, building, and optimizing Data Warehouses (DWH) and Data Lakes with scalable ETL applications. They should also be proficient in working with metadata-driven processing, CDC, and advanced...

  • Databricks

    2 weeks ago

    Mumbai, Maharashtra, India Talent Worx Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Every day, your work will make an impact that matters, while you thrive in a dynamic culture of inclusion, collaboration, and high performance. As the undisputed leader in professional services, Our Client is where you will find unrivaled opportunities to succeed and realize your full potential.We are seeking highly skilled Databricks Data Engineers to join...

  • Data Architect-Databricks

    5 days ago

    Navi Mumbai, Maharashtra, India Celebal Technologies Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    We Are Hiring Databricks Data Architect | Navi Mumbai (Onsite)Are you passionate about designing scalable, enterprise-grade data platforms?Join Celebal Technologies and work on cutting-edge Azure Databricks solutions in the manufacturing and energy sectorsRole: Databricks Data ArchitectExperience: 10+ YearsLocation: Navi Mumbai (Onsite)Notice Period:...

  • Azure Data Engineer

    5 days ago

    Mumbai, Maharashtra, India Celebal Technologies Full time US$ 1,00,000 - US$ 1,50,000 per year

    Overview: As a Data Engineer you will work with multiple teams to deliver solutions on the Azure Cloud using core cloud data warehouse tools (Azure Data Factory, Azure Databricks, Azure Synapse Analytics and other Big Data related technologies). In addition to building the next generation of application data platforms (not infrastructure) and/or improving...

  • High-Performance Databricks Developer

    2 weeks ago

    Mumbai, Maharashtra, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Contract Role: High-Performance Databricks DeveloperAbout the PositionWe are seeking an experienced Databricks developer to join our team on a short-term contract basis. As a key member of our engineering team, you will be responsible for designing and implementing high-performance data pipelines using Azure Data Factory.Key Responsibilities:Design and...

  • Azure Data Engineer

    7 days ago

    Mumbai, Maharashtra, India LTIMindtree Full time

    Skill: Azure Data EngineerPrimary skill: ADB,ADF,SQLSecondary skill: Py spark or PythonExperience: 5-12YRSLocation: Hyderabad, Pune, Mumbai, Kolkata, Nagpur, Bangalore,ChennaiNotice period: Imm to 30 DaysJD: Proactive Q2 Azure Data Factory ADFP3 Job Description Azure Databricks DeveloperLocation India ISTOverview Data Engineer having good experience on Azure...

  • Azure Data Engineer

    3 days ago

    Mumbai, Maharashtra, India LTIMindtree Full time

    Skill: Azure Data Engineer Primary skill: ADB,ADF,SQL Secondary skill: Py spark or Python Experience: 5-12YRS Location: Hyderabad, Pune, Mumbai, Kolkata, Nagpur, Bangalore,Chennai Notice period: Imm to 30 Days JD: Proactive Q2 Azure Data Factory ADFP3 Job Description Azure Databricks Developer Location India IST Overview Data Engineer having good experience...