Current jobs related to Azure Databricks Lead Engineer - India - Alcor Solutions, Inc.
-
Azure Databricks Engineer
3 days ago
India Proso Full timeProso AI is Hiring – Azure Databricks Engineer (Contract Role)We are looking for an experienced Azure Databricks Engineer for a short-term role (3–4 months). Experience Required: 7+ years Key Skills & Responsibilities:- Create/modify Azure Data Factory pipelines- Develop and optimize Databricks notebooks using SQL & Python- Work with file formats:...
-
Azure Databricks Engineer
5 days ago
India Proso Full timeProso AI is Hiring – Azure Databricks Engineer (Contract Role) We are looking for an experienced Azure Databricks Engineer for a short-term role (3–4 months). Experience Required: 7+ years Key Skills & Responsibilities: Create/modify Azure Data Factory pipelines Develop and optimize Databricks notebooks using SQL & Python Work with file formats:...
-
Azure Databricks
2 days ago
India Zensar Technologies Full time**Data Architect (Azure, ADF, Databricks) - 14+ Years Experience** **About the Role**: **Responsibilities**: - **Data Platform Architecture**: - Design and architect a comprehensive data platform on Azure, leveraging best practices for scalability, performance, security, and cost optimization. - Define the overall data architecture framework, including...
-
Lead Azure Data Engineer
1 day ago
India RandomTrees Full timeImmediate #HIRING for a highly motivated and experienced Lead Azure Data Engineer to join our growing team. We're a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies If you're passionate about building robust applications and thrive in a dynamic...
-
Azure Databricks
39 minutes ago
India Zensar Technologies Full timeAs an Azure Developer, you will be responsible for migration to Azure Data Cloud or implementing data solutions using Azure Data Cloud services. Here's how you'll contribute: You will contribute to the role by: - Creating/maintaining scalable ETL pipeline with Azure data factory or Azure Data BricksIngesting data from various sources and building scalable...
-
Innovative Azure Databricks Specialist
3 hours ago
India beBeeEngineer Full time ₹ 15,00,000 - ₹ 20,00,000Azure Databricks Ingestion EngineerWe are seeking an experienced data ingestion specialist to join our team. This role requires a strong understanding of data ingestion processes, focusing on integrating various data sources into Azure Databricks.Key Responsibilities:Data Ingestion Pipeline Development: Design, develop, and optimize data ingestion pipelines...
-
India Nexify Infosystems Full timeTitle: Azure Databricks Engineer with Injection experienceLocation: Remote (India)Duration: 6 Month, (if the candidate is good with work, it can be extended to 1+ year )Time: 10.00 AM- 7.00 PM ISTNeed good comm skills – they will be coordinating with Americans on daily basesfor clarity : We are seeking an Ingestion Engineer with expertise in CI/CD and...
-
Data Engineer – Databricks
5 hours ago
India Epergne Solutions Full timeProject Overview We are seeking skilled Data Engineers with strong hands-on experience in Databricks for a high-impact data platform project The project involves building scalable data pipelines integrating with cloud platforms and managing data governance using Unity Catalog Key Requirements Primary Skills Expertise in Databricks working on...
-
Databricks Engineer
3 weeks ago
India DataE2E Technologies Full timeKey Responsibilities- Design, develop, and maintain data pipelines and ETL processes on Databricks.- Lead the architecture and implementation of scalable, secure, and high-performance data solutions.- Collaborate with business stakeholders, data scientists, and analysts to gather requirements and deliver actionable data solutions.- Optimize data workflows...
-
Databricks Engineer
3 weeks ago
India DataE2E Technologies Full timeKey ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes on Databricks.Lead the architecture and implementation of scalable, secure, and high-performance data solutions.Collaborate with business stakeholders, data scientists, and analysts to gather requirements and deliver actionable data solutions.Optimize data workflows for...
Azure Databricks Lead Engineer
4 weeks ago
Alcor Solutions is a leading digital transformation and cloud consulting firm, enabling enterprises to leverage modern data platforms, cloud-native solutions, and emerging technologies to drive innovation and business outcomes.
We are seeking an Azure Databricks Lead Engineer to architect, administer, and scale our enterprise Databricks environment while establishing best practices and governance across the organization.
Role Overview
The Azure Databricks Lead Engineer will serve as the subject matter expert (SME) for our Databricks platform, overseeing its architecture, administration, and optimization across the enterprise.
This individual will be responsible for designing the Databricks Medallion architecture, implementing Unity Catalog governance, managing cluster policies, and driving the adoption of Databricks best practices. The role requires a combination of hands-on administration and strategic platform design, ensuring Databricks is secure, scalable, and fully aligned with enterprise data initiatives.
- Design and implement Databricks Medallion architecture for enterprise-wide data usage.
- Define workspace strategy, including design, configuration, and organization for multi-team and multi-project usage.
- Establish best practices for data ingestion, transformation, and consumption using Databricks.
- Administer and optimize the Azure Databricks platform (clusters, pools, compute policies).
- Set up and manage Unity Catalog, including access control, catalog/schema security, and governance for PII and PHI data.
- Create and enforce cluster management policies, workspace permissions, and environment segregation (dev/test/prod).
- Monitor platform health, usage, and costs; implement scaling and tuning strategies.
- Collaborate with cloud infrastructure, networking, and security teams to ensure Databricks integrates with enterprise systems and policies.
- Support data engineers, data scientists, and analytics teams by enabling seamless deployment of applications and pipelines on Databricks.
- Act as the primary technical advisor and escalation point for Databricks-related issues.
- Define processes for sizing and provisioning Databricks clusters based on project requirements.
- Establish governance for non-production and production environments to ensure compliance and cost-efficiency.
- Build guidelines for version control, Unity Catalog repository configuration, and DevOps pipelines.
- Serve as the Databricks SME across the enterprise, evangelizing best practices and new features.
- Mentor internal teams and conduct workshops or training sessions to upskill stakeholders on Databricks usage.
- Stay current with Databricks and Azure advancements, ensuring the enterprise platform evolves with industry standards.
Experience & Qualifications:
- 7+ years of relevant experience in data engineering, cloud data platforms, or similar roles.
- Proven track record of Databricks administration and architecture, ideally with formal Medallion architecture experience.
- Experience in managing large-scale Databricks environments across multiple teams and projects.
- Deep expertise in Azure Databricks, Unity Catalog, and Delta Lake.
- Strong knowledge of cluster configuration, compute policies, and workspace design
- Familiarity with Azure ecosystem (Azure Data Lake, Key Vault, Azure Active Directory).
- Understanding of data governance, security policies, and compliance standards (handling of PII/PHI).
- Experience with CI/CD pipelines, version control (Git), and infrastructure-as-code practices is desirable.
