Databricks Lead

2 weeks ago

Alappuzha, India Mastech Digital Full time

Job Title: Databricks Lead Location: Chennai/Hyderabad/Bangalore Experience: 8-10 years of overall experience and 5+ years of hands-on leadership in data engineering projects Budget: Open to discuss Notice Period: Immediate joiner/ Serving notice with less than 60 days/Notice is less than 60 days About the Role We are looking for a highly skilled Databricks Lead with 8–10 years of overall experience and 5+ years of hands-on leadership in data engineering projects. The ideal candidate will lead teams in designing, building, and optimizing scalable data pipelines and Lakehouse solutions on Databricks. Key Responsibilities - Lead the development and delivery of end-to-end data pipelines using Databricks, Spark, Delta Lake, and PySpark. - Drive implementation of Lakehouse/Medallion architectures (Bronze/Silver/Gold). - Guide and mentor data engineers, ensure best coding and architectural practices. - Optimize Spark jobs, Databricks clusters, and workflows for performance and cost. - Implement CI/CD pipelines, version control, and DevOps practices for Databricks (Repos, Workflows, Terraform). - Collaborate with architects, data scientists, and business teams to deliver scalable data solutions. - Ensure data quality, governance, security, and compliance standards are followed (Unity Catalog preferred). Required Skills & Experience - 8–10 years in Data Engineering or Cloud Data Platforms. - 5+ years leading data engineering teams or technical delivery. - Strong hands-on experience with Databricks, Spark, PySpark, SQL, and Delta Lake. - Experience with cloud platforms (AWS/Azure/GCP) and related data services. - Solid understanding of data modeling, ingestion frameworks, orchestration, and performance tuning. - Experience with CI/CD, Git, Terraform, Databricks Repos, and Workflows. - Strong problem-solving, leadership, and communication skills. Preferred Qualifications - Databricks certifications (Associate or Professional). - Experience with streaming technologies (Kafka, Kinesis, Event Hubs). - Familiarity with MLflow and data science integration. What We Offer - Opportunity to lead cutting-edge data engineering initiatives. - Collaborative, high-growth environment. - Competitive compensation and professional development opportunities.

  • Lead Data Architect for Sustainability Initiative

    2 weeks ago

    alappuzha, India beBeeDataEngineering Full time

    Job Title:Databricks Data Engineer LeadThis role involves building data pipelines and models to support product-level carbon footprint analysis.The engineer will be responsible for integrating structured engineering, manufacturing, and supplier data into a unified model using Databricks on AWS.About the Role:We are seeking a senior Databricks Data Engineer...

  • Cloud Data Engineer

    4 days ago

    alappuzha, India beBeeDataEngineer Full time

    Azure data engineer plays a pivotal role in leading cross-functional teams to develop innovative analytics solutions on Azure Databricks. With experience in Agile methodologies and tools like Jira for planning Program Increments and Sprints, this leader is well-equipped to drive collaboration and deliver high-quality results.Designs and implements end-to-end...

  • Lead Data Solutions Architect

    2 weeks ago

    alappuzha, India beBeeDataEngineer Full time

    Azure Data Engineer RoleAs a data engineer, you play a crucial role in designing, developing, and maintaining the data platform and systems required for data storage, processing, and analysis. Your responsibilities will include creating efficient data pipelines and ETL processes to support data integration and analysis, collaborating with data scientists and...

  • Chief Data Pipeline Architect

    2 weeks ago

    alappuzha, India beBeeDataEngineer Full time

    Senior Data Engineer RoleWe are looking for a highly skilled Senior Data Engineer to lead our data engineering team. The ideal candidate will have extensive hands-on experience in building and optimizing data ingestion and transformation pipelines using Azure Data Platform, Databricks, Snowflake, and Kafka.The selected individual will be responsible for...

  • Chief Data Officer

    2 weeks ago

    alappuzha, India beBeeDataEngineering Full time

    Data Engineering OpportunityWe are seeking an experienced Data Engineer to join our team.Key Responsibilities:Databricks and Unity Catalog ManagementAchieve a high level of expertise in Unity Catalog, including creating and managing catalogs, schemas, and tables.Automation of Data Onboarding and Metadata RegistrationLeverage Unity Catalog APIs or Databricks...

  • Leading Data Science Expert

    2 weeks ago

    alappuzha, India beBeeDataScientist Full time

    Job OverviewAs a Senior Data Scientist, you will play a pivotal role in driving business growth through data-driven insights.This is an excellent opportunity to leverage your expertise in data processing and modeling to propel our organization forward.We are seeking a highly skilled professional with strong hands-on experience in PySpark and Databricks data...

  • Senior Enterprise Data Architect

    2 weeks ago

    alappuzha, India beBeeData Full time

    Job DescriptionWe are seeking a highly skilled Senior Data Architect to lead the design and delivery of enterprise data & analytics solutions. This is a senior-level role that requires 12-15 years of experience in data engineering, analytics, or data architecture roles with progressive ownership.The ideal candidate will have hands-on experience building and...

  • Chief Data Migration Specialist

    2 weeks ago

    alappuzha, India beBeeData Full time

    Senior Data Engineer (Telecommunications Domain)We are seeking an experienced Senior Data Engineer with a strong background in the Telecommunications (Telco) industry to lead and execute data migration initiatives from on-premises environments to cloud platforms.The ideal candidate will have hands-on experience with Databricks, deep knowledge of data...

  • Advanced Cloud Data Processing Specialist

    7 days ago

    alappuzha, India beBeeDataEngineer Full time

    As a Cloud Data Engineer, you will play a key role in designing and developing scalable data processing pipelines using cloud-native tools.Job Responsibilities:Data Pipeline Development: Design, develop, and maintain complex ETL processes using cloud-based services like AWS DMS, AWS Glue, and Azure Data Factory.Cloud Platform Expertise: Architect and...

  • Senior Backend Systems Developer

    5 days ago

    alappuzha, India beBeeEngineer Full time

    Backend Systems EngineerA seasoned professional with extensive experience in designing and implementing high-performance backend systems for large-scale B2C platforms is sought after. Key responsibilities include developing scalable, fault-tolerant architectures that ensure exceptional user experience.Key Skills:Backend Systems: Java, Go, Spring Boot, REST,...