Data Modernization Lead- Databricks Expert

About the job

Job description

Job Title: Senior Data Modernization Lead– Databricks Expert (Contract) 

Location: Remote

Experience Level: 15+ Years

Role Overview

We are looking for an experienced Data Modernization Lead to spearhead our team’s transition to modern data engineering practices . This role will focus on migrating data assets and workflows from a Microsoft on-premise ecosystem to Databricks, along with modernizing development practices. The successful candidate will play a key role in migrating from Team Foundation Server (TFS) to Bitbucket for version control, establishing a Data Development Lifecycle (DDLC), and leading the transformation with Databricks expertise. This role requires a hands-on approach, strong leadership abilities , and a passion for coaching and mentoring through workshops and process design.

Key Responsibilities

Lead the strategy and execution for migrating data systems from Microsoft on-premise platforms to Databricks.

Evaluate and implement best practices for data pipelines, governance, and quality assurance within the Databricks environment.

Guide the team in transitioning from Team Foundation Server (TFS) to Bitbucket , ensuring effective branching, merging, and release management practices.

Establish a scalable Data Development Lifecycle (DDLC), including strategies for environments, testing frameworks, CI/CD for data pipelines, and deployment best practices.

Facilitate workshops and training sessions to develop Databricks skills and modern data engineering competencies within the team.

Mentor team members on version c ontrol workflows (Git) and data DevOps practices.

Collaborate with data architects, business stakeholders , and IT infrastructure teams to ensure alignment of modernization efforts with organizational goals.

Keep abreast of new data engineering technologies and propose strategies to improve team productivity and system reliability.

Required Skills and Experience

Proven experience leading data platform migrations, preferably from Microsoft on-premise systems to cloud-based platforms such as Databricks.

Expertise in Databricks, including Spark, Delta Lake, Python/Scala/SQL, MLFlow, and Lakehouse architecture .

Strong knowledge of version control migration and practices, particularly moving from TFS to Git/Bitbucket.

Experience in designing and implementing data DevOps processes, including building CI/CD pipelines for data solutions.

Familiarity with data governance, quality frameworks, and lifecycle management.

Exceptional facilitation and coaching skills, with a proven ability to lead practical, engaging workshops that drive results.

Strong communication and stakeholder management skills to align technical strategy with business objectives.

Preferred Qualifications

Experience in leading large-scale enterprise data modernization initiatives.

Hands-on experience with Azure Data Services, AWS, or GCP alongside Databricks.

Familiarity with Agile methodologies within data and analytics teams.

Leadership experience in defining team processes and standards for modern data engineering practices

Why Work With Us?

Flexible, remote work with a Full time commitment.

Opportunity to drive Databricks architecture and practices across data initiatives.

Collaborate with a smart, agile, and inclusive team.

Contribute to impactful enterprise-level data solutions.

How to Apply:

Send your updated resume to ( ) with the subject line:

“Full time Databricks Expert ”

or connect with us via LinkedIn’s Easy Apply option.

Let’s build intelligent data solutions together