Lead Consultant- Databricks Developer ??

1 week ago

Mumbai, India Genpact Full time

Job Description Inviting applications for the role of Lead Consultant- Databricks Developer In this role, the Databricks Developer is responsible for solving the real world cutting edge problem to meet both functional and non-functional requirements. Responsibilities - Design and develop scalable data pipelines using Databricks (PySpark/SQL/Delta Live Tables). - Implement ETL/ELT frameworks leveraging the Lakehouse (Bronze, Silver, Gold) architecture. - Implement and manage data governance using Unity Catalog to ensure secure access, compliance, and centralized management of data, users, and permissions across the Databricks Lakehouse. - Optimize data models, queries, and workflows for scalability, cost, and performance. - Act as Databricks SME and provide guidance on best practices, governance, and security. - Mentor junior developers, review code, and enforce coding standards. - Integrate Databricks with cloud storage, APIs, warehouses, and BI tools. - Implement orchestration using ADF, Airflow, Step Functions, or Databricks Workflows. - Build reusable accelerators and frameworks for ingestion, transformation, and monitoring. - Enable data quality, validation, and reconciliation (using tools like Great Expectations or custom). - Set up monitoring, logging, and alerting dashboards for pipeline health. - Collaborate with business stakeholders, architects, and analysts to deliver solutions. - Support migration of legacy pipelines into Databricks Lakehouse. - Contribute to architectural decisions, POCs, and innovation initiatives. Qualifications we seek in you Minimum qualifications - Bachelor's degree (CS, CE, CIS, IS, MIS, Engineering) or equivalent work experience. - Experience in data engineering with at least Databricks experience. - End-to-end implementation of at least 2 Databricks projects(migration/integration). - Strong background in batch and streaming data pipelines. - Proficiency in Python (preferred) or Scala for Spark-based development. - Expertise in SQL & Spark-SQL, data structures, and algorithms. - Deep knowledge of Databricks components: Delta Lake, DLT, dbConnect, REST API 2.0, Workflows orchestration. - Strong in performance optimization for pipelines (efficiency, scalability, cost reduction). - Hands-on experience with Apache Spark, Hive, and Lakehouse architecture. - Cloud expertise (Azure/AWS) includes storage (ADLS/S3), messaging (ASB/SQS), compute (ADF/Lambda), and databases (CosmosDB/DynamoDB/Cloud SQL). - Experience writing unit tests and integration tests for data pipelines. - Ability to work with architects and lead engineers to design solutions meeting functional & non-functional requirements. - Team player with experience in teams of 5+ engineers. - Strong communication and client-facing skills. - Keeps updated with emerging technologies and industry trends. - Strong analytical and problem-solving abilities. - Positive attitude towards continuous learning and upskilling - Good to have Databricks SQL Endpoint understanding. - Good to have understanding on LakeflowConnect, Lakeflow Declarative Pipelines - Good To have CI/CD experience to build the pipeline for Databricks jobs. - Good to have if worked on migration project to build Unified data platform. - Good to have knowledge of DBT. - Good to have knowledge of docker and Kubernetes. - Certification on Databricks Associate level. - Any one Cloud Certification (AWS/Azure) Practitioner or Associate Level

  • Principal Solutions Architect

    7 days ago

    Mumbai, India Databricks Full time

    As a Solutions Architect (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor** internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve tangible...

  • Solutions Data Architect

    7 days ago

    Mumbai, India Databricks Full time

    As a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...

  • Principal Solutions Architect

    7 days ago

    Mumbai, India Databricks Full time

    As a Solutions Architect (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor** internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve tangible...

  • Solutions Architect

    7 days ago

    Mumbai, India Databricks Full time

    As a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...

  • Solutions Architect

    2 weeks ago

    Mumbai, India Databricks Full time

    **What we look for**: As a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our...

  • Delivery Solutions Architect

    7 days ago

    Mumbai, India Databricks Full time

    At Databricks we are on a mission to empower our customers to solve the world's toughest data problems by utilizing the Lakehouse platform. As a Delivery Solutions Architect (DSA), you will play a critical role during this journey. The DSA works across a small number of our **largest or highest potential key accounts**, collaborating across Databricks teams...

  • Resident Solutions Architect

    7 days ago

    Mumbai, Maharashtra, India Databricks Full time

    CSQ426R257 We are seeking an experienced Resident Solution Architect (RSA) to join our Professional Services team and work directly with strategic customers on their data and AI transformation initiatives using the Databricks platform. As an RSA, you will serve as a trusted technical advisor and hands-on expert, guiding customers to solve complex big data...

  • Field Engineering Manager

    7 days ago

    Mumbai, Maharashtra, India Databricks Full time ₹ 20,00,000 - ₹ 60,00,000 per year

    Manager, Field Engineering FEQ326R99 As a Manager, Field Engineering (Data & AI) (Strategic Accounts, Data & AI) will lead a team of Solutions Architects (Data & AI) focusing on large enterprise & strategic customers in India. You will help lead our Global Accounts team in India expansion, you will build and lead a technical pre-sales team. You will...

  • Delivery Solutions Architect

    4 days ago

    Mumbai, Maharashtra, India Databricks Full time

    FEQ426R137 At Databricks, we are on a mission to empower our customers to solve the world's toughest data problems by utilizing the Databricks Data Intelligence Platform. As a Delivery Solutions Architect (DSA), you will play an important role during this journey. You will collaborate with our sales and field engineering teams to accelerate the adoption and...

  • Enterprise Account Executive

    6 days ago

    Mumbai, India Databricks Full time

    As we continue to increase our presence in the Unified Data Analytics and AI world, we're looking for a creative, motivated, and execution-oriented Enterprise Account Executive to sell to the rapidly growing Enterprise Segment within India to maximize the huge market opportunity that exists for Databricks today. Reporting to the Senior Director, Enterprise...