Databricks Lead Engineer

1 day ago

Bengaluru Chennai Hyderabad, India Rhohyve Ai Labs Full time US$ 1,25,000 - US$ 1,75,000 per year

Required Skills & Experience

  • 6-9 years of experience in Consulting, Data, and Analytics, with a strong background in Databricks solutions.
  • Hands-on experience in designing and implementing big data solutions, including data pipelines for large and complex datasets.
  • Expertise in Python, PySpark, and orchestration frameworks such as Airflow, Oozie, Luigi, etc.
  • Strong understanding of Databricks workspace and its components (workspace, compute, clusters, jobs, Unity Catalog, UC permissions).
  • Ability to configure, schedule, and manage Databricks clusters and jobs, including performance optimization and monitoring.
  • Experience with data modeling, SQL queries, joins, stored procedures, relational schemas, and schema design.
  • Proficiency in working with structured, semi-structured, and unstructured data (CSV, Parquet, Delta Lake, Delta Tables, Delta Live Tables).
  • Strong knowledge of Spark architecture, Spark-Streaming, and analyzing jobs using Spark UI for monitoring and troubleshooting.
  • Knowledge of using REST API endpoints for data consumption and integration with external sources.
  • Proven experience in ETL / Data Warehouse transformation processes and building streaming/real-time data processing solutions.
  • Experience in cloud-based data platforms such as AWS (preferred), GCP, or Azure, including managing data sources like AWS S3.
  • Ability to design and implement end-to-end data ingestion pipelines, ensuring data quality and consistency.
  • Experience with CI/CD pipelines configuration in Databricks.
  • Strong logical structuring, problem-solving, verbal, written, and presentation skills.
  • Ability to work as an individual contributor as well as lead a team in an Agile setup.
  • Capable of delivering and presenting Proof of Concepts (POCs) to stakeholders.

Optional / Good to Have

  • Experience in Databricks Administration (user access, Spark tuning, data frame manipulation, etc.).
  • Exposure to real-time data movement solutions with security and encryption protocols.
  • Experience in Application Development or Data Warehousing across enterprise technologies.
  • Life Sciences industry domain knowledge.

  • Director of Engineering

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    (P-1384)About The TeamAt Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs.Ingesting data into the Lakehouse is a strategic area of investment for Databricks and a key enabler for Data and AI...

  • Director of Engineering

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    (P-1384)About the TeamAt Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data...

  • Senior Staff Software Engineer

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    (P-1174)We are hiring aSenior Staff Software Engineerfor Databricks' Engineering team reporting to an Engineering Leader. You will be part of the Databricks engineering organization, working with teams that develop Databricks products and features for thousands of enterprises worldwide.Key CharacteristicsAs an executive engineering individual contributor at...

  • Director of Engineering

    1 week ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    (P-1384)About the TeamAt Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data...

  • Lead Databricks Engineer

    6 days ago

    Bengaluru, Karnataka, India DXC Technology Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Description:Lead Databricks Engineer / ArchitectJob Location: Hyderabad / Bangalore / Chennai / Kolkata / Noida/ Gurgaon / Pune / Indore / MumbaiRole OverviewWe are seeking an experienced Lead Data Engineer / Architect with over 10 years of industry experience (5+ years in Databricks and modern data platforms) to design, implement, and lead...

  • Senior Software Engineer

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    P-1385Companies are investing billions of dollars into developing and deploying AI and the data platforms that enable it. But do they know what is happening?At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical...

  • Senior Software Engineer

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    P-1346At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Staff Software Engineer

    2 days ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,50,000 - US$ 2,00,000 per year

    P-1346At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to...

  • Lead Databricks Engineer

    1 week ago

    Bengaluru, Karnataka, India DXC Technology Full time US$ 1,50,000 - US$ 2,00,000 per year

    Job Description:Lead Databricks Engineer / Architect Job Location: Hyderabad / Bangalore / Chennai / Kolkata / Noida/ Gurgaon / Pune / Indore / Mumbai Role OverviewWe are seeking an experienced Lead Data Engineer / Architect with over 10 years of industry experience (5+ years in Databricks and modern data platforms) to design, implement, and lead...

  • Sr. Solutions Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Databricks Full time US$ 1,20,000 - US$ 2,00,000 per year

    FEQ426R153As a Sr. Solutions Engineer (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...