Lead Databricks Engineer

1 day ago

Hyderabad, Telangana, India Milestone Technologies, Inc. Full time ₹ 30,00,000 - ₹ 50,00,000 per year

Role Overview:

We are looking for a seasoned and visionary Lead Databricks Engineerwith over 14 years of experience to lead the design and implementation of enterprise-grade data architecture, governance frameworks, and scalable data platforms. This role requires deep expertise in modern data technologies, big data ecosystems, and cloud-native solutions. The ideal candidate will be a strategic leader who can bridge business needs with technical solutions, ensuring data integrity, quality, and accessibility across the organization.

Responsibilities:

  • Lead the design and implementation of Data Architecture, Data Quality frameworks, and Data Governance initiatives.
  • Define data modeling standards and establish foundational best practices across data platforms.
  • Champion and promote data quality principles by developing and institutionalizing enterprise-wide standards and methodologies.
  • Build and oversee data governance policies, procedures, and frameworks to ensure data integrity, security, and compliance.
  • Drive enterprise-wide adoption of self-service data platforms and enablement of organizational data literacy and utilization.
  • Define and maintain critical data standards and metadata to facilitate the consistent understanding and use of data as a shared enterprise asset.
  • Create reusable templates and establish coding standards for sourcing, collecting, and transforming both streaming and batch data.
  • Design scalable data schemas, object models, and flow diagrams to effectively structure, process, and integrate data across systems.
  • Provide architectural strategy, assessments, and roadmaps to guide enterprise data management practices.
  • Offer hands-on expertise in architecting and managing Big Data platforms and Data Lake technologies (e.g., AWS S3, Hive), along with exposure to ML and Data Science tools.
  • Implement and manage industry-standard tools and frameworks such as Databricks, Delta Lake, Apache Spark, Airflow, Hive Metastore, Redshift, Kafka, Kubernetes, Docker, and CI/CD pipelines.
  • Translate complex big data and analytics requirements into high-performance, scalable data models and guide analytics engineers on implementing these models.
  • Define reusable templates and processes for data modeling, data flow design, and system integration.
  • Lead, mentor, and upskill data analytics teams by promoting best practices, modern technologies, and effective processes in data platform engineering.
  •  

Tech Skills:

  • 6+ years of hands-on experience in Data Warehouse, ETL, Data Modeling & Reporting
  • 5+ years of hands-on experience in productionizing and deploying Big Data platforms and applications.
  • Minimum of 3 years of hands-on DataBricks experience required.
  • Hands-on experience working with: Relational/SQL, distributed columnar data stores/NoSQL databases, time-series databases, Spark streaming, Kafka, Hive, Delta Parquet, Avro, and more
  • Extensive experience in understanding a variety of complex business use cases and modeling the data in the data warehouse
  • Highly skilled in SQL, Python, Spark, AWS S3, Hive Data Catalog, Parquet, Redshift, Airflow, and Tableau or similar tools
  • Proven experience in building a Custom Enterprise Data Warehouse or implementing tools like Data Catalogs, Spark, Tableau, Kubernetes, and Docker
  • Demonstrated industry leadership in the fields of Data Warehousing, Data Science, and Big Data related technologies
  • Design and Implement Machine learning models with Tensorflow ,Keras and pytorch And MLops

  • Databricks Engineer

    7 days ago

    Hyderabad, Telangana, India ValueMomentum Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Company DescriptionValueMomentum is a leading solutions provider for the global property and casualty insurance industry, backed by deep domain and technology capabilities. Our mission is to help insurers achieve sustained growth and high performance by enhancing stakeholder value and building resilient societies. Trusted by over 100 insurers globally, we...

  • Lead Databricks Engineer

    1 day ago

    Hyderabad, Telangana, India Milestone Technologies, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Company OverviewMilestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over...

  • Azure Databricks Engineer

    1 week ago

    Hyderabad, Telangana, India Tekskills Inc. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Role: Azure Databricks EngineerJob Location: Pune, MH / Chennai, TN / Hyderabad. TG / Indore, MP / Kolkata, WB / Bangalore, TNWork Mode: (WFO)Skills Required: Digital: DatabricksExperience: 6-8 Years(5+ yrs)Job Description:Should build Databricks Notebook developed in spark sql/pysparkShould be able to create unity catalog tablesShould have good hands on...

  • Senior Databricks Engineer

    1 day ago

    Hyderabad, Telangana, India Xenon7 Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About us:Where elite tech talent meets world-class opportunitiesAt Xenon7, we work with leading enterprises and innovative startups on exciting, cutting-edge projects that leverage the latest technologies across various domains of IT including Data, Web, Infrastructure, AI, and many others. Our expertise in IT solutions development and on-demand resources...

  • AWS-Databricks Engineer

    5 days ago

    Hyderabad, Telangana, India Coforge Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    We are seeking an experiencedAWS Databricks Developerto design, develop, and maintain scalable data processing and analytics solutions on theAWS and Databricksplatform. The ideal candidate will have strong expertise in building data pipelines, optimizing performance, and integrating multiple AWS services to deliver high-quality data solutions.Key...

  • Senior Databricks Engineer

    3 days ago

    Hyderabad, Telangana, India SID Information Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role: Senior Data Engineer (Python, Spark/Databricks, SQL, AWS)Experience:6–12 yearsLocation:HyderabadWork Mode:Hybrid (3 days/week in-office)Join Time:ImmediateMust-Have Technical Skills:Strong programming skills in Python, DatabricksHands-on experience with Apache Spark & Databricks for big data processing on AWS cloudProficiency with AWS services ...

  • Databricks+spark

    1 day ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryWe are seeking a highly skilled Sr. Developer with 8 to 12 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. Experience in Sales & Marketing domains is a plus. The role involves developing and optimizing data solutions to...

  • DataBricks-BigData Development – Lead Programmer Analyst

    1 week ago

    Hyderabad, Telangana, India Bilvantis Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Description: DataBricks-BigData Development – Lead Programmer Analyst (Exp: 5 Years to 7 Years)We are looking for a highly skilled and experienced DataBricks-BigData Development skilled as a Lead Programmer Analyst.Experience should have 5 Years to 7 Years into BigData, DataBricks, Python & PySpark development.Experience in data engineering with strong...

  • Databricks + Pyspark

    1 day ago

    Hyderabad, Telangana, India Cognizant Full time ₹ 1,00,00,000 - ₹ 3,00,00,000 per year

    Skills- Databricks+ PysparkExperience: 4 to 13 yearsLocation: AIA-PuneWe are looking for a highly skilled Data Engineer with expertise in PySpark and Databricks to design, build, and optimize scalable data pipelines for processing massive datasets.Key Responsibilities:Build & Optimize Pipelines: Develop high-throughput ETL workflows using PySpark on...

  • Databricks Engineer

    5 days ago

    Hyderabad, Telangana, India Anblicks Solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Role & responsibilities5+ years of experience in data engineering roles.• Proficiency in programming languages such as Scala or Python, or Java.• Expertise in distributed data processing frameworks like Apache Spark or Flink.• Strong knowledge of cloud platforms (e.g., AWS) and cloud-native data platforms like Databricks or Redshift.• Solid...