DataZymes - Databricks Platform Engineer - ETL

7 days ago

Bengaluru, Karnataka, India DATAZYMES ANALYTICS PRIVATE LIMITED Full time

Role Overview :

Seasoned Databricks Senior Platform Engineer who can implement scalable, end-to-end data solutions on Databricks over AWS. This role is pivotal in building the foundational infrastructure for data products, ensuring data quality, and enabling machine learning capabilities

Key Responsibilities :

- Design and implement end-to-end ETL pipelines in Databricks, sourcing data from Amazon S3 and Amazon RDS and other sources.

- Design and deploy Databricks workspaces and clusters optimized for performance, scalability, and cost-efficiency.

- Set up data quality frameworks and anomaly detection mechanisms to ensure reliability and trust in data products.

- Enable machine learning capabilities within Databricks by configuring necessary services, libraries, and integrations.

- Collaborate with data scientists, engineers, and DevOps teams to ensure seamless platform operations.

- Automate deployment and monitoring using CI/CD pipelines, GitHub

- Document architecture, workflows, and operational procedures for platform governance.

Requirements :

Core Platform & Cloud :

- 5+ years of experience in platform engineering roles

- Strong expertise in Databricks on AWS

- Proficient with Amazon S3, Amazon RDS, and IAM roles/policies

- Experience with Delta Lake, Structured Streaming, and Databricks Workflows

ETL & Data Quality :

- Hands-on experience building ETL pipelines using Spark

- Familiarity with data validation, profiling, and anomaly detection frameworks

- Knowledge of tools like Great Expectations or custom validation scripts

ML Platform Enablement :

- Experience setting up ML infrastructure including :

- MLflow for experiment tracking and model registry

- Feature Store for reusable feature pipelines

- Databricks Runtime for ML

- Integration with AWS SageMaker or custom ML libraries

- Understanding of model lifecycle management, though model development is out of scope

DevOps & Automation :

- Proficiency with GitHub, CI/CD pipelines, and Terraform or CloudFormation

Preferred Qualifications :

- Experience working with healthcare data

- Exposure to data mesh or data product architecture principles

(ref:hirist.tech)

  • Databricks Engineer

    3 weeks ago

    Bengaluru, Karnataka, India DataZymes Full time

    ABOUT US:  Founded in 2016, DataZymes is a next-generation analytics and data science company driving technology and digital-led innovation for our clients, thus helping them get more value from their data and analytics investments. Our platforms are built on best-of-breed technologies, thus protecting current investments while providing clients more bang...

  • Databricks Engineer

    3 weeks ago

    Bengaluru, Karnataka, India DataZymes Full time

    ABOUT US:  Founded in 2016, DataZymes is a next-generation analytics and data science company driving technology and digital-led innovation for our clients, thus helping them get more value from their data and analytics investments. Our platforms are built on best-of-breed technologies, thus protecting current investments while providing clients more bang...

  • Databricks Engineer

    6 days ago

    Bengaluru, Karnataka, India DataZymes Full time US$ 1,20,000 - US$ 2,00,000 per year

    ABOUT US:Founded in 2016, DataZymes is a next-generation analytics and data science company driving technology and digital-led innovation for our clients, thus helping them get more value from their data and analytics investments. Our platforms are built on best-of-breed technologies, thus protecting current investments while providing clients more bang for...

  • DataZymes - Databricks Engineer

    7 days ago

    Bengaluru, Karnataka, India DATAZYMES ANALYTICS PRIVATE LIMITED Full time

    We are seeking a skilled and motivated Databricks Engineer to join our dynamic data team. The ideal candidate will have 3-5 years of robust experience in data engineering, with a strong focus on the Databricks ecosystem. You will be responsible for designing, developing, and maintaining scalable and reliable data pipelines using PySpark. A key part of this...

  • AWS Engineer

    3 weeks ago

    Bengaluru, Karnataka, India DataZymes Full time

    ABOUT US:  Founded in 2016, DataZymes is a next-generation analytics and data science company driving technology and digital-led innovation for our clients, thus helping them get more value from their data and analytics investments. Our platforms are built on best-of-breed technologies, thus protecting current investments while providing clients more bang...

  • AWS Engineer

    3 weeks ago

    Bengaluru, Karnataka, India DataZymes Full time

    ABOUT US:  Founded in 2016, DataZymes is a next-generation analytics and data science company driving technology and digital-led innovation for our clients, thus helping them get more value from their data and analytics investments. Our platforms are built on best-of-breed technologies, thus protecting current investments while providing clients more bang...

  • AWS Engineer

    6 days ago

    Bengaluru, Karnataka, India DataZymes Full time US$ 90,000 - US$ 1,20,000 per year

    ABOUT US:Founded in 2016, DataZymes is a next-generation analytics and data science company driving technology and digital-led innovation for our clients, thus helping them get more value from their data and analytics investments. Our platforms are built on best-of-breed technologies, thus protecting current investments while providing clients more bang for...

  • Talent Acquisition Internship in Bangalore

    6 days ago

    Bengaluru, Karnataka, India DataZymes Full time ₹ 3,00,000 - ₹ 5,00,000 per year

    Selected Intern's Day-to-day Responsibilities IncludeAssist in sourcing candidates through job portals, LinkedIn, and other platforms.Screen resumes and schedule interviews in coordination with hiring managers.Support end-to-end activities including candidate follow-ups and feedback.Maintain and update ATS - Zoho RecruitAssist in maintaining HR records and...

  • Databricks ETL Specialist

    13 hours ago

    Bengaluru, Karnataka, India beBeeDeveloper Full time ₹ 12,00,000 - ₹ 24,00,000

    ETL DeveloperAbout this role:We are looking for a highly skilled ETL developer with experience in designing, developing and maintaining scalable data pipelines on the Azure Databricks platform.Key Responsibilities:Design and develop efficient ETL pipelines using Azure Databricks (Spark) to transform and load large-scale data sets into our data...

  • Scale Solutions Engineer

    6 days ago

    Bengaluru, Karnataka, India Databricks Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    CSQ326R201MissionAt Databricks, we are on a mission to empower our customers to solve the world's toughest data problems by utilising the Data Intelligence platform. As a Scale Solution Engineer, you will play a critical role in advising Customers in their onboarding journey. You will directly work with customers to help them onboard and deploy Databricks in...