Databricks Data Operations Specialist

3 days ago

Hyderabad, Telangana, India beBeeDataOperations Full time ₹ 1,50,00,000 - ₹ 2,00,00,000
Job Description

As a Data Operations Specialist, you will play a crucial role in ensuring the reliability and efficiency of our Databricks platform. Your primary responsibility will be to oversee the monitoring and maintenance of Databricks jobs, identify and troubleshoot issues, and implement fixes to minimize downtime.

Key responsibilities include:

  • Monitoring hourly and daily Databricks jobs to prevent failures and implementing fixes to minimize downtime.
  • Identifying, logging, and tracking defects/bugs through ticketing systems, ensuring timely resolution and maintaining system stability.
  • Managing Databricks access via Azure AD groups with Admin, Edit, and Read permissions for efficient collaboration.
  • Providing production support for Databricks environments, including cluster operations, job failures, and notebook troubleshooting.
  • Collaborating with data engineers and platform teams to resolve platform-related incidents and performance bottlenecks.
  • Proactively monitoring system health, resource utilization, and performance metrics to maintain optimal performance.
  • Implementing and enforcing archival/retention policies for Databricks storage to optimize costs and performance.
  • Supporting CI/CD pipelines (Jenkins, Azure Automation) and automating repetitive operational tasks for enhanced efficiency.
  • Maintaining technical documentation, SOPs, and runbooks for Databricks operations, ensuring seamless knowledge transfer.
  • Ensuring security compliance with RBAC, MFA, and encryption best practices to safeguard sensitive information.
Required Skills and Qualifications

To be successful in this role, you will need:

  • 3+ years of hands-on experience with Databricks and Apache Spark in production environments.
  • Strong knowledge of Azure (AWS/GCP acceptable) and cloud-native services, enabling efficient integration.
  • Experience in SRE or production support environments with SLAs and ticketing systems (ServiceNow, Jira).
  • Proficiency in Python or Scala for data processing and automation, streamlining workflows.
  • Familiarity with Power BI/Tableau for building monitoring and cost dashboards, enhancing decision-making capabilities.
  • Knowledge of CI/CD tools, version control (Git), and scripting languages (PowerShell, Bash).
  • Understanding of cloud cost optimization and usage tracking, ensuring responsible resource allocation.
  • Excellent problem-solving skills, communication, and cross-team collaboration abilities, driving project success.
Benefits

This is an excellent opportunity to join a dynamic team and take your career to the next level. We offer a competitive salary, comprehensive benefits package, and opportunities for professional growth and development.

Others

If you are passionate about data operations and want to work with a cutting-edge technology, we encourage you to apply for this exciting opportunity.

  • Cloud Databricks Operations Specialist

    2 weeks ago

    Hyderabad, Telangana, India beBeeDatabricks Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OpportunityOur organization is seeking a skilled Cloud Databricks Administrator to join our Engineering team. This role will play a key part in operational support, defect management, and proactive issue resolution to keep data pipelines running smoothly.Monitor hourly and daily Databricks jobs, investigate failures, and implement fixes to minimize...

  • Cloud Databricks Operations Specialist

    3 days ago

    Hyderabad, Telangana, India beBeeDatabricks Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Summary:We are seeking a highly skilled Databricks professional to join our team. As a Cloud Databricks Administrator, you will be responsible for ensuring the optimal performance and reliability of our Databricks environments.In this role, you will work closely with data engineers and platform teams to resolve platform-related incidents and performance...

  • Databricks Infrastructure Specialist

    2 days ago

    Hyderabad, Telangana, India beBeeinfrastructure Full time ₹ 25,00,000 - ₹ 40,00,000

    Job Title: Databricks Infrastructure SpecialistWe are seeking a highly skilled Databricks Infrastructure Specialist to assess and improve our Databricks implementation on Azure, integrating it with AzureDevOps.This role involves conducting a thorough audit of the current Databricks implementation on Azure, identifying areas for improvement.The successful...

  • Azure Data Engineer – Databricks

    6 days ago

    Hyderabad, Telangana, India Innovatz Global Full time US$ 90,000 - US$ 1,20,000 per year

    Company DescriptionInnovatz is a Global Management Consulting, Technology Services, and Business Process Outsourcing Company headquartered in Kuala Lumpur, Malaysia. We have a strong presence in major markets including the Americas, China, India, Australia, Korea, Hong Kong, Singapore, Thailand, Philippines, Indonesia, and Vietnam. Boasting a skilled...

  • Lead Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Lead Data Engineer - Databricks SpecialistJob Summary:We are seeking a seasoned data engineering expert with expertise in Databricks, PySpark, and SQL to spearhead our data engineering initiatives.Main Responsibilities:Design and develop end-to-end data pipelines using Azure Data Factory for seamless data ingestion and transformation.Collaborate closely with...

  • Databricks

    3 days ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 1,04,000 - ₹ 13,08,780 per year

    Job SummaryWe are seeking a highly skilled Sr. Developer with 8 to 12 years of experience to join our team. The ideal candidate will have extensive experience in Spark in Scala Delta Sharing Databricks Unity Catalog Admin Databricks CLI Delta Live Pipelines Structured Streaming Risk Management Apache Airflow Amazon S3 Amazon Redshift Python Databricks SQL...

  • Databricks Developer

    4 weeks ago

    Hyderabad, Telangana, India Wipro Full time

    xefxbfxbd Responsibilities will include designing implementing and maintaining the Databricks platform and providing operational support Operational support responsibilities include platform set-up and configuration workspace administration resource monitoring providing technical support to data engineering Data Science ML and Application...

  • Databricks Administrator

    1 week ago

    Hyderabad, Telangana, India EverestDX Inc Full time

    We are seeking a Cloud Databricks Administrator to join our Cloud Engineering team. In this role, you will monitor and maintain Databricks jobs, troubleshoot issues, ensure platform reliability, and manage access controls. You will play a key part in operational support, defect management, and proactive issue resolution to keep our data pipelines running...

  • Databricks Administrator

    5 days ago

    Hyderabad, Telangana, India EverestDX Inc Full time

    We are seeking a Cloud Databricks Administrator to join our Cloud Engineering team. In this role, you will monitor and maintain Databricks jobs, troubleshoot issues, ensure platform reliability, and manage access controls. You will play a key part in operational support, defect management, and proactive issue resolution to keep our data pipelines running...

  • Azure Databricks Data Engineer

    4 days ago

    Hyderabad, Telangana, India Innova ESI Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Role: Azure Databricks Data EngineerExperience: 5+ YearsLocation: Hyderabad | BangaloreNotice Period: Immediate Joiners OnlyJob Description:Minimum of 4 yrs of experience on Azure cloud environment.Experience on ADLS, Azure Databricks, Azure SQL DB, Cosmos DB, Azure Data Factory and Datawarehouse.Strong working experience in Implementation of Azure cloud...