Databricks Cluster Optimization Specialist

6 days ago


Kolkata Hyderabad Secunderabad Telangana, India beBeeData Full time
Senior Databricks Administrator

This role involves providing exceptional support to our analyst community by managing and optimizing Databricks clusters, ensuring high availability and performance.

  • Key Responsibilities:
  • Databricks Infrastructure Management: Manage and optimize Databricks clusters at scale.
  • Deployment & Configuration: Oversee deployments within Databricks and work with Databricks System tables for monitoring and management.
  • Troubleshooting & Support: Provide strong troubleshooting capabilities for issues within Databricks notebooks and address Databricks-related queries from the analyst community.
  • Cloud & Data Platform Fundamentals: Apply good knowledge of AWS services and strong fundamentals of Data Warehousing (DW) / Data Lakehouse (DH) concepts.
  • Coding & Automation: Utilize Spark/Python for scripting and automation within the Databricks environment.
  • Containerization & Orchestration: Work with Docker and Kubernetes.
  • AWS Services Integration: Leverage knowledge of relevant AWS services such as S3, Lambda, and EMR.

Required Skills and Qualifications:

  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Proven experience as a Databricks Admin or in a similar role.
  • Typically seeking candidates with 6+ years of relevant experience.
  • Certified Databricks Engineer is required.

What You'll Gain:

  • Opportunity to work with cutting-edge technology.
  • Chance to develop your skills in cloud and data platform fundamentals.
  • Experience in managing and optimizing Databricks clusters.

Why This Role Matters:

  • This role plays a crucial part in supporting our analyst community.
  • You'll be responsible for ensuring seamless deployments and providing critical support.

How to Thrive in This Role:

  • Strong expertise in Databricks administration.
  • Good knowledge of AWS, Databricks, and Unity Catalog.
  • Strong fundamentals of Data Warehousing (DW) / Data Lakehouse (DH) concepts.
  • Strong troubleshooting abilities within the Databricks environment.

Our Ideal Candidate:

  • Excellent problem-solving skills.
  • Ability to work effectively in a team environment.
  • Strong communication and interpersonal skills.


  • Hyderabad / Secunderabad, Telangana, Kolkata, India beBeeData Full time ₹ 9,00,000 - ₹ 12,00,000

    We're seeking a highly skilled Databricks Administrator to join our team. This pivotal role requires expertise in Databricks administration, AWS infrastructure management, and data warehousing concepts. As a key member of the team, you'll be responsible for managing Databricks clusters, ensuring seamless deployments, providing critical support to our...


  • Hyderabad, Telangana, India ShyftLabs Full time

    **Position Overview**: ShyftLabs is a growing data product company that was founded in early 2020 and works primarily with Fortune 500 companies. We deliver digital solutions built to accelerate business growth across various industries by focusing on creating value through innovation. **Responsibilities**: - Architect, design, and optimize big data and...

  • Databricks Admin

    3 weeks ago


    Hyderabad, Bengaluru, India Zentek Infosoft Full time

    Job DescriptionResponsibilities:- Workspace Management: Create and manage Databricks workspaces, ensuring proper configuration and access control.- User & Identity Management: Administer user roles, permissions, and authentication mechanisms.- Cluster Administration: Configure, monitor, and optimize Databricks clusters for efficient resource utilization.-...

  • Databricks Developer

    3 weeks ago


    Hyderabad, Telangana, India Wipro Full time

    xefxbfxbd Responsibilities will include designing implementing and maintaining the Databricks platform and providing operational support Operational support responsibilities include platform set-up and configuration workspace administration resource monitoring providing technical support to data engineering Data Science ML and Application...

  • Cloud Data Engineer

    4 days ago


    Hyderabad / Secunderabad, Telangana, Bengaluru / Bangalore, India beBeeData Full time ₹ 8,00,000 - ₹ 20,00,000

    Databricks ExpertThis role focuses on leveraging Databricks as a cloud-based data and AI platform to drive business growth and innovation. Key responsibilities include:Workspace Optimization: Streamline Databricks workspace management, ensuring seamless collaboration and efficient resource allocation.User Identity Management: Develop and implement robust...


  • Hyderabad, Telangana, India beBeeAdministrative Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Description:We are seeking a highly skilled Databricks Platform Admin to join our team. As a key member of our organization, you will be responsible for managing workspaces, AWS accounts, and high-level usage monitoring.You will have a deep understanding of Unity catalog and experience in optimizing usage for performance and cost.Experience with building...


  • Hyderabad, Telangana, India EverestDX Inc Full time

    We are seeking a Cloud Databricks Administrator to join our Cloud Engineering team. In this role, you will monitor and maintain Databricks jobs, troubleshoot issues, ensure platform reliability, and manage access controls. You will play a key part in operational support, defect management, and proactive issue resolution to keep our data pipelines running...


  • Hyderabad, Telangana, India beBeeDatabricks Full time ₹ 20,00,000 - ₹ 25,00,000

    Job OpportunityWe are seeking a skilled Databricks professional to join our team.Azure/AWS Databricks expertiseCloud computing knowledgeStrong problem-solving skillsKey Responsibilities:Manage and maintain Databricks clustersDevelop and implement data processing solutionsOptimize Spark performanceRequirements:7+ years of experience in Databricks...


  • Hyderabad, Telangana, India beBeeDatabricks Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OpportunityOur organization is seeking a skilled Cloud Databricks Administrator to join our Engineering team. This role will play a key part in operational support, defect management, and proactive issue resolution to keep data pipelines running smoothly.Monitor hourly and daily Databricks jobs, investigate failures, and implement fixes to minimize...


  • Kolkata, West Bengal, India beBeeElasticsearch Full time ₹ 9,00,000 - ₹ 12,00,000

    We seek an accomplished Senior Elasticsearch Cluster Specialist to ensure the performance, stability, and scalability of large-scale cluster deployments.The ideal candidate will play a vital role in diagnosing complex system issues, optimizing cluster performance, and ensuring the reliability of Elasticsearch clusters in production environments.Key...