Kafka Cluster Architect

4 days ago

Agra, Uttar Pradesh, India beBeeClusterEngineer Full time ₹ 15,00,000 - ₹ 20,00,000
Job Description

We are seeking a seasoned Kafka Cluster Engineer to join our team.

The successful candidate will be responsible for designing, deploying, and managing production-grade Kafka clusters, ensuring high availability and fault tolerance.

Key responsibilities include:

  • Deploying and upgrading Kafka clusters using industry-leading tools such as Ansible and Chef.
  • Automating the installation and configuration of Kafka clusters.
  • Troubleshooting critical production issues, optimizing Kafka performance and minimizing downtime.
  • Configuring MirrorMaker for data replication between two data centers, ensuring effective disaster recovery.
  • Deploying and managing Provectus Kafka UI on a Kubernetes cluster for real-time monitoring and management.
  • Integrating Schema Registry and deploying multiple Kafka Connectors to facilitate seamless data flow.
  • Onboarding new users by managing ACL permissions, creating topics, and implementing end-to-end security with SSL/TLS.
  • Designing and managing Kafka infrastructure to enhance message processing efficiency.
  • Conducting performance testing to optimize partitioning, replication, and retention policies.
  • Integrating Kafka with ELK Stack and Solr for scalable data storage and analysis.
Required Skills and Qualifications

To be successful in this role, you will need:

  • 3+ years of experience in designing and deploying large-scale Kafka clusters.
  • Proficiency in programming languages such as Java, Python, or Scala.
  • Knowledge of containerization platforms like Docker and Kubernetes.
  • Experience with automation tools like Ansible and Terraform.
  • Strong understanding of messaging systems, data pipelines, and data architecture.
Benefits

This role offers a competitive salary and benefits package, including:

  • Opportunities for career growth and professional development.
  • A collaborative and dynamic work environment.
  • Access to cutting-edge technology and tools.
  • Flexible working hours and remote work options.
Location

Our office is located in Bangalore, India.

Keyword

ClusterEngineer

  • High-Performance Computing Architect

    4 days ago

    Agra, Uttar Pradesh, India beBeeInfrastructure Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Role:AI Infrastructure Architect for HPC and Kubernetes Projects This is an infrastructure-focused position where you will design, build, and deploy platforms to support AI workloads. You will have opportunities to architect HPC clusters, integrate Kubernetes with AI/ML applications, and deploy NVIDIA AI Enterprise and related frameworks. The ideal...

  • Lead Database Strategist

    3 days ago

    Agra, Uttar Pradesh, India beBeeDatabase Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Database ProfessionalWe are seeking an experienced Senior Database Administrator to oversee our database systems.Analyze and optimize existing databases to ensure high performance and efficiency.Develop and maintain robust database security protocols to safeguard sensitive data, adhering to industry standards.Manage Cloudera Data Platform (CDP)...

  • Senior Data Architect

    5 days ago

    Agra, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Large-Scale Data ArchitectKey Responsibilities:Design and develop scalable data architectures for real-time systems.Develop and maintain complex data processing pipelines using Kafka, Flink, and Spark.Collaborate with cross-functional teams to ensure high-quality data delivery.Requirements:Expertise in large-scale data engineering, including experience with...

  • SQL Database Architect

    3 days ago

    Agra, Uttar Pradesh, India beBeeDatabase Full time ₹ 18,00,000 - ₹ 21,50,000

    Job Summary:We are seeking an experienced SQL Database Administrator to support our team. The ideal candidate will have strong technical skills and experience in managing MS SQL databases, including failover clustering, transaction replication, link servers, and hardening.Key Responsibilities:Monitor database logs and event files indicating potential...

  • Advanced Machine Learning Architect

    3 days ago

    Agra, Uttar Pradesh, India beBeeMachine Full time ₹ 1,80,00,000 - ₹ 2,10,00,000

    Senior AI/ML Engineer RoleWe are seeking a Senior AI/ML Engineer to lead the development of innovative data solutions using cutting-edge technologies.This is an exciting opportunity for a seasoned professional to drive projects from conception to deployment, leveraging expertise in Python, SQL, and machine learning algorithms.Design, develop, and optimize...

  • Databricks admin

    1 week ago

    Agra, Uttar Pradesh, India Numeric Technologies Full time

    Remote, Rotational shiftNeed immediate joiners onlyRequired Skills & Qualifications:7+ years of experience as a Databricks Administrator or in a similar role.Strong knowledge of Azure/AWS Databricks and cloud computing platforms.Hands-on experience with Databricks clusters, notebooks, libraries, and job scheduling.Expertise in Spark optimization, data...

  • Infrastructure Specialist

    3 days ago

    Agra, Uttar Pradesh, India beBeeMLOps Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title:MLOps ArchitectDescription:We are seeking a seasoned Platform Architect to spearhead the design and implementation of cutting-edge build and test environments for critical GenAI workloads.In this role, you will be instrumental in designing, developing, and maintaining production-scale platforms within the GenAI or ML domain. You will partner with...

  • Chief Data Architect

    1 day ago

    Agra, Uttar Pradesh, India beBeeDataModeling Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking a skilled professional to design, develop, and optimize data pipelines using cloud-based technologies.Role: Data ModelerLocation: Remote or Onsite Opportunities AvailableExperience: 6+ YearsJob Type: Full Time EmploymentKey Responsibilities:Design scalable data architectures using Azure Data Factory, Databricks, and Kafka.Implement...

  • Chief Data Architect Leader

    1 day ago

    Agra, Uttar Pradesh, India beBeeDataStrategy Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Strategy DirectorJob Summary:Lead the development and implementation of a comprehensive data strategy, driving business growth through informed decision-making.Main Responsibilities:Develop and lead large-scale enterprise data platforms (Snowflake, Databricks, AWS, Kafka), ensuring scalability and reliability.Create AI/ML models for...

  • High-Impact AI Solutions Architect

    1 week ago

    Agra, Uttar Pradesh, India beBeeArchitect Full time ₹ 17,00,000 - ₹ 37,00,000

    Job Summary:We are seeking a seasoned AI Solutions Architect to spearhead the design and deployment of cutting-edge Generative AI solutions in a dynamic life sciences environment.The ideal candidate will have a strong background in AI/ML solution architecture, with extensive experience working directly with Large Language Models (LLMs) and Generative AI. Key...