Kafka Cluster Architect

2 weeks ago

Amrāvati, Maharashtra, India beBeeMigration Full time ₹ 18,00,000 - ₹ 20,00,000
Job Description

The ideal candidate will be responsible for designing and implementing automated migration pipelines for moving enterprise Kafka workloads to open-source or cloud-native clusters. This includes scripting and provisioning migration steps, automating validation, rollback, and monitoring tasks.

In addition, the candidate will develop and integrate custom plugins for Kafka Connect as part of migration workflows. They will also build scalable CI/CD processes for Kafka resource provisioning and configuration rollouts.

Furthermore, the successful candidate will ensure automated tests cover data integrity, performance, and integration interfaces post-migration. They will also integrate with cloud orchestration (AWS MSK, Kubernetes operators) and manage infrastructure-as-code.

Key Responsibilities:
  • Design automated migration pipelines for moving enterprise Kafka workloads to open-source or cloud-native clusters
  • Script and provision migration steps, automate validation, rollback, and monitoring tasks
  • Develop and integrate custom plugins for Kafka Connect as part of migration workflows
  • Build scalable CI/CD processes for Kafka resource provisioning and configuration rollouts
  • Ensure automated tests cover data integrity, performance, and integration interfaces post-migration
  • Integrate with cloud orchestration (AWS MSK, Kubernetes operators)
Requirements:
  • CI/CD (Jenkins, GitHub Actions), IaC (Terraform, CloudFormation)
  • Apache Kafka administration and migration
  • Scripting (Python, Go, Bash)
  • Kafka Connect custom plugin API, Maven/Gradle
  • Containerization (Docker, Kubernetes)
  • Monitoring/validation: Prometheus, Grafana
  • Automation of rollback/validation steps for migrations
  • Exposure to cloud services (AWS, GCP, Azure)
  • Data ops and security automation

  • Upgrading Service Mesh, Migrating Kafka Clusters: Experienced Kafka Engineer Wanted

    2 weeks ago

    Amrāvati, Maharashtra, India beBeeKafkaEngineer Full time ₹ 1,36,80,000 - ₹ 2,16,00,000

    Kafka Engineer for BankingAre you an experienced engineer looking to take on a challenging role?We are seeking a skilled Kafka engineer to work onsite for a leading banking client. In this role, you will be responsible for upgrading Service Mesh, migrating Kafka clusters, and ensuring secure integration in a restricted environment.Upgrade Service Mesh from...

  • Data Processing Expert

    1 week ago

    Amrāvati, Maharashtra, India beBeeKafka Full time ₹ 15,00,000 - ₹ 20,00,000

    Job DescriptionOrganizations are seeking a seasoned expert in Kafka cluster administration to contribute to their data processing infrastructure.Large-scale Kafka installations, configurations, and management are essential responsibilities of this role. Upgrades, patches, and disaster recovery planning must ensure security, availability, scalability, and...

  • Cloud Security Architect

    2 weeks ago

    Amrāvati, Maharashtra, India beBeeSecurity Full time ₹ 13,07,920 - ₹ 28,11,766

    Job Title: Cloud Security ArchitectWe are seeking a skilled Cloud Security Architect to design, implement and secure cloud environments.This role involves managing infrastructure security, containerized environments, and integrating security into development activities to ensure a robust and compliant cloud security ecosystem.Main Responsibilities:Design and...

  • Data Architect

    2 weeks ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title:Data ArchitectJob DescriptionWe are seeking a highly skilled Data Architect to join our team. As a Data Architect, you will play a key role in designing and implementing our data pipeline architecture.Key ResponsibilitiesCreate and maintain optimal data pipeline architecture.Design and implement internal process improvements: automating manual...

  • Senior Data Architect

    1 week ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job SummaryWe are seeking a seasoned Data Architect to join our team. This is an exciting opportunity for a highly skilled professional with expertise in designing and implementing large-scale data systems.The ideal candidate will have a strong track record of delivering complex data solutions, with hands-on experience in distributed data processing systems,...

  • Big Data Architect

    1 week ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 21,64,800 - ₹ 25,18,800

    Job Title: Senior Data ArchitectClients rely on us to create, modernize and scale intelligent products that provide connected experiences driven by artificial intelligence.We specialize in developing cutting-edge solutions across various sectors including healthcare, technology, communications, manufacturing, and finance.In this role, you will have...

  • Chief Data Architect

    1 week ago

    Amrāvati, Maharashtra, India beBeeDataScientist Full time US$ 1,20,000 - US$ 1,50,000

    Job DescriptionWe are a global team of technology innovators and entrepreneurs with deep experience in clinical research, medical product safety, and public health surveillance.We work with regulators, major pharmaceutical companies, and academia to develop analytical solutions used to improve drug, vaccine, and device safety monitoring and public health...

  • Enterprise OpenShift Specialist

    1 week ago

    Amrāvati, Maharashtra, India beBeeOpenShift Full time ₹ 20,00,000 - ₹ 30,00,000

    Senior OpenShift/Kubernetes Architect Job SummaryAs a seasoned Senior OpenShift/Kubernetes Architect, you will be responsible for leading the design, deployment, and management of enterprise-scale OpenShift clusters across production and lab environments.Key Responsibilities:Cluster Deployment & Architecture:Design and implement RedHat OpenShift clusters...

  • Expert Data Technology Architect

    2 weeks ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking a highly skilled Big Data Engineer with deep expertise in data technologies and cloud services. The ideal candidate will bring strong hands-on experience in designing, architecting, and implementing scalable data engineering solutions while driving innovation within the team.Key Responsibilities-Design, develop, and optimize big data...

  • Chief Data and AI Platform Architect

    1 week ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Lead Architect for Data and AI PlatformsWe are seeking a highly experienced Lead Architect to spearhead the development and implementation of our data and AI platform. The ideal candidate will have a deep understanding of distributed systems, ML platform engineering, and large language model (LLM) architecture.10+ years of experience in data architecture,...