Apache Kafka Architect

5 days ago

Kanpur, Uttar Pradesh, India beBeeKafka Full time ₹ 9,00,000 - ₹ 12,00,000
Job Description

The primary responsibility of this role is to manage and maintain scalable Kafka clusters. This includes deploying, upgrading, and configuring production-grade environments that ensure high availability and fault tolerance.

To achieve this, you will utilize various tools such as Ansible and Chef to automate the installation and configuration of Kafka clusters.

Additionally, you will troubleshoot critical production issues, optimize Kafka performance, and minimize downtime. MirrorMaker for data replication between two data centers will also be configured to ensure effective disaster recovery.

Furthermore, you will deploy and manage Provectus Kafka UI on a Kubernetes cluster for real-time monitoring and management. Schema Registry integration and deployment of multiple Kafka Connectors will facilitate seamless data flow.

You will also onboard new users by managing ACL permissions, creating topics, and implementing end-to-end security with SSL/TLS. Designing and managing Kafka infrastructure to enhance message processing efficiency is another key aspect of this role.

Performance testing to optimize partitioning, replication, and retention policies will be conducted regularly.

Lastly, integrating Kafka with ELK Stack and Solr for scalable data storage and analysis will be a significant part of this job.

Qualifications

This role requires a strong foundation in computer science, with a Bachelor's degree in Computer Applications (BE) or a related field being preferred.

Strong programming skills, particularly in Java, are essential, along with experience in working with distributed systems.

A minimum of 3+ years of experience in a similar role is required.

Required Skills and Qualifications

Technical Skills:

  • Kafka Clusters
  • MirrorMaker
  • Kafka Streams
  • Schema Registry
  • Kubernetes
  • Docker
  • Ansible
  • Terraform
  • Monitoring & Logging
  • Programming & Scripting
  • Shell Scripting

Preferred Qualifications:

  • Experience with CI/CD pipelines
  • Familiarity with cloud-based services
  • Knowledge of containerization using Docker
Location

Bangalore, India

  • Data Pipeline Architect

    1 week ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About the Role:The ideal candidate will have a solid foundation in data engineering with a minimum of 7 years of experience in development on data-centric projects.Mandatory Skills:Expertise in working with real-time data and Kafka framework, including kSQL/Mirror Maker etc.Proficiency in at least one programming language: Groovy or Java.Thorough...

  • Senior Data Architect

    5 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Overview:We are seeking a skilled Data Engineer to join our team and contribute to the design, implementation, and maintenance of our data infrastructure.About the Role:Design and develop scalable data pipelines and ETL processes to ingest, transform, and store data from various sources.Collaborate with business stakeholders to understand data...

  • Cloud Data Architect

    21 hours ago

    Kanpur, Uttar Pradesh, India beBeeCloudDataEngineer Full time US$ 1,50,000 - US$ 2,00,000

    Job Title: Cloud Data ArchitectLemongrass is a software-enabled services provider, synonymous with SAP on Cloud, focused on delivering superior, highly automated Managed Services to Enterprise customers.We are seeking an experienced Cloud Data Engineer with a strong background in AWS, Azure, and GCP. The ideal candidate will have extensive experience with...

  • Senior Data Pipeline Specialist

    20 hours ago

    Kanpur, Uttar Pradesh, India beBeeBigData Full time ₹ 15,00,000 - ₹ 20,00,000

    Unlock the Power of Data Engineering with a Career that Transforms.We're seeking an accomplished Big Data Engineer to spearhead our data engineering team. The ideal candidate will be highly skilled in designing, developing, and optimizing large-scale data pipelines and distributed data processing systems using Scala, Spark (PySpark), Python, NiFi, and Apache...

  • Architect of Distributed Systems

    11 hours ago

    Kanpur, Uttar Pradesh, India beBeeDistributedSystem Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Architect of Distributed SystemsWe are seeking an experienced professional to lead the development and implementation of a cutting-edge distributed system.The role involves designing, upgrading, and maintaining a Service Mesh, migrating Kafka clusters, deploying microservices on OpenShift, and ensuring secure integration in a highly restricted...

  • Database architect

    7 days ago

    Kanpur, Uttar Pradesh, India Unlink Technologies Private Limited Full time

    We are seeking a highly skilled Database Architect to join our development team. This role goes beyond traditional database management—you will design scalable, high-performance database systems that seamlessly integrate with AI and ML components. The ideal candidate will have deep expertise in database architecture while also contributing to broader...

  • Senior Big Data Engineer

    17 hours ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job TitleAn esteemed role that requires a seasoned professional with expertise in big data technologies.LocationHyderabad & NoidaExperience5 to 8 years of experience is required for this position.Skills and QualificationsWe are looking for an individual with hands-on experience in:Big Data Technologies (AWS preferred)AWS Dynamo DB, EKS, Kafka, Kinesis, Glue,...

  • Enterprise SSO Architect

    5 days ago

    Kanpur, Uttar Pradesh, India beBeeSecurity Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    **Job Title:** Enterprise SSO ArchitectWe seek an experienced Enterprise SSO Architect to design and implement secure single sign-on solutions for our enterprise applications. As a key member of our team, you will be responsible for configuring and managing authentication mechanisms, integrating with internal and external applications, and ensuring the...

  • Database Architect

    1 week ago

    Kanpur, Uttar Pradesh, India Unlink Technologies Private limited Full time

    We are seeking a highly skilled Database Architect to join our development team. This role goes beyond traditional database management—you will design scalable, high-performance database systems that seamlessly integrate with AI and ML components. The ideal candidate will have deep expertise in database architecture while also contributing to broader...

  • Software Backend Systems Development Leader

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeBackendLeader Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe seek a seasoned leader to drive our backend systems development.Key Responsibilities:Lead and mentor Java developers ensuring high quality and timely delivery of features and bug fixes.Design, develop, and maintain scalable, reliable backend services primarily using Java and Spring Boot.Architect and implement solutions involving Kafka for...