Kafka Specialist

3 weeks ago

Hyderabad, Telangana, India Troy Consultancy Full time
Job Description

Job Summary:

We are looking for a skilled Kafka Specialist with strong experience in Kafka Administration, Cloudera Kafka Platform, and managing large-scale, enterprise-level messaging infrastructures. The ideal candidate will be responsible for the deployment, configuration, maintenance, and monitoring of Kafka clusters and related ecosystem components.

Key Responsibilities:

- Install, configure, upgrade, and maintain Apache Kafka and Cloudera Kafka clusters.
- Set up Kafka Connect, Kafka MirrorMaker, and Kafka REST Proxy.
- Monitor performance and fine-tune Kafka brokers, topics, partitions, producers, and consumers.
- Implement security measures including TLS encryption, Kerberos authentication, and role-based access control (RBAC).
- Develop scripts for automation using Shell, Python, or other scripting tools.
- Set up and manage Kafka Metrics, Monitoring Dashboards (Grafana, Prometheus), and log analysis tools (Splunk).
- Handle incident management and root cause analysis for Kafka-related issues.
- Collaborate with DevOps and development teams to support Kafka-based applications.
- Maintain documentation for Kafka infrastructure, SOPs, and upgrade procedures.

Required Skills & Experience:

- 5+ years of experience in Kafka Administration and Infrastructure.
- Expertise in Apache Kafka and Cloudera Kafka (CDP/CDH).
- Strong understanding of Kafka internals, producer-consumer architecture, and message delivery semantics.
- Experience with Kafka Tools like:
- Kafka Connect
- Kafka MirrorMaker
- Kafka REST Proxy
- Proficient in:
- Linux/Unix Administration
- Scripting (Shell/Python)
- Monitoring Tools (Grafana, Prometheus)
- Logging Tools (Splunk, ELK)
- Experience with SSL/TLS, SASL, Kerberos, and RBAC configuration for Kafka security.
- Familiarity with Zookeeper management and troubleshooting.
- Experience in CI/CD pipelines and DevOps practices (Jenkins, Git, etc.) is a plus.
- Understanding of Cloud environments (AWS, Azure) and containerization (Docker/Kubernetes) is a plus.

-

  • Distributed Messaging Specialist

    2 days ago

    Hyderabad, Telangana, India beBeeKafka Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Distributed Messaging SpecialistWe are looking for a talented Distributed Messaging Specialist to join our team. The ideal candidate will have expertise in designing, implementing, and maintaining distributed messaging systems.Key Responsibilities:Design and implement high-performance distributed messaging systems using Kafka.Develop and maintain...

  • Distributed Messaging System Expert

    4 days ago

    Hyderabad, Telangana, India beBeeKafka Full time ₹ 15,00,000 - ₹ 25,00,000

    Kafka Infrastructure SpecialistThe role of Kafka Infrastructure Specialist is to ensure the smooth operation of our distributed messaging system. This involves designing, developing, and executing tests to guarantee that our Kafka systems meet the required standards.Key Responsibilities:Develop a strong understanding of Kafka concepts, including topics,...

  • Data Architectural Specialist

    4 days ago

    Hyderabad, Telangana, India beBeeDataSpecialist Full time ₹ 9,00,000 - ₹ 12,00,000

    **Job Opportunity: Data Architectural Specialist** Location: Various | Type: Full-timeOverview:We're seeking an exceptional Data Architectural Specialist to design, develop, and implement large-scale data architecture solutions. This role will focus on building high-performance, scalable data pipelines using the latest technologies.Key...

  • Senior Data Architecture Specialist

    4 days ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    **Job Title:** Senior Data Architecture Specialist**Job Description:** We are seeking an experienced Senior Data Architecture Specialist to lead our data engineering team. The ideal candidate will have strong experience in designing, developing, and maintaining robust and scalable data pipelines using Python.The successful candidate will be responsible for...

  • Advanced API Architect

    4 days ago

    Hyderabad, Telangana, India beBeeApiDesigner Full time ₹ 15,00,000 - ₹ 18,00,000

    API Design SpecialistWe are seeking a highly skilled API Design Specialist to join our organization. As an API Design Specialist, you will be responsible for designing and developing Rest APIs, Java Spring boot, and other data streaming services.Design and develop new as well as update Rest API and Kafka event ServicesDefine endpoints, data formats, and...

  • Data Streaming Specialist

    7 hours ago

    Hyderabad, Telangana, India beBeeFlink Full time ₹ 20,00,000 - ₹ 25,00,000

    Apache Flink EngineerThis role will be instrumental in building and maintaining robust, scalable, and reliable data pipelines using Confluent Kafka, ksqlDB, Kafka Connect, and Apache Flink.The ideal candidate will have a strong understanding of data streaming concepts, experience with real-time data processing, and a passion for building high-performance...

  • Data Architect Specialist

    6 days ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Architect SpecialistWe are seeking a highly skilled Data Architect Specialist to join our team. As a key member of our organization, you will play a crucial role in designing and implementing scalable data pipelines using Databricks and PySpark.Required Skills and Qualifications:Expertise in building scalable data pipelines using Databricks...

  • Data Assurance Specialist

    8 hours ago

    Hyderabad, Telangana, India beBeeDataQuality Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Quality Assurance SpecialistWe are seeking a highly skilled Quality Assurance Specialist with 5-10 years of professional experience to ensure the integrity, reliability, and performance of our data pipelines and AI/ML solutions. The ideal candidate will be responsible for defining and implementing comprehensive quality assurance strategies for data...

  • API Development Specialist

    18 hours ago

    Hyderabad, Telangana, India beBeeAPI Full time ₹ 14,47,050 - ₹ 21,31,050

    Job Opportunity: API Developer SpecialistAbout the RoleWe are seeking an experienced API developer to join our team as a specialist in API development and testing. The ideal candidate will have a strong background in software development, with a focus on designing, developing, and executing automated functional and performance tests for RESTful APIs.Key...

  • Information Specialist

    5 days ago

    Hyderabad, Telangana, India The Adecco Group Full time

    About the roleAdecco is hiring a Information Specialist - Integration for our client, a leading company based in India. This is an exciting opportunity to join a dynamic organization that values innovation, teamwork, and professional growthJob Title: Information Specialist – Integration Expert (Team Lead)Location: India (Remote)Employment Type - 12-month...