Current jobs related to Kafka Platform Engineers - Hyderabad - SID Global Solutions

  • Kafka Streaming Platform Expert

    8 hours ago

    Hyderabad, Telangana, India DSV Full time

    Hiring OverviewDSV is expanding its Hybrid Computing Platform department and seeking a skilled engineer to join our Event Streaming team. We are looking for a hands-on expert in Kafka streaming platform development, maintenance, and improvement.About the RoleThe successful candidate will be responsible for participating in development using Confluent Cloud,...

  • Kafka Engineer for Hybrid Computing Platform

    2 weeks ago

    Hyderabad, Telangana, India DSV Full time

    Event Streaming ExpertiseWe are seeking a skilled Kafka engineer to join our Hybrid Computing Platform department in Global IT. As a Site Reliability Engineer for Event Streaming Solutions, you will be responsible for developing, publishing, and maintaining infrastructure products.Key ResponsibilitiesParticipate in development using Confluent Cloud,...

  • Kafka Engineer

    3 weeks ago

    Hyderabad, Telangana, India DSV Full time

    About the RoleWe are seeking a highly skilled Kafka Engineer to join our Hybrid Computing Platform department in Global IT. As a key member of our Event Streaming team, you will be responsible for designing, developing, and maintaining our event streaming solutions using Confluent Cloud, Confluent Platform, and Apache Kafka.Key ResponsibilitiesParticipate in...

  • Hybrid Computing Platform Kafka Specialist

    7 days ago

    Hyderabad, Telangana, India DSV Full time

    Kafka Specialist & Site Reliability Engineer OpportunityWe are seeking a skilled Kafka Specialist & Site Reliability Engineer to join our Hybrid Computing Platform department in Global IT. As a member of our Event Streaming team, you will be responsible for designing, developing, and maintaining event streaming solutions using Confluent Cloud, Confluent...

  • Kafka Engineer

    3 weeks ago

    Hyderabad, Telangana, India DSV Full time

    Unlock the Power of Event StreamingAt DSV, we're shaping the future of hybrid computing platforms, and we need a talented Kafka expert to join our team. As a Site Reliability Engineer, you'll play a crucial role in ensuring the smooth operation of our event streaming solutions.Key Responsibilities:Design and implement scalable event streaming architectures...

  • Kafka Engineer

    1 day ago

    Hyderabad, India Health Catalyst Full time

    About UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...

  • Kafka engineer

    2 days ago

    Hyderabad, India Health Catalyst Full time

    About UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...

  • Kafka Engineer

    3 weeks ago

    hyderabad, India Health Catalyst Full time

    About UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...

  • Kafka engineer

    3 weeks ago

    Hyderabad, India Health Catalyst Full time

    About Us The healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends....

  • Kafka Engineer

    3 weeks ago

    Hyderabad, India Health Catalyst Full time

    About Us The healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends....

  • Kafka Specialist and Site Reliability Engineer for Hybrid Computing Platform

    2 weeks ago

    Hyderabad, Telangana, India DSV Full time

    About the RoleWe are seeking a highly skilled Kafka Specialist and Site Reliability Engineer to join our Hybrid Computing Platform department. As a key member of our team, you will be responsible for designing, implementing, and maintaining our event streaming solutions using Apache Kafka.Key ResponsibilitiesParticipate in development using Confluent Cloud,...

  • Kafka Engineer

    2 weeks ago

    hyderabad, India Health Catalyst Full time

    About Us The healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends....

  • Kafka Architect

    8 hours ago

    Hyderabad, Telangana, India Alignity Solutions Full time

    About the RoleWe are looking for a highly skilled Kafka Architect to lead the design, development, and deployment of our Kafka-based data pipeline. As a Kafka Architect, you will be responsible for designing and implementing scalable, fault-tolerant, and high-performance Kafka clusters.Key Responsibilities:Kafka Cluster Design: Design and implement scalable...

  • Kafka Engineer

    1 month ago

    Remote/Hyderabad, India Tekvaly Full time

    Responsibilities : - Design, develop, and implement scalable and fault-tolerant data pipelines using Kafka topics, producers, and consumers.- Configure and manage Kafka clusters for high availability, performance, and security.- Monitor and troubleshoot Kafka infrastructure to identify and resolve performance issues and potential bottlenecks.- Implement...

  • Kafka Developer

    3 weeks ago

    Hyderabad, Telangana, India ScaleneWorks Full time

    Kafka Developer Job DescriptionJob Summary: We are seeking an experienced Kafka Developer to join our team at ScaleneWorks. The ideal candidate will have a strong background in developing and maintaining high-performance data pipelines and streaming applications using Confluent Kafka.Key Responsibilities:Design, develop, and deploy Confluent Kafka Streams...

  • Kafka Architect

    3 weeks ago

    Hyderabad, Telangana, India ScaleneWorks Full time

    ScaleneWorks - Confluent Kafka DeveloperJob Summary: We are seeking a highly skilled Confluent Kafka Developer to join our team at ScaleneWorks. The ideal candidate will have a strong background in developing high-performance data pipelines and streaming applications using Confluent Kafka.Key Responsibilities:Design, develop, and deploy Confluent Kafka...

  • Kafka Architect

    3 weeks ago

    Hyderabad, Telangana, India Alignity Solutions Full time

    Job Title: Kafka SMEWe are seeking a highly skilled Kafka Subject Matter Expert (SME) to lead the architecture, management, and scaling of Kafka environments in 24x7 production settings. The ideal candidate will have a deep understanding of Apache Kafka, a proven track record of resolving complex production and non-production issues, and strong leadership...

  • Kafka Architect

    3 weeks ago

    Hyderabad, Telangana, India Alignity Solutions Full time

    Job Title: Kafka SMEWe are seeking a highly skilled Kafka Subject Matter Expert (SME) to lead the architecture management and scaling of Kafka environments in 24x7 production settings. The ideal candidate will have a deep understanding of Apache Kafka, a proven track record of resolving complex production and non-production issues, and strong leadership...

  • Centralized Kafka Specialist

    2 weeks ago

    hyderabad, India ValueLabs Full time

    Dear Aspirants,Greetings from ValueLabs !!We are seeking a skilled Centralized Kafka Specialist who can join our team immediately.Primary Skills : Centralized Kafka SpecialistExperience: 9- 12 yearsLocation: HyderabadNotice period: Immediate -7 days JoinersKey Responsibilities:Design and implement Kafka clusters and topologies to meet the organization's...

  • Kafka Architect

    3 weeks ago

    Hyderabad, Telangana, India Alignity Solutions Full time

    Job Title: Kafka SMEWe are seeking a highly skilled Kafka Subject Matter Expert (SME) to lead the architecture, management, and scaling of Kafka environments in 24x7 production settings.Key Responsibilities:Kafka Management:Design, build, manage, and upgrade Kafka clusters in production and non-production environments.Ensure the Kafka infrastructure supports...

Kafka Platform Engineers

2 months ago

Hyderabad, India SID Global Solutions Full time

Job Description -


The Confluent Kafka Platform Engineer plays a critical role in managing Kafka infrastructure on a day-to-day basis. This involves creating and managing Kafka topics, configuring queues, and ensuring the platform is optimized for performance and reliability. The engineer will collaborate with various teams to meet their streaming data needs while ensuring that Kafka is running smoothly and efficiently.


Daily Routine and Key Responsibilities:


1. Kafka Topic and Queue Management

  • Topic Creation and Management:
  • Create, configure, and manage Kafka topics based on the requirements from various teams.
  • Set appropriate partitions and replication factors to ensure data durability and scalability.
  • Monitor topic health, including under-replicated partitions, and take corrective actions as necessary.
  • Queue Configuration and Management:
  • Configure consumer groups and manage Kafka queues to ensure efficient data processing.
  • Tune consumer group settings, such as max.poll.records, fetch.min.bytes, and session.timeout.ms, to optimize throughput and latency.
  • Monitor and resolve issues related to consumer lag, ensuring that data is processed in a timely manner.

2. Configuration Management

  • Broker Configuration:
  • Adjust broker-level configurations (e.g., log.retention.hours, log.segment.bytes, zookeeper.connection.timeout.ms) to maintain optimal performance and stability.
  • Apply updates and patches to Kafka brokers and ensure configurations are in line with best practices.
  • Security Configuration:
  • Manage Kafka security settings, including SSL/TLS encryption, Kerberos, and SASL authentication.
  • Configure ACLs (Access Control Lists) to control access to topics and consumer groups, ensuring secure and compliant operations.
  • Schema Registry and Compatibility:
  • Work with the Confluent Schema Registry to enforce schema validation and compatibility.
  • Manage versioning of schemas and ensure backward and forward compatibility across different services.

3. Monitoring and Troubleshooting

  • Platform Monitoring:
  • Monitor Kafka clusters using tools like Prometheus, Grafana, and Confluent Control Center to track key metrics (e.g., broker CPU usage, disk I/O, topic throughput).
  • Set up and manage alerts for critical issues like broker failures, high consumer lag, and message backlog.
  • Issue Resolution:
  • Respond to incidents involving Kafka infrastructure, such as broker crashes, network partitions, or performance degradations.
  • Troubleshoot and resolve issues related to topic misconfigurations, consumer group errors, and producer failures.
  • Performance Tuning:
  • Continuously optimize Kafka performance by adjusting configurations based on observed behavior and workload patterns.
  • Conduct periodic health checks and capacity planning to anticipate and prevent potential bottlenecks.

4. Collaboration and Support

  • Inter-Team Collaboration:
  • Work closely with development, DevOps, and data engineering teams to understand their Kafka usage patterns and requirements.
  • Provide guidance on best practices for Kafka topic and consumer group configurations.
  • Support and Documentation:
  • Assist teams with onboarding to Kafka, including setting up topics, consumer groups, and security credentials.
  • Document Kafka configurations, operational procedures, and incident resolutions to ensure knowledge sharing and continuity.


Tools and Technologies Used:

  • Kafka Management: Apache Kafka, Confluent Control Center
  • Monitoring and Alerts: Prometheus, Grafana, Elasticsearch, Kibana, Confluent Health+
  • Automation: Shell Scripting, Ansible, Jenkins, GitLab CI
  • Security: SSL/TLS, Kerberos, SASL, ACLs
  • Collaboration: Jira, Confluence, Slack


NEED TO HAVE CERTIFICATION IN APACHE.


Qualifications:

  • Strong knowledge and hands-on experience with Apache Kafka and the Confluent Platform.
  • Proficiency in configuring and managing Kafka topics, queues, and consumer groups.
  • Experience with Kafka monitoring, troubleshooting, and performance tuning.
  • Understanding of Kafka security configurations and best practices.
  • Excellent problem-solving skills and the ability to work in a fast-paced environment.