Apache Kafka Admin

**Role**:
**Responsibilities**:

- Install, configure, upgrade, and manage Apache Kafka clusters in production and non-production environments.
- Partner with architects and developers to define messaging strategies, topics, partitions, and retention policies.
- Monitor, tune, and optimize Kafka for throughput, latency, and scalability.
- Implement security controls including SSL, SASL, authentication, and authorization.
- Manage Kafka Connect, Schema Registry, and Kafka Streams to support integrations.
- Develop backup, recovery, disaster recovery, and high availability strategies for Kafka.
- Troubleshoot and resolve incidents related to brokers, producers, and consumers.
- Automate cluster provisioning, scaling, and configuration via scripts and Infrastructure-as-Code.
- Integrate Kafka monitoring and alerting with observability tools.
- Provide expertise and best practices guidance to engineering teams consuming Kafka services.
- Implement and maintain backup and recovery strategies for Kafka clusters to ensure data durability and reliability.
- Assist in migrating legacy messaging or data workflows (e.g., MSMQ, SQL Server-based queues) to Apache Kafka, ensuring mínimal disruption.
- Monitor Kafka cluster performance using tools like Confluent Control Center, Prometheus, Grafana, or custom scripts, proactively identifying and resolving issues.
- Troubleshoot complex Kafka-related issues, including connectivity, message delivery failures, and performance bottlenecks.
- Implement monitoring and alerting systems to ensure 24/7 reliability of Kafka clusters for SaaS customers.
- Ensure Kafka cluster security by implementing best practices for access control (e.g., SASL, ACLs), encryption (e.g., SSL/TLS), and auditing.
- Maintain compliance with industry standards (e.g., GDPR, SOC 2, or other relevant regulations) for data streaming and processing.
- Work closely with.NET Core and React developers, data engineers, and DevOps teams to optimize Kafka integration and ensure efficient data processing.
- Document Kafka configurations, processes, and optimization strategies to support team knowledge sharing.

**Required Skills**:

- Minimum 7-8+ years of hands-on experience as a Kafka Administrator in a production environment, preferably within a SaaS or product-based organization.
- Experience with migrating legacy messaging systems to Apache Kafka is highly desirable.
- Deep expertise in Apache Kafka administration, including cluster setup, configuration, topic management, replication, and performance tuning.
- Strong knowledge of Kafka ecosystem tools (e.g., Kafka Connect, Schema Registry, Kafka Streams, ksqlDB).
- Experience with monitoring tools (e.g., Confluent Control Center, Prometheus, Grafana) and scripting (e.g., Bash, Python) for automation.
- Knowledge of cloud-based Kafka deployments (e.g., Confluent Cloud, AWS MSK, Azure Event Hubs for Kafka) is a plus.
- Bachelor’s degree in Computer Science, Information Technology, or a related field (or equivalent experience).
- Good to Have Kafka certifications (e.g., Confluent Certified Administrator for Apache Kafka) or cloud certifications (e.g., AWS, Azure) are a plus.
- Good to Have Familiarity with DevOps practices, including CI/CD pipelines and infrastructure-as-code (e.g., Terraform, Ansible).