Kafka Data Architect Leader

1 week ago

Gurgaon, Haryana, India Airtel International LLP-Airtel Africa Full time

Job Summary:

Airtel International LLP-Airtel Africa seeks a highly skilled and experienced Kafka Data Architect Leader to join our team. As a key member of our data engineering team, you will lead the design, implementation, and optimization of real-time data streaming platforms using Apache Kafka.

You will provide technical leadership and expertise in integrating Kafka with various systems, ensuring high availability, fault tolerance, and performance of large-scale distributed architectures. Your role will involve collaborating with cross-functional teams, offering guidance on best practices, troubleshooting complex Kafka-related issues, and driving improvements in data streaming solutions across the organization.

Key Responsibilities:

  • Kafka Architecture & Design:
  • Design and architect robust and scalable Kafka-based solutions for real-time data streaming, event processing, and analytics.
  • Provide expertise in Kafka's core concepts such as topics, partitions, producers, consumers, consumer groups, and replication.
  • Ensure high availability, fault tolerance, and performance of Kafka clusters.
  • Implementation & Integration:
  • Lead the integration of Kafka with other platforms, services, and databases, ensuring seamless data flow across systems.
  • Collaborate with software engineers to design and implement Kafka producers, consumers, and stream processing pipelines.
  • Performance Optimization:
  • Analyze and optimize Kafka cluster performance, including memory, disk, network utilization, and throughput.
  • Troubleshoot and resolve issues related to Kafka performance, reliability, and scalability.
  • Support & Troubleshooting:
  • Act as the go-to expert for troubleshooting Kafka-related issues in production and non-production environments.
  • Provide guidance on monitoring Kafka clusters, implementing alerts, and using tools such as Kafka Manager, Confluent Control Center, and others.
  • Best Practices & Documentation:
  • Establish and enforce best practices for deploying, managing, and scaling Kafka clusters.
  • Create and maintain documentation for Kafka deployment, monitoring, configuration, and troubleshooting procedures.
  • Mentorship & Knowledge Sharing:
  • Mentor team members on Kafka-related topics, ensuring that the team stays up-to-date with best practices and new Kafka features.
  • Conduct knowledge-sharing sessions and workshops to promote Kafka adoption within the organization.
  • Security & Compliance:
  • Implement security measures for Kafka clusters, including authentication, authorization, encryption, and access control.
  • Ensure compliance with data privacy regulations and organizational security policies related to data streaming.

Required Skills and Qualifications:

  • Experience:
  • 5+ years of experience with Apache Kafka and related technologies (e.g., Kafka Streams, Kafka Connect, KSQL).
  • Strong understanding of distributed systems, event-driven architecture, and real-time data processing.
  • Technical Skills:
  • Expertise in Kafka cluster setup, configuration, and management.
  • Hands-on experience with Kafka topics, partitions, replication, and fault tolerance.
  • Proficiency with Kafka-related tools like Kafka Manager, Kafka Streams, Kafka Connect, and Confluent platform tools.
  • Experience in integrating Kafka with big data platforms, databases, and message queues.
  • Strong programming skills in languages such as Java, Scala, Python, or Go for developing Kafka producers and consumers.
  • Performance Tuning & Troubleshooting:
  • Experience optimizing Kafka performance (e.g., producer/consumer throughput, replication strategies).
  • Proficient in troubleshooting issues such as latency, consumer lag, and partition rebalancing.
  • Cloud & Containerization:
  • Experience with cloud platforms (AWS, Azure, GCP) and container orchestration tools like Kubernetes for managing Kafka clusters in cloud environments.
  • Security & Compliance:
  • Knowledge of Kafka security configurations, including SSL, SASL, ACLs, and data encryption at rest/in-transit.

Salary Estimate:$150,000 - $200,000 per year

About Us:

Airtel International LLP-Airtel Africa is a leading telecommunications company operating in Africa. We are committed to providing innovative solutions to our customers and creating a positive impact in the communities we serve.

  • Kafka Systems Architect

    20 hours ago

    Gurgaon, Haryana, India Steerlean Consulting Services Pvt Ltd. Full time

    Job Title: Kafka Systems ArchitectWe are seeking a skilled Kafka Systems Architect to join our team at Steerlean Consulting Services Pvt Ltd. as a Senior Confluent Engineer. The ideal candidate will have 5+ years of experience with Confluent Kafka and its ecosystem, including Control Center, Rest Proxy, HA Proxy, Connect, and Security features.The role...

  • Kafka Infrastructure Architect

    1 week ago

    Gurgaon, Haryana, India Steerlean Consulting Services Pvt Ltd. Full time

    About the JobWe are seeking a highly skilled Kafka Infrastructure Architect to join our team at Steerlean Consulting Services Pvt Ltd. The ideal candidate will have extensive experience in designing and implementing scalable Kafka infrastructure on Google Kubernetes Engine (GKE).Key ResponsibilitiesArchitect, implement, and manage a high-performance Kafka...

  • Data Architect and Engineering Leader

    20 hours ago

    Gurgaon, Haryana, India Markovate Full time

    Job SummaryWe are seeking an experienced Data Engineer with a background in data integration, ETL processes, and data warehousing to lead our team as a Data Architect and Engineering Leader.About MarkovateMarkovate is a leading organization that empowers innovation through cutting-edge technology solutions. We foster a collaborative environment where our...

  • Data Engineering Leader

    1 week ago

    Gurgaon, Haryana, India TalentXO Full time

    At TalentXO, we're looking to hire an experienced Data Engineering Leader for one of our clients in the EduTech industry.Company OverviewOur client offers online cohort-based courses for architects, engineers, and designers from 45+ countries worldwide. They collaborate with global firms like Autodesk, BIG, Google, UNStudio, AECOM, and their graduates are...

  • AWS Data Architect

    3 weeks ago

    Gurgaon, Haryana, India Infogain Full time

    Unlock Your Potential as a Cloud Data ArchitectWe are seeking an experienced AWS Data Architect to join our team at Infogain, a leading human-centered digital platform and software engineering company.About the Role:As an AWS Data Architect, you will be responsible for designing and implementing scalable data architectures on the Amazon Web Services (AWS)...

  • Chief Data Architect

    21 hours ago

    Gurgaon, Haryana, India Snap Find Full time

    Job Opportunity for a Chief Data ArchitectWe are seeking an experienced Chief Data Architect to join our team at Snap Find.About the RoleThis is a challenging position that requires a strong understanding of data architecture and design principles. The successful candidate will be responsible for designing scalable data architectures and pipelines for...

  • Data Architect and Engineer

    4 weeks ago

    Gurgaon, Haryana, India Snap Find Full time

    Company OverviewNovatr is a rapidly growing startup that empowers architects, engineers, and AEC professionals with cutting-edge technology skills.About the RoleWe are seeking an experienced Data Architect and Engineer to join our team. As a Senior Data Engineer at Novatr, you will design, implement, and optimize scalable and efficient data architectures and...

  • Senior Data Architect

    20 hours ago

    Gurgaon, Haryana, India Global KPO Full time

    In this role as Senior Data Architect at Global KPO, you will be responsible for designing and implementing data architectures and pipelines.About the CompanyGlobal KPO is a leading company providing data engineering services.The ideal candidate will have a strong background in designing, implementing, and optimizing scalable and efficient data architectures...

  • Cloud Data Engineering Leader

    21 hours ago

    Gurgaon, Haryana, India IGT Solutions Full time

    We are looking for an experienced Cloud Data Engineering Leader to join our team. As a key member of our leadership team, you will be responsible for driving the development and implementation of our cloud data engineering strategy.Responsibilities include designing and architecting scalable and robust IT solutions, leading cross-functional teams to develop...

  • Chief Cloud Data Architect

    20 hours ago

    Gurgaon, Haryana, India TalentXO Full time

    Job Title: Chief Cloud Data ArchitectWe are seeking a skilled professional to lead the design and development of cloud-based data models, ETL pipelines, and infrastructure for reporting, analytics, and data science at TalentXO.About UsTalentXO is a forward-thinking company committed to harnessing the power of data to drive business growth. Our team is...

  • Lead Data Architect

    1 week ago

    Gurgaon, Haryana, India Global KPO Full time

    About the RoleWe are seeking a highly skilled Lead Data Architect to join our team at Global KPO. The successful candidate will be responsible for designing, implementing, and optimizing scalable and efficient data architectures and pipelines to support various data use cases.Key ResponsibilitiesDevelop and maintain data models, ensuring they are optimized...

  • Data Architect Specialist

    3 weeks ago

    Gurgaon, Haryana, India Publicis Groupe Full time

    Job OverviewPublicis Groupe is a global leader in marketing, media, and digital transformation. We are seeking a highly skilled Data Architect Specialist to join our team.

  • Enterprise Integration Architect

    3 weeks ago

    Gurgaon, Haryana, India NTT DATA Services Full time

    About NTT DATA">NTT DATA is a $30 billion trusted global innovator of business and technology services.">We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success.">As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem of established...

  • Lead Data Architect

    21 hours ago

    Gurgaon, Haryana, India Snap Find Full time

    **Job Title:** Lead Data Architect**Location:** Delhi, India**Estimated Salary:** 25,00,000 - 35,00,000 per annumAbout the RoleInnovate with Novatr, a pioneering organization transforming AEC professionals' skills through cutting-edge technology. As a Lead Data Architect, you'll spearhead data architecture and pipeline development for various use cases.Key...

  • Senior Data Architect

    1 week ago

    Gurgaon, Haryana, India Snap Find Full time

    Company OverviewNovatr is a pioneering company that empowers architects, engineers, and AEC professionals with cutting-edge technology skills.About the RoleWe are seeking an experienced Senior Data Engineer to join our team in Delhi. As a key member of our data engineering team, you will design, implement, and optimize scalable and efficient data...

  • FPGA Architect and Developer Leader

    5 days ago

    Gurgaon, Haryana, India Yo HR Consultancy Full time

    Company Overview:Yo HR Consultancy is a dynamic organization seeking an experienced FPGA Architect and Developer Leader to join our team. We offer a competitive salary package, as outlined below.

  • Senior Big Data Solutions Architect

    1 month ago

    Gurgaon, Haryana, India SMARTWORK IT SERVICES Full time

    Job Title: Senior Big Data Solutions ArchitectJob Summary:We are seeking a skilled Senior Big Data Solutions Architect to join our team at SMARTWORK IT SERVICES. As a Senior Big Data Solutions Architect, you will be responsible for designing and implementing scalable big data solutions to meet the ever-growing needs of our clients.Key...

  • Software Engineering Leader

    2 weeks ago

    Gurgaon, Haryana, India PROCDNA ANALYTICS PRIVATE LIMITED Full time

    Company OverviewProcDNA Analytics Private Limited is a global consulting firm that combines design thinking with cutting-edge technology to deliver innovative Commercial Analytics and Technology solutions. With a team of 200+ professionals across 6 offices, we're a passionate and growing community.About the RoleWe're seeking an experienced Software...

  • Chief Data Architect and Engineer

    3 weeks ago

    Gurgaon, Haryana, India AzimuthGRC Technologies India Private Limited Full time

    Are you a seasoned professional looking to leverage your expertise in data architecture and engineering?We are seeking an experienced Chief Data Architect and Engineer to join our team at AzimuthGRC Technologies India Private Limited.About the RoleAs a Chief Data Architect and Engineer, you will be responsible for conceptualizing and communicating data...

  • Data Architect and Engineer

    1 week ago

    Gurgaon, Haryana, India Summit Consulting Services Full time

    At Summit Consulting Services, we're seeking a highly skilled Data Architect and Engineer to join our team. This is an exceptional opportunity for a seasoned professional with expertise in designing, building, and maintaining data lake and data warehousing solutions.About the RoleThis position involves participating in the entire data pipeline lifecycle,...