Kafka Infrastructure Architect

4 weeks ago

BangaloreDelhi, India EDGESOFT Full time
Job Title: Kafka Administrator

Job Summary: We are seeking an experienced Kafka Administrator to lead the migration of Kafka workloads from AWS to GCP, utilizing Kubernetes and GCP-native services. The ideal candidate will have a deep understanding of Kafka internals and ecosystem, as well as experience with Kafka administration, including cluster setup, configuration, and tuning.

Responsibilities:
  • Lead the migration of Kafka workloads from AWS to GCP, utilizing Kubernetes and GCP-native services.
  • Architect, implement, and manage a scalable Kafka infrastructure on GKE.
  • Develop and maintain comprehensive performance metrics dashboards for the Kafka ecosystem.
  • Collaborate with cross-functional teams to optimize Kafka API integration.
  • Conduct in-depth assessments of existing AWS Kafka deployments for migration planning.
  • Develop and execute a robust migration strategy encompassing data migration and consumer repointing.
  • Optimize Kafka performance and scalability for high-throughput data processing.
  • Implement robust monitoring and alerting for system health and performance.
  • Guarantee data integrity and security throughout the migration process.
  • Collaborate with engineering and data teams for seamless integration and optimal performance.
  • Design and implement a multi-tenant GKE environment to accommodate diverse workloads.
  • Deploy and configure Kafka Strimzi on the underlying infrastructure.
  • Identify and implement efficient data ingestion pipelines (e.g., Kafka Connect, Dataflow).
  • Conduct thorough fault simulation testing to mitigate potential risks.
  • Stay updated on Kafka developments and GCP services to recommend and implement improvements.
Requirements:
  • Bachelor's degree in Computer Science, Engineering, or related field (Master's degree preferred).
  • Proven hands-on experience as a Kafka Lead or similar role, with a minimum of 5 years of experience.
  • Deep knowledge of Kafka internals and ecosystem, including Kafka Connect, Kafka Streams, and KSQL.
  • In-depth knowledge of Apache Kafka architecture, internals, and ecosystem components.
  • Proficiency in scripting and automation for Kafka management and migration.
  • Hands-on experience with Kafka administration, including cluster setup, configuration, and tuning.
  • Proficiency in Kafka APIs, including Producer, Consumer, Streams, and Connect.
  • Strong programming skills in Java, Scala, or Python.
  • Experience with Kafka monitoring and management tools such as Confluent Control Center, Kafka Manager, or similar.
  • Solid understanding of distributed systems, data pipelines, and stream processing.
  • Experience leading migration projects to Google Cloud Platform (GCP), including migrating Kafka workloads.
  • Familiarity with GCP services such as Google Kubernetes Engine (GKE), Google Cloud Storage, Google Cloud Pub/Sub, and BigQuery.
  • Excellent communication and collaboration skills.
  • Ability to work independently and manage multiple tasks in a fast-paced environment.
  • Knowledge of containerization technologies such as Docker and Kubernetes.
  • Familiarity with related technologies such as Apache ZooKeeper, Apache Flink, or Apache Spark.
  • Working knowledge of hands-on experience with Strimzi Kafka.
Nice to Have:
  • Kafka certification (e.g., Confluent Certified Developer, Confluent Certified Operator).
  • Google Cloud certification (e.g., Google Cloud Certified - Professional Cloud Architect, Google Cloud Certified - Professional Data Engineer).
  • Additional experience with other cloud platforms such as AWS or Azure.

  • Kafka Infrastructure Architect

    4 weeks ago

    Bangalore/Delhi, India EDGESOFT Full time

    Job Title: Kafka AdministratorJob Summary: We are seeking an experienced Kafka Administrator to lead the migration of Kafka workloads from AWS to GCP, utilizing Kubernetes and GCP-native services. The successful candidate will be responsible for architecting, implementing, and managing a scalable Kafka infrastructure on GKE, developing and maintaining...

  • Kafka Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    We are looking for Kafka DeveloperKafka DeveloperExperience Range- 4 - 13 years.Work Location - Mumbai, Pune, Delhi, Hyderabad, Bangalore, Kolkata, BBSR, NoidaMust Have:Design, develop, and manage Kafka-based data pipelinesAbility to architect and create reference architecture for Kafka ImplementationsTroubleshoot Kafka related issuesStrong understating on...

  • Kafka Cluster Architect

    2 weeks ago

    Delhi, Delhi, India Talpro Full time

    Job SummaryWe are seeking a highly skilled Kafka Cluster Architect to join our team at Talpro. As a key member of our corporate messaging infrastructure team, you will play a crucial role in designing, building, and maintaining our enterprise-wide messaging architecture.Key ResponsibilitiesDesign and implement high-quality messaging and streaming solutions...

  • Kafka Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    We are looking for Kafka DeveloperKafka DeveloperExperience Range- 4 - 13 years.Work Location - Mumbai, Pune, Delhi, Hyderabad, Bangalore, Kolkata, BBSR, NoidaMust Have:Design, develop, and manage Kafka-based data pipelinesAbility to architect and create reference architecture for Kafka ImplementationsTroubleshoot Kafka related issuesStrong understating on...

  • Kafka Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    We are looking for Kafka Developer Kafka Developer Experience Range- 4 - 13 years. Work Location - Mumbai, Pune, Delhi, Hyderabad, Bangalore, Kolkata, BBSR, Noida Must Have: Design, develop, and manage Kafka-based data pipelines Ability to architect and create reference architecture for Kafka Implementations Troubleshoot Kafka related issues Strong...

  • Kafka Architecture Expert

    2 weeks ago

    Delhi/Bangalore, India Talentpro Edge Full time

    Job SummaryAs a Kafka Lead at Talentpro Edge, you will be responsible for leading the migration of Kafka workloads from AWS to GCP, utilizing Kubernetes and GCP-native services. You will be the key architect, implementer, and manager of a scalable Kafka infrastructure on GKE, ensuring seamless integration with cross-functional teams.Main ResponsibilitiesLead...

  • Kafka Architect

    2 weeks ago

    Bangalore/Pune/Bhubaneshwar/Cochin/Kochi, India Risk Resources LLP Full time

    Key Responsibilities:- Design and implement real-time data streaming solutions using Kafka at an enterprise level.- Develop scalable and high-performance Kafka-based solutions to handle real-time data streams.- Collaborate with stakeholders to understand and define requirements for Kafka-based event streaming applications.- Implement and optimize Kafka...

  • Kafka Cluster Architect

    4 weeks ago

    Delhi, Delhi, India Talpro Full time

    Kafka Administrator Job DescriptionAs a key member of our Talpro team, you will play a vital role in designing, building, and maintaining our corporate messaging infrastructure. Your expertise in Kafka will ensure the smooth delivery, security, retention, and recoverability of topics and messages.Key Responsibilities:Design and deploy high-quality messaging...

  • Senior Kafka Architect

    4 weeks ago

    Delhi, Delhi, India tlnt Full time

    Job Title: Senior Kafka ArchitectWe are seeking a highly experienced Kafka Architecture or Senior Developer with Supply Chain Domain & Realtime data stream experience with Kafka and Azure exposure.Key Responsibilities:Lead the design and implementation of a real-time stream data platform for cloud-based business applications.Define strategy and roadmap for...

  • Kafka Cluster Architect

    4 weeks ago

    Delhi, Delhi, India Talpro Full time

    Kafka Administrator Job DescriptionAs a key member of our Talpro team, you will play a vital role in designing, building, and maintaining our corporate messaging infrastructure. Your expertise in Kafka will ensure the smooth delivery, security, retention, and recoverability of topics and messages.Key Responsibilities:Design and deploy high-quality messaging...

  • Kafka Administrator

    1 month ago

    Bangalore/Delhi,Pune, IN EDGESOFT Full time

    Responsibilities : - Lead the migration of Kafka workloads from AWS to GCP utilising Kubernetes and GCP-native services.- Architect, implement, and manage a scalable Kafka infrastructure on GKE.- Develop and maintain comprehensive performance metrics dashboards for the Kafka ecosystem.- Collaborate with cross-functional teams to optimise Kafka API...

  • Kafka admin

    6 days ago

    Bangalore, India Eximietas Design Full time

    About Eximietas: Eximietas, a technology services and solutions company headquartered in San Jose, CA with a global footprint that extends to Bangalore, Chennai, and Bhubaneswar in India. Specializing in Cloud Computing, Cybersecurity, , Embedded Software, and Artificial Intelligence, Eximietas is dedicated to empowering businesses with cutting-edge...

  • Kafka Admin

    3 weeks ago

    Bangalore, India Eximietas Design Full time

    About Eximietas: Eximietas, a technology services and solutions company headquartered in San Jose, CA with a global footprint that extends to Bangalore, Chennai, and Bhubaneswar in India. Specializing in Cloud Computing, Cybersecurity, , Embedded Software, and Artificial Intelligence, Eximietas is dedicated to empowering businesses with cutting-edge...

  • Kafka Admin

    1 week ago

    Bangalore, India Eximietas Design Full time

    About Eximietas: Eximietas, a technology services and solutions company headquartered in San Jose, CA with a global footprint that extends to Bangalore, Chennai, and Bhubaneswar in India. Specializing in Cloud Computing, Cybersecurity, , Embedded Software, and Artificial Intelligence, Eximietas is dedicated to empowering businesses with cutting-edge...

  • Kafka Admin

    1 week ago

    Bangalore, India Eximietas Design Full time

    About Eximietas: Eximietas, a technology services and solutions company headquartered in San Jose, CA with a global footprint that extends to Bangalore, Chennai, and Bhubaneswar in India. Specializing in Cloud Computing, Cybersecurity, , Embedded Software, and Artificial Intelligence, Eximietas is dedicated to empowering businesses with cutting-edge...

  • Infrastructure Architect

    3 weeks ago

    Bangalore, India Cleartrip Full time

    Cleartrip Infrastructure ArchitectAs the Infrastructure Architect, you will oversee the design, optimization, and scaling of Cleartrip's infrastructure architecture. You will lead a team of engineers in maintaining reliability and efficiency, ensuring that our infrastructure supports the rapid deployment of new features while meeting the growing demands of...

  • Cloud Infrastructure Architect

    2 weeks ago

    Bangalore, India Selsoftinc Full time

    Job Title: Cloud Infrastructure ArchitectAbout the Role:We are seeking a skilled Cloud Infrastructure Architect to lead the design, implementation, and management of Google Cloud Platform (GCP) infrastructure for our company. As a Cloud Infrastructure Architect, you will collaborate with cross-functional teams to ensure that the cloud environment is...

  • Senior Engineer

    4 weeks ago

    Bangalore, India Conviva Apac Private Limited Full time

    Unlock the Power of Digital Customer ExperiencesConviva is the pioneering force in understanding and optimizing digital customer experiences. Our Operational Data Platform harnesses comprehensive client-side telemetry, capturing every aspect of customer experience and engagement across all devices. This real-time, cost-effective solution has earned the trust...

  • Kafka Administrator

    2 months ago

    delhi, India SID Global Solutions Full time

    Experience:3+ years in Kafka Admin Role Job Location: Ahmedabad Position: 5 days a week from office(on-site) Create, configure, and manage Kafka topics based on the requirements from various teams. Configure consumer groups and manage Kafka queues to ensure efficient data processing. Tune consumer group settings, such as max.poll.records, fetch.min.bytes,...

  • Kafka Admin/Developer

    3 weeks ago

    bangalore, India ITC Infotech Full time

    Interested candidates can share your resumes to Faijunnisa.shaik@itcinfotech.comNote: Please share your resume along with below details, if the Below JD suitable to your profile.Greetings from ITC Infotech!Total Exp:Rel Exp:Skill:Current company with Payroll:All companies Form 16 & UAN (yes/No): CTC:ECTC:NP:LWD:Current location:Preferred location:Holding any...