Kafka Infrastructure Architect

1 day ago

BangaloreDelhiPune, India EDGESOFT Full time

Job Description:

\

 

\

At Edgesoft, we are seeking an experienced Kafka Administrator to lead the migration of Kafka workloads from AWS to GCP utilizing Kubernetes and GCP-native services.

\

 

\

The successful candidate will be responsible for architecting, implementing, and managing a scalable Kafka infrastructure on Google Kubernetes Engine (GKE). They will also develop and maintain comprehensive performance metrics dashboards for the Kafka ecosystem, collaborate with cross-functional teams to optimize Kafka API integration, and conduct in-depth assessments of existing AWS Kafka deployments for migration planning.

\

 

\

The ideal candidate will have a strong background in Apache Kafka architecture, internals, and ecosystem components, as well as proficiency in scripting and automation for Kafka management and migration. They will also have hands-on experience with Kafka administration, including cluster setup, configuration, and tuning, and proficiency in Kafka APIs, including Producer, Consumer, Streams, and Connect.

\

 

\

In addition to technical expertise, the successful candidate will possess excellent communication and collaboration skills, as well as the ability to work independently and manage multiple tasks in a fast-paced environment.

\

 

\

Responsibilities:

\
  • \
  • Lead the migration of Kafka workloads from AWS to GCP using Kubernetes and GCP-native services.\
  • Architect, implement, and manage a scalable Kafka infrastructure on GKE.\
  • Develop and maintain comprehensive performance metrics dashboards for the Kafka ecosystem.\
  • Collaborate with cross-functional teams to optimize Kafka API integration.\
  • Conduct in-depth assessments of existing AWS Kafka deployments for migration planning.\
  • Develop and execute a robust migration strategy encompassing data migration and consumer repointing.\
  • Optimize Kafka performance and scalability for high-throughput data processing.\
  • Implement robust monitoring and alerting for system health and performance.\
  • Guarantee data integrity and security throughout the migration process.\
  • Collaborate with engineering and data teams for seamless integration and optimal performance.\
  • Design and implement a multi-tenant GKE environment to accommodate diverse workloads.\
  • Deploy and configure Kafka Strimzi on the underlying infrastructure.\
  • Identify and implement efficient data ingestion pipelines (e.g., Kafka Connect, Dataflow).\
  • Conduct thorough fault simulation testing to mitigate potential risks.\
  • Stay updated on Kafka developments and GCP services to recommend and implement improvements.\
  • Maintain deep knowledge of Kafka internals and ecosystem components, including Kafka Connect, Kafka Streams, and KSQL.\
  • Work closely with development teams to ensure smooth deployment of Kafka clusters.\
  • Prioritize and manage competing tasks to meet project deadlines.\
  • Communicate effectively with stakeholders to ensure alignment on project objectives.\
  • Demonstrate a commitment to delivering high-quality results and exceeding customer expectations.\
  • Actively seek feedback and continuous learning opportunities to improve professional growth and development.\
  • Stay current with industry trends, emerging technologies, and best practices in software development and operations.\
  • Foster a culture of innovation, experimentation, and calculated risk-taking within the team.\
  • Promote a collaborative and inclusive work environment that values diversity, equity, and inclusion.\
  • Support and uphold the company's mission, vision, and values in all aspects of work.\
  • Suggest and implement process improvements to enhance productivity, efficiency, and effectiveness.\
  • Utilize project management tools to track progress, identify roadblocks, and adjust plans as needed.\
  • Build and maintain relationships with cross-functional teams, vendors, and external partners to drive business growth and success.\
  • Maintain confidentiality and handle sensitive information with discretion.\
  • Perform other duties as assigned by management to support business needs and objectives.\
  • Suggest and participate in training, mentorship, or coaching programs to enhance skill sets and knowledge.\
  • Demonstrate adaptability, flexibility, and resilience in response to changing priorities, unexpected challenges, or shifting requirements.\
  • Exhibit exceptional problem-solving skills, analytical thinking, and creative approach to addressing complex issues.\
  • Engage in self-reflection and continuous improvement to address areas for personal and professional growth.\\

  • Kafka Architect

    2 months ago

    Pune, India ScaleneWorks Full time

    KAFKA / Confluent Architect – 12 – 15 yrs. of experience • Strong experience on Confluent Platform and Kafka Streams, Kafka KSQL, Kafka connect. • Design and architect Kafka-based streaming solutions to meet business requirements. • Develop and maintain Kafka cluster infrastructure, including configuration, tuning, and scaling. • Lead...

  • Kafka Cluster Architect

    3 days ago

    Pune, Maharashtra, India Wipro Full time

    About the RoleWe are seeking an experienced Kafka Cluster Architect to join our team at Wipro. This is a unique opportunity to work with a talented group of professionals and contribute to the success of our organization.As a Kafka Cluster Architect, you will be responsible for designing, implementing, and managing Apache Kafka clusters in production...

  • Kafka Expert

    6 months ago

    Pune, Maharashtra, India growel softech Full time

    **Position : Kafka Expert Exp-4-6 yrs Budget - 7 LPA to 11 LPA (slightly negotiable) Notice : Immediate joiner or 10 Days **Location - Remote Mandatory: **Kafka, confluent Kafka, AWS, KUBERNETES, kafka architect ( **in-depth **knowledge), python ,terraform **Good communication skills Mandatory Skills**: - In Depth knowledge of Kafka architecture** - MSK,...

  • Cloud Infrastructure Architect

    10 hours ago

    Pune, Maharashtra, India TripleLift Full time

    Job TitleA highly skilled Cloud Infrastructure Architect is required to join our team at TripleLift. As a key member of our engineering organization, you will play a crucial role in designing and implementing secure, scalable, and efficient cloud infrastructure using AWS.About the RoleThis position involves setting up and maintaining infrastructure in our...

  • Kafka Administrator

    2 weeks ago

    Bangalore/Delhi,Pune, India EDGESOFT Full time

    Responsibilities : - Lead the migration of Kafka workloads from AWS to GCP utilising Kubernetes and GCP-native services.- Architect, implement, and manage a scalable Kafka infrastructure on GKE.- Develop and maintain comprehensive performance metrics dashboards for the Kafka ecosystem.- Collaborate with cross-functional teams to optimise Kafka API...

  • Senior Apache Kafka Engineer

    1 week ago

    Bangalore/Delhi,Pune, India EDGESOFT Full time

    About the RoleWe are seeking a highly skilled Senior Apache Kafka Engineer to join our team at Edgesoft. As a key member of our technical staff, you will be responsible for designing, implementing, and managing a scalable and high-performance Apache Kafka infrastructure on Google Cloud Platform (GCP).Job DescriptionThe successful candidate will have...

  • Kafka Solutions Architect

    3 weeks ago

    Pune, Maharashtra, India Encora Inc. Full time

    We are seeking an experienced Kafka Solutions Architect to join our team at Encora Inc.In this role, you will be responsible for designing and implementing scalable and highly available Kafka-based systems.The ideal candidate will have a deep understanding of Kafka architecture, scalability, and performance tuning.As a Solutions Architect, you will work...

  • Kafka Systems Architect

    2 weeks ago

    Delhi, Delhi, India Talpro Full time

    Job Description for Kafka AdministratorAt Talpro, we are seeking an experienced Kafka Administrator to play a pivotal role in the development and maintenance of our corporate messaging infrastructure. As a key member of our team, you will be responsible for ensuring the smooth delivery, security, retention, and recoverability of topics and messages.Main...

  • Enterprise Kafka Solutions Architect

    1 day ago

    Bangalore/Pune/Bhubaneshwar/Cochin/Kochi, India Risk Resources LLP Full time

    Job SummaryWe are seeking an experienced Enterprise Kafka Solutions Architect to join our team at Risk Resources LLP.About the RoleThis is a challenging and rewarding role that requires strong technical expertise in designing, developing, and implementing scalable, high-performance data pipelines using Apache Kafka. As an Enterprise Kafka Solutions...

  • Kafka Messaging Architect

    3 weeks ago

    Pune, Maharashtra, India VOIS Full time

    Job DescriptionHiring a skilled Kafka Messaging Architect to join our team at _VOIS (Vodafone Intelligent Solutions)About the RoleWe are seeking an experienced professional with 2-8 years of experience in designing and implementing scalable messaging architectures using Apache Kafka.Key Responsibilities:Design and develop complex message flows and...

  • Kafka Architect

    4 weeks ago

    bangalore, India Connectpro Management Consultants Private Limited Full time

    Location- Bangalore Years of experience - 8 to 15 Years Qualification - B.Tech / MCA Key Responsibilities : Design, deploy, and manage Kafka clusters in a production environment, ensuring high availability, scalability, and reliability. Collaborate with engineering teams to understand data processing requirements and design Kafka solutions that meet...

  • Kafka Architect

    4 weeks ago

    bangalore, India Connectpro Management Consultants Private Limited Full time

    Location- BangaloreYears of experience - 8 to 15 YearsQualification - B.Tech / MCAKey Responsibilities :Design, deploy, and manage Kafka clusters in a production environment, ensuring high availability, scalability, and reliability.Collaborate with engineering teams to understand data processing requirements and design Kafka solutions that meet their...

  • Kafka Architect

    2 weeks ago

    Pune, Maharashtra, India Risk Resources LLP Full time

    About the Role :Overview :This role is a critical part of our data engineering team, where you will work on designing and developing real-time data streaming solutions using Kafka. You will be responsible for building scalable and high-performance data pipelines to meet the organization's evolving data needs. Key Responsibilities :- Designing and developing...

  • Kafka Architect

    6 days ago

    Delhi, India Connectpro Management Consultants Private Limited Full time

    Location- Bangalore Years of experience - 8 to 15 YearsQualification - B.Tech / MCAKey Responsibilities :Design, deploy, and manage Kafka clusters in a production environment, ensuring high availability, scalability, and reliability.Collaborate with engineering teams to understand data processing requirements and design Kafka solutions that meet...

  • Senior Kafka Solutions Architect

    2 weeks ago

    Pune, Maharashtra, India Wipro Full time

    Company Overview: We are Wipro, a leading global information technology consulting company. Our event streaming team is seeking a seasoned expert to join our developer advocacy function.Job Description: As a Senior Kafka Solutions Architect, you will be responsible for ensuring smooth integration of Apache Kafka for our Line of Business (LOB) users. This...

  • Senior Cloud Infrastructure Architect

    5 days ago

    Delhi, Delhi, India Antal International Full time

    We are seeking a highly skilled Senior Cloud Infrastructure Architect to join our team at Antal International. As a key member of our infrastructure team, you will play a crucial role in designing and implementing cloud-based solutions that meet the needs of our clients.About YouTo be successful in this role, you will need to have a strong background in...

  • Kafka Architect

    4 weeks ago

    Bangalore Urban, India Connectpro Management Consultants Private Limited Full time

    Location- BangaloreYears of experience - 8 to 15 YearsQualification - B.Tech / MCAKey Responsibilities: Design, deploy, and manage Kafka clusters in a production environment, ensuring high availability, scalability, and reliability.Collaborate with engineering teams to understand data processing requirements and design Kafka solutions that meet their...

  • Kafka Developer and Solutions Architect

    1 week ago

    Pune, Maharashtra, India Wipro Full time

    Job Title: Kafka Developer and Solutions ArchitectLocation: Toronto/Offshore -PuneWe are seeking a seasoned Solutions IT Developer with a strong background in Apache Kafka to join our developer advocacy function in our event streaming team.The ideal candidate will be responsible for Kafka code reviews with clients, troubleshooting client connection issues...

  • Kafka Admin

    1 month ago

    Bangalore, India Eximietas Design Full time

    About Eximietas: Eximietas, a technology services and solutions company headquartered in San Jose, CA with a global footprint that extends to Bangalore, Chennai, and Bhubaneswar in India. Specializing in Cloud Computing, Cybersecurity, , Embedded Software, and Artificial Intelligence, Eximietas is dedicated to empowering businesses with cutting-edge...

  • Kafka Lead

    3 months ago

    Pune, India Talentpro Edge Full time

    Responsibilities :- Lead the migration of Kafka workloads from AWS to GCP utilising Kubernetes and GCP-native services.- Architect, implement, and manage a scalable Kafka infrastructure on GKE.- Develop and maintain comprehensive performance metrics dashboards for the Kafka ecosystem.- Collaborate with cross-functional teams to optimise Kafka API...