Senior Kafka Architect

1 month ago

Bengaluru, India Coforge Full time

Job Description

Position: Senior Kafka Architect

Location: Bangaluru

Duration: Full Time

MUST HAVE SKILLS:

A: Kafka

B: Storm

C: Architectural components: brokers, Zookeeper, Producers/Consumers

JOB DESCRIPTION:

Looking for a senior Kafka engineer/architect with an established track record in Kafka technology, with hands-on production experience and a deep understanding of the Kafka architecture and internals of how it works, along with interplay of architectural components: brokers, Zookeeper, Producers/Consumers etc.

SOME OF THE KEY RESPONSIBILITIES ARE LISTED BELOW:

Architecture Design: Design Kafka-based data streaming architecture that meet scalability, fault-tolerance, and performance requirements. Cluster Sizing: Determine the appropriate cluster size, partitioning, and replication factors based on workload and data volume. Topic Modeling: Define Kafka topic schemas, naming conventions, and access control policies for efficient data organization. Security Planning: Implement security measures like SSL/TLS encryption, SASL authentication, and ACLs to protect data and Kafka infrastructure. High Availability: Ensure Kafka clusters are highly available and fault-tolerant by configuring replication, leader election, and monitoring. Disaster Recovery: Plan and implement disaster recovery strategies to ensure data integrity in case of cluster failures. Performance Optimization: Fine-tune Kafka configurations, monitor performance metrics, and optimize resource utilization for efficient data processing. Monitoring and Alerting: Set up monitoring tools and alerting systems to detect issues, track performance, and respond to anomalies. Load Testing: Conduct load testing and benchmarking to validate the Kafka architecture’s performance and scalability. Cost Management: Optimize resource usage to control operational costs while maintaining performance and reliability. Training and Support: Provide training to other team members and support staff on Kafka best practices and troubleshooting. Documentation: Maintain comprehensive documentation for Kafka configurations, data flows, and architecture diagrams. Scaling Strategies: Develop strategies for horizontally and vertically scaling Kafka clusters based on changing workloads. Hands-On Experience: Hands-on experience as a developer who has used the Kafka API to build producer and consumer applications/components. Be able to come up with library/framework for common functionality/use cases to have different teams reuse code rather than writing new code. Message Formats: Strong familiarity of message formats such as XML, JSON, CSV, etc. along with serialization/deserialization options. Posted On

Position: Senior Kafka Architect

Location: Bangaluru

Duration: Full Time

 

MUST HAVE SKILLS:

A: Kafka

B: Storm

C: Architectural components: brokers, Zookeeper, Producers/Consumers

 

JOB DESCRIPTION:

  • Looking for a senior Kafka engineer/architect with an established track record in Kafka technology, with hands-on production experience and a deep understanding of the Kafka architecture and internals of how it works, along with interplay of architectural components: brokers, Zookeeper, Producers/Consumers etc.

 

SOME OF THE KEY RESPONSIBILITIES ARE LISTED BELOW:

  • Architecture Design: Design Kafka-based data streaming architecture that meet scalability, fault-tolerance, and performance requirements.
  • Cluster Sizing: Determine the appropriate cluster size, partitioning, and replication factors based on workload and data volume.
  • Topic Modeling: Define Kafka topic schemas, naming conventions, and access control policies for efficient data organization.
  • Security Planning: Implement security measures like SSL/TLS encryption, SASL authentication, and ACLs to protect data and Kafka infrastructure.
  • High Availability: Ensure Kafka clusters are highly available and fault-tolerant by configuring replication, leader election, and monitoring.
  • Disaster Recovery: Plan and implement disaster recovery strategies to ensure data integrity in case of cluster failures.
  • Performance Optimization: Fine-tune Kafka configurations, monitor performance metrics, and optimize resource utilization for efficient data processing.
  • Monitoring and Alerting: Set up monitoring tools and alerting systems to detect issues, track performance, and respond to anomalies.
  • Load Testing: Conduct load testing and benchmarking to validate the Kafka architecture’s performance and scalability.
  • Cost Management: Optimize resource usage to control operational costs while maintaining performance and reliability.
  • Training and Support: Provide training to other team members and support staff on Kafka best practices and troubleshooting.
  • Documentation: Maintain comprehensive documentation for Kafka configurations, data flows, and architecture diagrams.
  • Scaling Strategies: Develop strategies for horizontally and vertically scaling Kafka clusters based on changing workloads.
  • Hands-On Experience: Hands-on experience as a developer who has used the Kafka API to build producer and consumer applications/components. Be able to come up with library/framework for common functionality/use cases to have different teams reuse code rather than writing new code.
  • Message Formats: Strong familiarity of message formats such as XML, JSON, CSV, etc. along with serialization/deserialization options .
Skills Required

KAFKA AND STORM

Posted On KAFKA AND STORM Location

Bangalore

Posted On Bangalore

  • Kafka Architect

    4 weeks ago

    Bengaluru, India Brillio Full time

    **Kafka Architect**: **Work Location** : Bangalore **Experience**: 10 to 15Years **Roles and Responsibilities**: - Provide consultative recommendations to clients to solve their Big Data challenges - Work as an individual contributor and or team player based on the projects - Confluent Kafka Platform consulting, Platform architecture and implementation -...

  • Kafka Architect

    3 weeks ago

    Bengaluru, India ConnectPro Management Consultants Pvt Ltd. Full time

    Work Location: Bangalore. Hybrid model - Working in office 3 days in a week. Experience : 8+ Years. We are looking for a Kafka Architect who can Work alongside multiple Production Support, Development, System Admin, Project Managers and Engineering teams.Key Responsibilities: - Design, deploy, and manage Kafka clusters in a production environment, ensuring...

  • Kafka architect

    2 weeks ago

    Bengaluru, India Rakuten India Full time

    About Rakuten:Rakuten India is the Development Centre and key technology hub of the Rakuten Group, Inc. We enable our businesses with our depth of knowledge in multiple streams of technology such as Mobile and Web Development, Web Analytics, Platform Development, Backend Engineering, Data Science, Machine Learning, Artificial Intelligence and much more. Our...

  • Kafka architect

    2 weeks ago

    Bengaluru, India Rakuten India Full time

    About Rakuten:Rakuten India is the Development Centre and key technology hub of the Rakuten Group, Inc. We enable our businesses with our depth of knowledge in multiple streams of technology such as Mobile and Web Development, Web Analytics, Platform Development, Backend Engineering, Data Science, Machine Learning, Artificial Intelligence and much more. Our...

  • Kafka architect

    3 weeks ago

    Bengaluru, India Rakuten India Full time

    About Rakuten:Rakuten India is the Development Centre and key technology hub of the Rakuten Group, Inc. We enable our businesses with our depth of knowledge in multiple streams of technology such as Mobile and Web Development, Web Analytics, Platform Development, Backend Engineering, Data Science, Machine Learning, Artificial Intelligence and much more. Our...

  • Kafka Technical Lead

    1 week ago

    Bengaluru, India TEKsystems Global Services in India Full time

    We are seeking a highly skilled AWS Senior Technical Lead having an experience of 10+ years with Kafka cluster setup, implementation and a strong hands-on background on Data Streaming, Engineering and Computing. As an AWS Senior Lead, you will play a crucial role in designing, developing, and implementing cloud-based solutions on the AWS platform. The ideal...

  • Kafka Technical Lead

    1 week ago

    Bengaluru, India TEKsystems Global Services in India Full time

    We are seeking a highly skilled AWS Senior Technical Lead having an experience of 10+ years with Kafka cluster setup, implementation and a strong hands-on background on Data Streaming, Engineering and Computing. As an AWS Senior Lead, you will play a crucial role in designing, developing, and implementing cloud-based solutions on the AWS platform. The ideal...

  • Java Developer

    2 weeks ago

    Bengaluru, India MLOPS SOLUTIONS PRIVATE LIMITED Full time

    Job Summary :We are seeking a talented Java Developer with expertise in Kafka to join our dynamic team. The ideal candidate will be responsible for designing, developing, and maintaining high-performance, scalable Java applications with Kafka integration. This role requires strong Java programming skills, deep understanding of Kafka architecture and...

  • Kafka Data Engineer

    2 weeks ago

    Bengaluru, India Codersbrain technology pvt ltd Full time

    Mandatory skills : - Azure, Kafka, Data Engineering or any Data Platforms background- Experience with Azure/MS Data Platforms: Postgres Single and Flex, Azure SQL, MS SQL server, Kubernetes- Terraform (Preferred not Compulsory), Bicep or other IaC.- Helm (helm deployments as well writing charts).- Azure DevOps pipelines.- Engineer, Data Engineer, or similar...

  • Senior Technical Architect

    1 month ago

    Bengaluru, India Coforge Full time

    12+ years of experience needed for this role, We are looking for a hands-on Technical architect who is good on coding in Java + microservices . Please do not apply if you are not meeting the criterion. Coforge is seeking a skilled and experienced Java Technical Architect to join our dynamic team. As a Technical Architect, you will play a crucial role in...

  • Senior Technical Architect

    1 month ago

    Bengaluru, India Coforge Full time

    12+ years of experience needed for this role, We are looking for a hands-on Technical architect who is good on coding in Java + microservices . Please do not apply if you are not meeting the criterion. Coforge is seeking a skilled and experienced Java Technical Architect to join our dynamic team. As a Technical Architect, you will play a crucial role in...

  • Senior Technical Architect

    1 month ago

    Bengaluru, India Coforge Full time

    12+ years of experience needed for this role, We are looking for a hands-on Technical architect who is good on coding in Java + microservices . Please do not apply if you are not meeting the criterion. Coforge is seeking a skilled and experienced Java Technical Architect to join our dynamic team. As a Technical Architect, you will play a crucial role in...

  • RabbitMQ & Kafka Administrator

    1 month ago

    Bengaluru, India Leadway Resource Full time

    We are seeking a highly skilled and experienced Messaging Platform Administrator to join our team. You will be responsible for the administration, operations, and optimization of our on-premises and cloud-native Kafka and RabbitMQ clusters. You will play a key role in ensuring the high availability, performance, and security of these messaging platforms,...

  • Java Architect/Senior Architect

    1 month ago

    Bengaluru, India Squareroot Consulting Pvt Ltd. Full time

    Hiring Java Architect or Senior Architect for SaaS Product Company in Fintech domain.Looking for Tech Architects who have built fault tolerant, Scalable and High Available Systems. Must be hands on coding/designing & have built for scale.Position: Software ArchitectExperience : 10+ YearsLocation: Bangalore, IndiaEducation: BE/BTech/MS/MTech (Preferably from...

  • Senior DevOps Engineer

    1 month ago

    Bengaluru, India PradeepIT Consulting Services Pvt Ltd Full time

    Job Description:We are seeking a highly skilled Senior DevOps Engineer with expertise in Confluent Cloud and Kafka to join our dynamic team. The ideal candidate will have a strong background in Kubernetes cluster deployment, CI/CD pipelines with Jenkins, and extensive experience working with various database systems, including Google BigQuery, MySQL, XtraDB,...

  • Senior Middleware Engineer

    3 weeks ago

    Bengaluru, India PeopleStrong Full time

    Our Client :Our client is a product and services engineering company. We are a team of Software Engineers, Architects, Managers, and Cloud Experts with expertise in Technology and Delivery ManagementRoles and Responsibilities:- The role is to be a hands-on engineer to set up and manage messaging platforms such as Apache Kafka, Rabbit MQ, Redis, etc..- The...

  • Architect- Architect

    4 weeks ago

    Bengaluru, India Apexon Full time

    **About us** Apexon is a digital-first technology services firm backed by Goldman Sachs Asset Management and Everstone Capital. We specialize in accelerating business transformation and delivering human-centric digital experiences. For over 25 years, Apexon has been meeting customers wherever they are in the digital lifecycle and helping them outperform...

  • Java Solution Architect

    1 month ago

    Bengaluru, India Luxoft Full time

    **Project** Description**: We've been engaged by a large European Bank to provide resources to their Mobile banking migration team working on a wide range of projects like CTB changes, risk and regulatory projects, version upgrades etc. We require an experienced Java Solution Architect to work within the existing Team. **Responsibilities**: - Embedded in...

  • Technical Architect

    1 month ago

    Bengaluru, India Blue Yonder Full time

    Scope:Core responsibilities to include Architecting and designing (along with counterparts and distinguished Architects) for implementing commerce micro-servicesWhat you’ll do:Implementing Comemrce Microservices for a specific customerUnderstanding the Yantriks products and customization process Design and development of the custom components required for...

  • Kafka Administrator

    4 weeks ago

    Bengaluru, India Western Digital Full time

    Company Description At Western Digital, our vision is to power global innovation and push the boundaries of technology to make what you thought was once impossible, possible. At our core, Western Digital is a company of problem solvers. People achieve extraordinary things given the right technology. For decades, we’ve been doing just that. Our technology...