Kafka Architect

1 month ago


Hyderabad, Telangana, India Health Catalyst Full time
About Us

Health Catalyst is a leading healthcare technology company that is revolutionizing the industry with innovative solutions. We are a fast-growing organization that values smart, hardworking, and humble individuals who are passionate about making a difference in the lives of millions of people.

Job Summary

We are seeking a highly skilled Senior Software Engineer to join our team in designing, implementing, and maintaining complex Kafka pipelines in an AWS environment. The ideal candidate will have extensive experience in distributed systems, data engineering, and team leadership.

Key Responsibilities:
  • Lead the design and architecture of Kafka pipelines using AWS MSK and Kafka Connect
  • Develop advanced Kafka Sink Connectors for various data stores, including RDS, DocumentDB, S3, OpenSearch, and potentially AWS KeySpaces and ScyllaDB
  • Create complex Single Message Transformations (SMTs) for sophisticated ETL operations
  • Design and implement robust error handling and fault tolerance mechanisms, including DLQ systems
  • Optimize Kafka cluster performance and develop advanced monitoring solutions using Prometheus and Grafana
  • Implement best practices for logging and observability using AWS CloudWatch
  • Develop comprehensive automation solutions using shell scripting and potentially other technologies
  • Provide technical leadership and mentorship to team members
  • Collaborate with other teams to design and implement Kafka Producers in Java
Requirements:
  • Bachelor's or Master's degree in Computer Science, Engineering, or related field
  • 5+ years of extensive experience working with Apache Kafka and its ecosystem
  • Strong Java programming skills with experience in developing Kafka Producers
  • Deep understanding of AWS services, particularly MSK, EC2, CloudWatch, and other relevant data services
  • Expert-level knowledge of Kafka Connect and developing custom connectors
  • Proven experience in designing and implementing complex distributed systems
  • Advanced skills in performance tuning and optimization of Kafka clusters
  • Expertise in monitoring and observability tools such as Prometheus and Grafana
  • Strong scripting skills (shell, Python, etc.) for developing sophisticated automation solutions
  • Excellent problem-solving, communication, and leadership skills
Preferred Qualifications:
  • Experience with additional AWS services like AWS KeySpaces and Aurora
  • Knowledge of emerging technologies in the data streaming space
  • Contributions to open-source Kafka projects or connectors
  • Experience with Kubernetes and containerization of Kafka ecosystems
  • Familiarity with real-time data processing frameworks (e.g., Apache Flink, Spark Streaming)

  • Kafka Architect

    2 weeks ago


    Hyderabad, Telangana, India Alignity Solutions Full time

    About the RoleWe are looking for a highly skilled Kafka Architect to lead the design, development, and deployment of our Kafka-based data pipeline. As a Kafka Architect, you will be responsible for designing and implementing scalable, fault-tolerant, and high-performance Kafka clusters.Key Responsibilities:Kafka Cluster Design: Design and implement scalable...


  • Hyderabad, Telangana, India DSV Full time

    About the Role:Kafka Systems ArchitectThe Kafka Systems Architect will be responsible for designing and implementing event-driven architectures using Apache Kafka. This includes developing and maintaining Kafka clusters, ensuring high availability, and troubleshooting issues.Key Responsibilities:Design and implement event-driven architectures using Apache...


  • Hyderabad, Telangana, India Tata Consultancy Services Full time

    Tata Consultancy Services is seeking a skilled Kafka Systems Architect to join our team.Key Responsibilities:Design and implement event-driven architecture using KafkaDevelop and maintain high-performance Kafka clustersCollaborate with cross-functional teams to integrate Kafka with other systemsRequirements:5-8 years of experience in designing and...

  • Kafka Architect Lead

    3 weeks ago


    Hyderabad, Telangana, India Qode Full time

    Kafka Architect Lead PositionAt Qode, we are looking for a seasoned expert in designing and implementing scalable Kafka architectures. The ideal candidate will have extensive experience with Kafka Streams, KSQL, and the associated clustering model. Additionally, they should possess solid programming skills in Java and a deep understanding of best practices...

  • Kafka Architect

    1 month ago


    Hyderabad, Telangana, India ScaleneWorks Full time

    ScaleneWorks - Confluent Kafka DeveloperJob Summary: We are seeking a highly skilled Confluent Kafka Developer to join our team at ScaleneWorks. The ideal candidate will have a strong background in developing high-performance data pipelines and streaming applications using Confluent Kafka.Key Responsibilities:Design, develop, and deploy Confluent Kafka...


  • Hyderabad, Telangana, India Virtusa Full time

    Job Description:As a Senior Java Architect at Virtusa, you will be responsible for designing and implementing scalable and efficient microservices architecture using Java. You will work closely with the development team to ensure seamless integration with Kafka and other systems.Key Responsibilities:Design and implement microservices architecture using...


  • Hyderabad, Telangana, India ProArch Full time

    Job DescriptionWe are seeking a highly skilled Sr. Kafka Developer to join our team at ProArch. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining scalable and efficient Kafka-based messaging solutions and microservices applications.Key Responsibilities:Configure and manage Kafka clusters, ensuring...


  • Hyderabad, Telangana, India Qode Full time

    Key ResponsibilitiesWe are seeking a highly skilled Java-Kafka Lead to join our team at Qode. The ideal candidate will have a strong background in Kafka Streams, KSQL, and Java, with a proven track record of designing and implementing scalable and efficient Kafka ecosystems.Technical RequirementsMinimum 8 years of experience in software development, with a...


  • Hyderabad, Telangana, India ProArch Full time

    ProArch is seeking a skilled Senior Kafka Developer with AWS experience to join our innovative team. In this role, you will be responsible for designing, developing, and managing Kafka-based solutions that improve our data streaming capabilities and support our cloud infrastructure on AWS.Key Responsibilities:Design, develop, and implement Kafka streams and...


  • Hyderabad, Telangana, India ProArch Full time

    OverviewProArch is seeking a skilled Senior Kafka Developer with AWS experience to join our innovative team.Key Responsibilities:Design, develop, and implement Kafka streams and data pipelines to facilitate real-time data processing and analytics.Integrate Kafka with various data sources and sinks, ensuring data flow across different applications.Work...


  • Hyderabad, Telangana, India ProArch Full time

    Data Streaming Solutions Engineer - KafkaProArch is seeking a skilled Senior Kafka Developer with AWS experience to join our innovative team in the field of cloud data streaming solutions.Key Responsibilities:Design, develop, and implement Kafka streams and data pipelines to facilitate real-time data processing and analytics for our cloud infrastructure on...


  • Hyderabad, Telangana, India Riverforest Connections Private Limited Full time

    Job Title: Senior Software ArchitectWe are seeking a skilled Senior Software Architect to lead the migration of existing Kafka infrastructure to Amazon MSK. The ideal candidate will have strong expertise in Kafka core Java and migration projects with additional experience in Apache Storm being advantageous.Key Responsibilities:Lead the migration of existing...

  • Pega Architect

    1 month ago


    Hyderabad, Telangana, India Thryve Digital Health LLP Full time

    Job Title: Pega ArchitectAt Thryve Digital Health LLP, we are seeking a highly skilled Pega Architect to join our team. As a Pega Architect, you will play a key role in providing complex application development services and leading project teams in various project roles.Key Responsibilities:Analyze, design, code, test, and implement complex application...

  • Software Architect

    3 weeks ago


    Hyderabad, Telangana, India Virtusa Full time

    **Job Summary**We are seeking a highly skilled Java expert to join our team as a Senior Java Architect - Kafka Expert. The ideal candidate will have extensive experience in Java and will be able to design and implement Kafka solutions.Key Responsibilities:Design and implement Kafka solutions for large-scale applications.Develop and maintain Java applications...


  • Hyderabad, Telangana, India ConsultBae India Private limited Full time

    Job Title: Java Backend ArchitectAbout the Role:We are seeking a highly skilled Java Backend Architect to join our team at ConsultBae India Private Limited. As a Java Backend Architect, you will be responsible for designing and architecting microservices using J2EE technology.Key Responsibilities:Design and architect microservices using J2EE...


  • Hyderabad, Telangana, India Ingrain Systems Inc Full time

    At Ingrain Systems Inc, we seek a highly skilled Senior Java Software Architect to spearhead the design and development of our complex software applications. The ideal candidate will have a strong background in Java, J2EE, and Spring Boot, with expertise in architecting and implementing Microservices-based solutions.Key Responsibilities:• Design, develop,...


  • Hyderabad, Telangana, India travash Full time

    Cloud Infrastructure Solutions ArchitectWe are looking for a skilled Cloud Infrastructure Solutions Architect to join our team at Travash. The ideal candidate will have a strong understanding of cloud and infrastructure components, including server, storage, network, data, and applications, to deliver end-to-end cloud infrastructure architectures and...


  • Hyderabad, Telangana, India DBS Bank Full time

    About the RoleAs part of Group Technology and Operations, we are looking for a Senior Enterprise Data Solutions Architect to join our team. The successful candidate will be responsible for designing and implementing highly resilient data infrastructure solutions using Apache Kafka and Confluent Kafka.Key Responsibilities• Design and implement data...


  • Hyderabad, Telangana, India Experian Full time

    About this roleWe are seeking a highly skilled Senior Messaging Architect to join our team at Experian. As a key member of our engineering team, you will be responsible for designing, developing, and implementing the Kafka ecosystem by creating a framework for leveraging technologies such as Kafka Connect, KStreams/KSQL, Attunity, Schema Registry, and other...


  • Hyderabad, Telangana, India Tata Consultancy Services Full time

    Job Title: Java Microservices ArchitectAbout the Role:We are seeking a highly skilled Java Microservices Architect to join our team at Tata Consultancy Services. As a Java Microservices Architect, you will be responsible for designing and building microservices using Springboot and Java 8.Key Responsibilities:Design and build microservices using Springboot...