Kafka engineers
1 month ago
Experience 10+ yrs.
Location: Hyderabad
Scope
The scope of work involves the finalization and optimization of the Kafka Streams solution for the IEE job, focusing on achieving production-level performance, scalability, observability, and cost efficiency. The key objectives are outlined as follows:
Cost Optimization
Estimate and benchmark the current cost.
Prioritize iterative performance improvements with a focus on cost efficiency.
Functional Enhancements
Ensure functional parity with the main branch.
Upgrade library versions, including Java, Spring Boot, Spring, Kafka Streams, etc.
Eliminate Hibernate dependencies.
Remove unnecessary libraries and redundant functionalities.
Integration and Optimization
Optimize boot-up time, including multi-threaded loading of data sources (e.g., whitelists, lookups).
Continuously identify and resolve performance bottlenecks.
Logging and Metrics
Implement detailed logging and metrics collection for monitoring and analysis.
Production Readiness
Prepare for production deployment by addressing the following areas:
Containerization of the solution
Development of runbooks
Creation of monitoring dashboards and alert mechanisms
Deployment scripts for streamlined rollouts
Auto-scalable container orchestration
Cluster sizing recommendation tool
Auto-healing mechanisms for handling: slow or erroneous policies and invalid events
Quality Assurance
Develop comprehensive automated tests, including: Unit tests, integration tests with data mocking and policy types with violation handling
Execute extensive unit tests, Sonar scans, and security scans.
Implement CI/CD pipelines for continuous deployment to Dev and Performance environments based on quality gate criteria.
Documentation
Create comprehensive documentation, including: Design documents, Code walkthroughs, Runbooks for operational guidance
Deliverables & Milestones
Optimized the (Kafka Streams) codebase with production-level performance enhancements, implemented monitoring and alerting and AWS resources, and provided comprehensive documentation for production deployment, including architecture details and operational procedures like runbooks. Additionally, conducted end-to-end testing with recommendations for any further improvements. The project will be executed in phases with the following key milestones:
Weekly Progress Reports: Regular updates on the project status and achievements.
Bi-Weekly Demos: Demonstrations every two weeks to showcase progress and gather feedback.
Ongoing Cost Optimization Monitoring: Continuous visibility into the current state of the Cost Optimization estimates
Functional Testing: Conduct parallel testing for functional parity with the existing Spark-based IEE job on a selected tenant within the first two months.
Multi-Tenant Testing: Perform multiple parallel tests across different tenants prior to final sign-off
Why Join Us?
At Critical River, you’ll have the opportunity to lead and shape a key practice within our organization, work with high-caliber clients, and make a meaningful impact on their digital transformation journey. We are committed to fostering a culture of innovation, growth, and collaboration where your ideas and leadership will thrive.
Critical River Inc. is an employee-owned technology consulting and IT services firm that empowers its workforce to have a direct stake in the company's growth and success. This ownership model fosters a collaborative and committed culture where employees are motivated to deliver high-quality solutions and services for clients. By giving its team members a shared interest in the company's achievements, Critical River enhances innovation, accountability, and a strong sense of community across all levels of the organization.
-
Kafka Engineer
2 months ago
Hyderabad, India Health Catalyst Full timeAbout UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...
-
Kafka Engineer
3 weeks ago
Hyderabad, India Health Catalyst Full timeAbout UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...
-
Kafka Engineer
3 weeks ago
Hyderabad, India Health Catalyst Full timeAbout UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...
-
Kafka engineer
3 months ago
Hyderabad, India Health Catalyst Full timeAbout Us The healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends....
-
Kafka Engineer
3 months ago
Hyderabad, India Health Catalyst Full timeAbout Us The healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends....
-
Kafka engineer
4 weeks ago
Hyderabad, India Health Catalyst Full timeAbout UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...
-
Kafka Engineer
2 months ago
hyderabad, India Health Catalyst Full timeAbout Us The healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends....
-
Health Catalyst | Kafka Engineer
3 weeks ago
hyderabad, India Health Catalyst Full timeAbout UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...
-
Kafka Architecture Engineer
4 days ago
Hyderabad, Telangana, India Health Catalyst Full timeAbout Health CatalystWe are a fast-growing company that values smart, hardworking, and humble individuals. Each product team is a small, mission-critical team focused on developing innovative tools to support our mission to improve healthcare performance, cost, and quality.Job Summary/ResponsibilitiesWe're looking for a Senior Software Development Engineer...
-
Kafka Architect
2 days ago
Hyderabad, India ProArch Full time**Responsibilities**: - Design new solutions and validate new ideas by either building on top of existing solutions or by prototyping new platform capabilities - Take ownership for the quality of design and implementation of software components. - Interact with various non-engineering teams such as Product Management, SRE, Risk Analysts and senior...
-
Kafka Developer
4 months ago
Hyderabad, India Health Catalyst Full timeAbout UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...
-
Kafka Developer
4 months ago
Hyderabad, India Health Catalyst Full timeAbout UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...
-
Kafka Developer
4 months ago
Hyderabad, India Health Catalyst Full timeAbout Us The healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends....
-
Apache Kafka Engineer Lead
3 weeks ago
Hyderabad, Telangana, India Criticalriver Technologies Full timeAbout Distributed Streaming Systems Expert Role at CriticalRiver Technologies: We are seeking an experienced Distributed Streaming Systems Expert to lead our efforts in designing, implementing, and maintaining distributed streaming platforms using Apache Kafka. The ideal candidate will have expertise in building real-time data streaming pipelines and...
-
Health Catalyst | Kafka Engineer | hyderabad
1 month ago
hyderabad, India Health Catalyst Full timeAbout UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...
-
Health Catalyst | Kafka Engineer | hyderabad
4 weeks ago
Hyderabad, India Health Catalyst Full timeAbout UsThe healthcare industry is the next great frontier of opportunity for software development, and Health Catalyst is one of the most dynamic and influential companies in this space. We are working on solving national-level healthcare problems, and this is your chance to improve the lives of millions of people, including your family and friends. Health...
-
Kafka Solutions Architect
3 weeks ago
Hyderabad, Telangana, India Criticalriver Technologies Full timeAbout CriticalRiver Technologies:CriticalRiver Technologies is a technology consulting and IT services firm empowering its workforce to have a direct stake in the company's growth and success. We are committed to fostering a culture of innovation, growth, and collaboration where your ideas and leadership will thrive.Job Overview:The estimated salary for this...
-
Kafka Engineers
1 month ago
Hyderabad, India CriticalRiver Inc. Full timeJob Title:Kafka LeadExperience10+ yrs.Location:HyderabadScopeThe scope of work involves the finalization and optimization of the Kafka Streams solution for the IEE job, focusing on achieving production-level performance, scalability, observability, and cost efficiency. The key objectives are outlined as follows:Cost OptimizationEstimate and benchmark the...
-
Kafka engineers
2 weeks ago
Hyderabad, India CriticalRiver Inc. Full timeJob Title: Kafka Lead Experience 10+ yrs. Location: Hyderabad Scope The scope of work involves the finalization and optimization of the Kafka Streams solution for the IEE job, focusing on achieving production-level performance, scalability, observability, and cost efficiency. The key objectives are outlined as follows: Cost Optimization ...
-
Kafka Engineers
1 month ago
Hyderabad, India CriticalRiver Inc. Full timeJob Title: Kafka LeadExperience 10+ yrs.Location: HyderabadScope The scope of work involves the finalization and optimization of the Kafka Streams solution for the IEE job, focusing on achieving production-level performance, scalability, observability, and cost efficiency. The key objectives are outlined as follows: Cost Optimization Estimate and...
