Senior Java Engineer – Model Inference Platform

3 weeks ago

Chennai Tamil Nadu India, Tamil Nadu Tata Consultancy Services Full time

Key Responsibilities


Design, build, and optimize high-performance microservices using Java 17+, Spring Boot, and reactive frameworks.

Develop and maintain APIs for model registration, inference request routing, and model version management.

Integrate with Seldon Core / KServe for model orchestration, deployment, A/B testing and shadow testing of ML models.

Implement asynchronous, event-driven architectures using Kafka, Pub/Sub, or RabbitMQ for high-throughput inference workloads.

Build monitoring, tracing, and observability into inference pipelines using Prometheus, Grafana, and OpenTelemetry.

Work closely with MLOps and data engineering teams to integrate CI/CD for ML models, manage containers, and deploy to Kubernetes / cloud-native platforms (GCP GKE, AWS EKS, or Azure AKS).

Collaborate with data scientists and architects to ensure efficient model packaging, feature ingestion, and inference scaling.

Apply performance tuning, caching strategies (Redis, Hazelcast), and load balancing to sustain millisecond-level response times.

Mentor engineers and participate in design reviews to ensure architectural consistency, quality, and scalability.



Required Qualifications


12+ years of backend engineering experience with Java HPC (17 or above) in large-scale, distributed systems.

Deep understanding of microservices, event-driven architectures, and reactive programming.

Proven expertise in Spring Boot, Kafka, REST/gRPC APIs, JPA/Hibernate, Redis, and SQL/NoSQL databases.

Strong understanding of containerization (Docker) and Kubernetes orchestration.

Hands-on experience integrating or developing AI/ML inference platforms such as Seldon, KServe, TensorFlow Serving, or TorchServe.

Familiarity with model lifecycle management, feature stores, and inference request optimization.

Experience with cloud environments (AWS, GCP, or Azure) and CI/CD pipelines (Jenkins, ArgoCD, GitLab CI).

Knowledge of observability and performance tuning in high-concurrency, low-latency applications.

Strong communication, analytical, and problem-solving skills with the ability to lead technical discussions across teams.



Preferred Skills


Exposure to Python-based ML pipelines and REST/gRPC integrations with data science environments.

Familiarity with model governance, A/B testing, and multi-model serving architectures.

Experience with API gateways (Kong, Istio, or Envoy) and service mesh architectures.

Understanding of data serialization formats (Avro, Protobuf, Parquet) and streaming analytics.

Knowledge of GPU/CPU optimization for inference workloads

Exp Range: 10 TO 15

Location: Pan India

Interview Type: Weekday Virtual Drive

Date: 16-Oct-2025

Day: Thursday

  • Technical Project Manager

    4 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    We are seeking a Senior Technical Project Manager with a strong hands-on background in Java, Python, and Generative AI to lead the development of cutting-edge AI-driven platforms and products. This role requires both strategic project leadership and deep technical expertise—someone who can manage teams, drive execution, and contribute directly to the...

  • Senior Infrastructure Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Citi Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    We are seeking an experienced Senior Backend Engineer to design, develop, and maintain the infrastructure powering our generative AI applications. You will work closely with AI engineers, platform teams, and product stakeholders to build scalable, reliable backend systems that support AI model deployment, inference, and integration. This role combines...

  • Senior Java Engineer

    1 day ago

    Chennai, Tamil Nadu, India TVS Supply Chain Solutions Full time ₹ 15,00,000 - ₹ 30,00,000 per year

    Company DescriptionWhite Data Systems India Private Limited (WDSI), incorporated in April 2015, focuses on improving reliability and service quality for the road transportation industry through its i-Loads platform. As a subsidiary of TVS Logistics and backed by Cholamandalam Investment and Finance Company Limited, WDSI offers integrated solutions that...

  • Senior Java Software Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu Fiserv Full time

    Fiserv is a Fortune 500 company and a global leader in Fintech and payment solutions. We’re always innovating in ways to connect financial institutions, corporations, merchants, small businesses, and consumers to each other – quickly, reliably, and securely. With more than 6 million merchant locations across the world, over 1.4 billion global accounts on...

  • Senior Java Developer

    3 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu InfoBeans Full time

    About the JobJob Title: Senior Java Developer – Microservices (DM-Suite)Skills: Java, Spring Boot, Microservices, SQL (MySQL), JUnit, Test Automation (Selenium/Playwright), Agile/ScrumExperience Required: 5+ Years Location: ChennaiNotice Period: Immediate to 30 days preferredJob DescriptionRole Overview:We are looking for an experienced Senior Java...

  • Senior Java Developer

    4 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    TCS present an excellent opportunity for Senior Java DeveloperJob Location: ChennaiExperience required : 6-10 YrsInterview Date : 27-Sep-25 (Saturday)Experience developing microservices and cloud native apps using Java/J2EE, REST APIs, Spring Core, Spring MVC Framework, Spring Boot Framework JPA (Java Persistence API) (Or any other ORM), Spring Security and...

  • MLOps Engineer || Bangalore/Chennai/Noida/Hyderabad

    2 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu HCLTech Full time

    HCLTech is hiring MLOps Engineer Job Title: ML Ops Engineer / ML Engineer Job Overview: We are looking for an experienced MLOps Engineer to help deploy, scale, and manage machine learning models in production environments. You will work closely with data scientists and engineering teams to automate the machine learning lifecycle, optimize model performance,...

  • Platform Engineering Senior Engineer

    1 week ago

    Chennai, Tamil Nadu, India TekWissen Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Overview:TekWissen is a global workforce management provider throughout India and many other countries in the world. The below clientis a global company with shared ideals and a deep sense of family. From our earliest days as a pioneer of modern transportation, we have sought to make the world a better place one that benefits lives, communities and the...

  • Senior Software Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Platform Science Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Who We AreAt Platform Science, we're working to connect everything that moves. Founded in 2015, we are an open IoT platform that partners with innovative fleets, application developers, vehicle manufacturers, and equipment providers in the transportation industry to deliver revolutionary solutions to supply chain professionals across the globe.Our employees...

  • Senior Java Software Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu Avensys Consulting Full time

    Avensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success, we have evolved to become one of the top trusted providers in Singapore and service a client base across...