Senior LLMOps Engineer

5 days ago

Bengaluru, Karnataka, India Kong Inc. Full time ₹ 15,00,000 - ₹ 20,00,000 per year

Are you ready to power the World's connections?
If you don't think you meet all of the criteria below but are still interested in the job, please apply. Nobody checks every box - we're looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others.

About The Role
Kong's API Management suite including the Kong Enterprise AI Gateway and Konnect support multiple AI capabilities for customers.

The AI Gateway offers a unified API for managing multiple Large Language Models (LLMs), allowing users to switch between different AI providers seamlessly. The AI Gateway also allows a variety of AI policies to be enforced such as semantic caching, advanced routing and load balancing, promt guards and Personally Identifiable Information (PII) sanitization. The Konnect SaaS platform exposes these same AI capabilities via Cloud Gateways and provides customers a variety of AI-driven insights into the operations of a customer's API surface.

We are building out a new platform team - LLMOps - in our Bangalore office to help all the other engineering teams in Kong work with Large Language Models (LLMs) and build new features into our product portfolio.

What You'll Do

  • Be the founding member of the LLMOps team
  • Provide an LLM SaaS platform for all the engineering teams at Kong, giving them access to a variety of LLMs based on their needs
  • Manage model lifecycle with high reliability and uptime SLOs (at least 99.99%)
  • Fine tune the platform to provide AI security and high throughput for LLM queries
  • Own the end-to-end deployment architecture of the LLM SaaS platform
  • Play a pivotal role in shaping the technical direction for AI innovation in Kong
  • Collaborate closely with product leadership to define and refine strategy, roadmap, and objectives for the LLM SaaS platforms, ensuring alignment between engineering efforts and business goals.
  • Assist in the hiring process and help grow and mentor the team

What You'll Bring

  • Bachelor's or Master's degree in Computer Science or a related field
  • 5+ years of experience in building and operating highly reliable SaaS/PaaS systems
  • Prior ownership of uptime SLOs for a platform operating at 99.99% or higher
  • Hands-on experience with at least one of the major cloud platforms (AWS, Azure, or GCP)
  • Strong experience with AI platforms in public clouds (like Bedrock, Vertex AI, etc.).
  • Familiarity with observability tools such as Datadog, Prometheus, Grafana, Victoria Metrics, Loki, or similar technologies.
  • Expertise in designing and developing highly scalable distributed systems
  • Experience managing incidents and communicating effectively under high-pressure situations
  • Strong verbal and written communication skills to effectively collaborate across teams

About Kong
Kong Inc., a leading developer of cloud API technologies, is on a mission to enable companies around the world to become "API-first" and securely accelerate AI adoption. Kong helps organizations globally — from startups to Fortune 500 enterprises — unleash developer productivity, build securely, and accelerate time to market. For more information about Kong, please visit or follow us on X @thekonginc.

  • LLMOps Engineer

    5 days ago

    Bengaluru, Karnataka, India Tekskills Inc. Full time US$ 1,25,000 - US$ 1,75,000 per year

    Role: LLM Ops EngineerLocation:BengaluruExperience: 3+ YearsWork Mode: OnsiteEmployment Type: Full-timeAbout the Role:We are looking for a forward-thinking LLMOps Engineer to join our team and help build the next generation of secure, scalable, and responsible Generative AI (GenAI) platforms. This role will focus on establishing governance, security, and...

  • LLMOps Architect

    5 days ago

    Bengaluru, Karnataka, India Enterpret Full time US$ 1,50,000 - US$ 2,50,000 per year

    About EnterpretEnterpret is at the forefront ofAI-nativeapplications, unlocking the power of customer feedback for businesses. We centralize feedback from every channel and transform it into actionable insights that drive customer-centric decisions for teams at the world's leading companies like Perplexity, Notion, Canva, and Figma. Backed by investors such...

  • LLMOps Engineer

    3 days ago

    Bengaluru, Karnataka, India Uplers Full time

    Job DescriptionResponsibilities- Deploy and scale LLM inference workloads on Kubernetes (K8s) with 99.9% uptime.- Build agentic tools and services for fraud investigations with complex reasoning capabilities.- Work with Platform Engineers to set up monitoring and observability (e.g., Prometheus, Grafana) to track model performance and system health.-...

  • Machine Learning Engineer – LLMOps

    1 day ago

    Bengaluru, Karnataka, India Delphi Consulting Middle East Full time

    Ready to embark on a journey where your growth is intertwined with our commitment to making a positive impact? Join the Delphi family - where Growth Meets Values.At Delphi Consulting Pvt. Ltd., we foster a thriving environment with a hybrid work model that lets you prioritize what matters most. Interviews and onboarding are conducted virtually, reflecting...

  • senior hydraulic systems design expert

    1 week ago

    Bengaluru, Karnataka, India beBeehydraulic engineer Full time ₹ 12,00,000 - ₹ 20,00,000

    Job Title:">Senior Hydraulic Systems Design Expert"> ">Job Summary:"]}We are seeking an experienced Senior Hydraulic Systems Design Expert to join our team. As a key member of the engineering department, you will be responsible for designing, developing, and improving hydraulic systems, components, and handling ECNs and ECRs for power equipment and...

  • Sr. staff data engineer

    1 week ago

    Bengaluru, Karnataka, India Aurigo Software Technologies Full time

    Senior Staff Data Engineer (Agentic AI Engineer)Location: Bengaluru, IndiaExperience: 5+ years Role Summary: Aurigo is leading the charge in transforming enterprise Saa S for infrastructure, and AI is at the core of this revolution. With Aurigo Lumina, our AI copilot, we are redefining how infrastructure owners interact with Masterworks, driving smarter,...

  • Sr Staff Data Engineer

    3 days ago

    Bengaluru, Karnataka, India Aurigo Software Technologies Full time

    Senior Staff Data Engineer (Agentic AI Engineer) Location: Experience: 5+ years Aurigo is leading the charge in transforming enterprise SaaS for infrastructure, and AI is at the core of this revolution. With Aurigo Lumina, our AI copilot, we are redefining how infrastructure owners interact with Masterworks, driving smarter, faster, and more efficient...

  • Senior Devops Engineer

    5 days ago

    Bengaluru, Karnataka, India Techno Facts Solutions Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Title: Senior / Lead DevOps Engineer (GenAI)Level: A4 & A5 (Experience: 5.1 9 Years)Key ResponsibilitiesDesign, build, and maintain CI/CD pipelines for GenAI model training and deployment.Automate infrastructure provisioning & scaling using Terraform, Ansible, Pulumi.Optimize GPU/TPU utilization and monitor model performance in production.Integrate GenAI...

  • Advanced AI Architect

    1 week ago

    Bengaluru, Karnataka, India beBeeGenAi Full time ₹ 1,50,00,000 - ₹ 2,25,00,000

    Artificial Intelligence Developer PositionThis senior-level role involves designing and developing advanced artificial intelligence applications. The ideal candidate will possess a deep understanding of cutting-edge technologies, including generative AI, natural language processing, and computer vision.Key Responsibilities:Implementing Gen AI models using...

  • AI - Senior Engineer

    5 days ago

    Bengaluru, Karnataka, India GEETHA TECHNOLOGY SOLUTIONS PRIVATE LIMITED Full time US$ 1,50,000 - US$ 2,00,000 per year

    Bachelors or Masters degree in Computer Science, AIML/Data Science, Statistics or a related field only.Preferred Institution: NIT (Any)Responsibilities:Architect and deploy scalable Large language models (LLM) and transformer-based architectures.Implement MLOps/LLMOps best practices for CI/CD, automated model retraining, and lifecycle management.Develop...