LLM Ops EngineerOpenAI OR Anthropic OR Bedrock) AND
3 days ago
Role Overview :-
Monitor, evaluate, and optimize AI/LLM workflows in production environments. Ensure reliable, efficient, and high-quality AI system performance by building out an LLM Ops platform that is self-serve for the engineering and data science departments.
Key Responsibilities:-
- Collaborate with data scientists and software engineers to integrate an LLM Ops platform (Opik by CometML) for existing AI workflows
- Identify valuable performance metrics (accuracy, quality, etc) for AI workflows and create on-going sampling evaluation processes using the LLM Ops platform that alert when metrics drop below thresholds
- Cross-team collaboration to create datasets and benchmarks for new AI workflows
- Run experiments on datasets and optimize performance via model changes and prompt adjustments
- Debug and troubleshoot AI workflow issues
- Optimize inference costs and latency while maintaining accuracy and quality Develop automations for LLM Ops platform integration to empower data scientists and software engineers to self-serve integration with the AI workflows they build
Requirements:-
- Strong Python programming skills
- Experience with generative AI models and tools (OpenAI, Anthropic, Bedrock, etc)
- Knowledge of fundamental statistical concepts and tools in data science such as: heuristic and non-heuristic measurements in NLP (BLEU, WER, sentiment analysis, LLM-as-judge, etc), standard deviation, sampling rate, and a high level understanding of how modern AI models work (knowledge cutoffs, context windows, temperature, etc)
- Familiarity with AWS
- Understanding of prompt engineering concepts
- People skills: you will be expected to frequently collaborate with other teams to help to perfect their AI workflows
Experience Level 3-5 years of experience in LLM/AI Ops, MLOps, Data Science, or MLE
-
AWS, AWS Bedrock/ DN
5 days ago
Pune, Maharashtra, India Voicecord Full time ₹ 30,000 - ₹ 36,00,000 per yearJob Description: AWS, AWS Bedrock/ DNLocation: Pune Phase 2 / Hyderabad SEZ / Chandigarh SEZ.Key ResponsibilitiesArchitect and deliver AI/ML solutions leveraging AWS Bedrock, SageMaker, Lambda, API Gateway, and EKS.Design and implement RAG-based architectures, LLM integration, and vector database solutions for scalable AI workloads.Lead cloud deployments...
-
Principal LLM Engineer
3 weeks ago
Pune, Maharashtra, India, Maharashtra Rapid7 Full timePrincipal LLM EngineerJoin Rapid7: Secure the Future with AIAre you ready to lead the charge in integrating cutting-edge Large Language Models (LLMs) into world-class Cyber Security products?Rapid7 is looking for a Principal LLM Engineer with a rare combination of deep Data Science expertise, mastery of production MLOps, and 13+ years of experience. You...
-
AI Ops Engineer
2 weeks ago
Pune, Maharashtra, India Iitjobs Inc. Full time ₹ 15,00,000 - ₹ 25,00,000 per yearRequired Qualifications3 to 6 years of experience in software engineering, DevOps, or ML Ops with exposure to AI/LLM workflows.Strong foundation in prompt engineering and experience with LLMs like GPT, Claude,LLaMA, etc.Practical understanding of AIOps platforms or operational AI use cases (incident triage,log summarization, root cause analysis,...
-
AI/LLM Lead
3 days ago
Pune, Maharashtra, India SWAI TECHNOLOGIES PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per yearDescription : - Lead architecture, design, and implementation of LLM-based and agentic AI systems for clinical and operational use cases. - Oversee the development of multi-agent orchestration frameworks (reasoning, planning, and task execution) using tools such as LangGraph, CrewAI, or Semantic Kernel. - Build scalable RAG pipelines...
-
Lead Data Analyst AI
6 days ago
Pune, Maharashtra, India Inwinteck Full time ₹ 15,00,000 - ₹ 20,00,000 per yearResponsibilities:Lead data analysis & AI/ML projects using Amazon Bedrock. Apply LLMs, prompt engineering & AWS AI tools for insights.Req: 5+ yrs exp, SQL, Python/R. Preferred: GenAI frameworks, ETL, big data, MLOps.
-
Gen AI(React +Node)
7 days ago
Pune, Maharashtra, India Apex One Full time ₹ 15,00,000 - ₹ 25,00,000 per yearPosition:Gen AI Engineer (React + Node)Experience:6–8 YearsLocation:Pune – HybridResponsibilitiesDesign & deliver AI-first features (contract summarization, semantic search, guided workflows).Integrate LLMs (OpenAI, Anthropic, Mistral) using LangChain, LlamaIndex, or custom orchestration.Build RAG pipelines & embedding-based search with vector DBs...
-
Gen AI Engineer
7 days ago
Pune, Maharashtra, India Apex One Full time ₹ 15,00,000 - ₹ 25,00,000 per yearPosition: Gen AI Engineer (React + Node)Experience: 6-8 YearsLocation: Pune HybridResponsibilities: Design & deliver AI-first features (contract summarization, semantic search, guided workflows). Integrate LLMs (OpenAI, Anthropic, Mistral) using LangChain, LlamaIndex, or custom orchestration. Build RAG pipelines & embedding-based search with vector DBs...
-
CrewAI Engineer| Remote| Contractual
7 days ago
Pune, Maharashtra, India Iitjobs Inc. Full time ₹ 20,00,000 - ₹ 25,00,000 per yearSkills required:8+ years of experience in AI/ML or automation engineering.Strong hands-on experience with CrewAI or other LLM orchestration frameworks like LangChain, AutoGen, or Semantic Kernel.Proficiency in Python, including experience with async programming and API integration.Deep understanding of LLMs (OpenAI, Anthropic, Mistral, etc.) and prompt...
-
Gen AI Engineer
2 weeks ago
Pune, Maharashtra, India SID Global Solutions Full time ₹ 6,00,000 - ₹ 18,00,000 per yearJob Description: Generative AI EngineerLocation:PuneExperience: 1-4 yearsRole OverviewWe are looking for aGenerative AI Engineerwho will design, build, deploy, and maintain large language model (LLM)-based solutions. This engineer will work closely with architecture, data science, product, and operations teams to integrate GenAI capabilities into our...
-
Gen AI
6 days ago
Pune, Maharashtra, India Apex One Full time ₹ 15,00,000 per yearPosition: Gen AI Engineer (React + Node)Experience: 6–8 YearsLocation: Pune – HybridResponsibilities:Design & deliver AI-first features (contract summarization, semantic search, guided workflows).Integrate LLMs (OpenAI, Anthropic, Mistral) using LangChain, LlamaIndex, or custom orchestration.Build RAG pipelines & embedding-based search with vector DBs...