senior Ai LLM lead

Job Title: Sr AI LLM lead

Primary Skill: Transformer architectures, LLM fine-tuning, and instruction

Secondary Skill: RAG systems, embedding models, and retrieval

Experience: 6-9 yrs

Overview

The Principal LLM Engineer will lead the design, fine-tuning, and optimization of large language model (LLM) systems that capture, contextualize, and deliver organizational expertise at scale. This role focuses on turning unstructured and tacit knowledge into intelligent, context-aware reasoning systems that support diagnostics, decision-making, and automation across the enterprise.

Key Responsibilities

LLM Architecture s Optimization

Design and implement LLM-based reasoning frameworks for domain-specific knowledge activation and expert

Architect Retrieval-Augmented Generation (RAG) pipelines integrating structured, semi-structured, and unstructured enterprise knowledge

Fine-tune and optimize foundation models for specific enterprise use cases (diagnostics, troubleshooting, process guidance, ).

Develop prompt orchestration frameworks, including hierarchical prompts, context injection, and adaptive prompt

Evaluate new LLM modalities (multimodal, reasoning-augmented, tool-using models) for continuous system

Knowledge Engineering s Representation

Collaborate with knowledge engineers to structure and embed expert content from documents, logs, and SME interviews.

Develop semantic embedding strategies, vector stores, and hybrid retrieval methods combining symbolic and statistical

Create and maintain domain ontologies and entity graphs to ground model outputs in factual, validated enterprise

Ensure model interpretability through citation, provenance tracking, and source- grounded

Agentic s Multi-Model Integration

Integrate LLMs within multi-agent architectures, enabling autonomous reasoning and tool-assisted execution.

Work with Agentic AI architects to design LLMtool interaction schemas and context handoff mechanisms.

Build adaptive memory systems combining vector, relational, and graph storage

to support contextual continuity and learning loops.

Performance, Safety s Evaluation

• Establish evaluation pipelines for accuracy, coherence, grounding, and bias mitigation.

• Implement reinforcement learning from human feedback (RLHF) or in-context learning feedback loops.

• Design A/B testing frameworks for comparing model prompts, architectures, and retrieval

• Partner with governance and security teams to ensure compliance with data privacy and ethical AI standards.

Required Skills s Expertise

Core (LLM Engineering s Optimization)

• Deep expertise with transformer architectures, LLM fine-tuning, and instruction

Strong experience with RAG systems, embedding models, and retrieval

• Familiarity with multi-agent orchestration frameworks (AutoGen, Semantic Kernel, LangChain Agents).

Proficiency in Python, PyTorch, and distributed inference optimization.

• Understanding of prompt engineering, context window management, and token efficiency

Knowledge Systems s Data Foundations

Experience building and maintaining knowledge graphs, vector databases, and

context retrieval APIs.

Ability to structure domain-specific taxonomies and embeddings for technical and industrial use cases.

Familiarity with knowledge provenance, explainability, and citation generation mechanisms.

Evaluation s Experimentation

Expertise in designing model evaluation metrics, including factual accuracy, reasoning depth, and human preference modeling.

Strong analytical background for evaluating LLM performance across complex domain- specific tasks.

Preferred Qualifications

7+ years of experience in applied NLP, ML, or AI systems

2+ years hands-on with LLM fine-tuning, retrieval-augmented systems, or enterprise AI

Graduate degree in Computer Science, Machine Learning, Computational Linguistics, or related

Background in industrial, engineering, or diagnostics-related domains