AI Evals

3 weeks ago

New Delhi, India BharatGen Full time

Job Summary:We are looking for an AI Evaluation & Test Engineer to join our growing team to ensure that our generative AI models and applications are safe, accurate, trustworthy, and deliver an elegant user experience. You will serve as the first customer of our AI systems. This role is ideal for product-minded engineers who obsess over product quality and customer-centricity, and are passionate about shaping the behavior of AI systems in the real world.Key Responsibilities:- Build and maintain AI evaluation pipelines to test, measure, and evaluate the behavior and performance of AI systems. - Implement traces, spans, and session tracking for observability and identify error propagation in multi-step pipelines. - Define AI quality metrics and KPIs around factuality, faithfulness, toxicity, grounding precision/recall, latency, cost, etc., with clear acceptance bars. - Implement evaluation and testing automation to enable end-to-end system and regression testing at scale. - Define criteria for and implement release gates in the CI/CD pipeline. - Find creative ways to break products. - Assist in root cause analysis and troubleshooting of bugs and field issues. - Collaborate with cross-functional teammates from product, engineering, linguistics, and customer support to shape human-AI interaction paradigms and ensure that our AI models and applications deliver the desired outcome and user experience.Minimum Qualifications and Experience:- Bachelor’s or Master’s degree in CS/CE/IT/EE/E&TC or related fields with 5+ years of experience in manual and automation testing of software products, with at least 2 years in evaluating and testing AI/ML products.Required Expertise:- Strong software testing fundamentals and expertise in writing test plans, executing test cases, and generating detailed reports and dashboards. - Strong analytical and debugging skills, and attention to detail. - Proficiency in Python, scripting, and software testing automation frameworks and tools such as Pytest, Selenium, Robot Framework, etc. - Working knowledge of generative AI models, AI agents, and related concepts such as retrieval augmented generation (RAG), prompt engineering, context engineering, explainability, traceability, observability, guard rails, reasoning, specificity, etc. - Sound understanding of the fundamental differences in the approach for testing conventional software versus evaluating generative AI systems. - Team player with excellent interpersonal skills and the ability to collaborate effectively with remote and cross-functional team members. - Go-getter attitude and ability to flourish in a fast-paced, startup environment. - Experience in any of the following would be a big plus -- AI evaluation frameworks such as Arize, Braintrust, DeepEval, LangSmith, Ragas- AI safety and red teaming experience, e.g., prompt injection, jailbreak, adversarial and stress testing.- Different types of AI evaluation methods, e.g, Human-in-the-loop, LLM-as-a-Judge.

  • Senior AI Developer

    5 days ago

    New Delhi, India SoluLab Full time

    Founding AI Engineer (Senior) About the Role We’re a stealth AI startup in Ahmedabad. As Founding AI Engineer, you will design and ship LLM-powered agents and RAG pipelines with rigorous evals, safety, and reliability, then shape the platform architecture across BE/FE and help hire the AI team.Read this first: This is a startup role. In intense weeks,...

  • Senior AI Developer

    1 day ago

    New Delhi, India SoluLab Full time

    Founding AI Engineer (Senior)About the Role We’re a stealth AI startup in Ahmedabad. As Founding AI Engineer, you will design and ship LLM-powered agents and RAG pipelines with rigorous evals, safety, and reliability, then shape the platform architecture across BE/FE and help hire the AI team.Read this first: This is a startup role. In intense weeks,...

  • AI Jewelry Designer

    4 weeks ago

    New Delhi, India Bez - The AI Copilot for Jewellers Full time

    About BezBez builds domain-specific AI for jewellery businesses. Our agents learn a brand’s unique style, then generate original designs, manufacturing models, and marketing visuals in minutes—shrinking the product lifecycle by 5×. We work with some of the world's largest jewellery manufacturers and are backed by LeapYear & Macroscopic Ventures.Why This...

  • Senior AI Developer

    6 days ago

    Delhi, India SoluLab Full time

    Founding AI Engineer (Senior)About the RoleWe’re a stealth AI startup in Ahmedabad. As Founding AI Engineer, you will design and ship LLM-powered agents and RAG pipelines with rigorous evals, safety, and reliability, then shape the platform architecture across BE/FE and help hire the AI team.Read this first: This is a startup role. In intense weeks, expect...

  • AI Engineer

    2 days ago

    Delhi, India Ravian AI Full time

    About Ravian AIRavian AI is building device-native AI systems that can think, decide, and act on behalf of users. Our platform goes beyond web-based agents, enabling true end-to-end automation directly on devices. We’re working with enterprises and consumers to unlock productivity and decision intelligence at scale.The RoleYou will design and build...

  • AI Engineer

    16 hours ago

    Delhi, India Ravian AI Full time

    About Ravian AIRavian AI is building device-native AI systems that can think, decide, and act on behalf of users. Our platform goes beyond web-based agents, enabling true end-to-end automation directly on devices. We’re working with enterprises and consumers to unlock productivity and decision intelligence at scale.The RoleYou will design and build...

  • AI Engineer Intern

    3 days ago

    New Delhi, India Rivi Full time

    About RiviWe build AI-first products across travel and beyond. We’re hiring a full-time paid AI intern (at least 3 to 6 months) to help train, tune, and ship AI systems that power real user experiences.What you’ll do- Build Python microservices that wrap LLM/RAG pipelines—vector search, embeddings, grounding—behind REST/gRPC. - Train + fine-tune...

  • Ai Jewelry Designer

    4 weeks ago

    New Delhi, India Whatjobs IN C2 Full time

    About Bez Bez builds domain-specific AI for jewellery businesses. Our agents learn a brand’s unique style, then generate original designs, manufacturing models, and marketing visuals in minutes—shrinking the product lifecycle by 5×. We work with some of the world's largest jewellery manufacturers and are backed by LeapYear & Macroscopic Ventures. Why...

  • Senior/Staff Software Engineer

    3 days ago

    New Delhi, India Processity Full time

    Senior/Staff Software Engineer - AI‑AcceleratedSalary: 40 LPALocation: Hybrid(Chennai)Belief: We believe modern AI tools exponentially compound the output of the very best engineers. Our bar is intentionally high: we hire exceptional builders and give them an AI multiplier.The RoleOwn end‑to‑end delivery of production systems in an AI‑first...

  • Backend + AI Engineer

    3 days ago

    New Delhi, India Rivi Full time

    About RiviWe build AI-first products across travel and beyond. We’re looking for a backend-builder passionate about scalable APIs, microservices, databases, and LLM integrations to power seamless, high-performance AI tools for our customers.What you’ll do- Design, build, and version REST + gRPC microservices in Python / Node.js / TypeScript. - Model and...