Principal AI Evaluation Engineer

2 weeks ago

New Delhi, India Backbase Full time

The Job in shortAs a a Principal AI Evaluation Engineer you will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and reporting, and make evaluation a cornerstone of release decisions.Meet the job● Define and lead the evaluation strategy and roadmap for AI-powered SDLC core product● Build and oversee evaluation pipelines and guardrails.● Build and maintain evaluation datasets (synthetic and real project data) to benchmark AI behavior.● Analyze evaluation results, identify gaps, and produce clear, actionable reports for engineering and product stakeholders.● Build a culture of innovation and excellence, encouraging continuous improvement and adoption of best practices in AI evaluation and deployment.● Collaborate with cross-functional teams to integrate evaluation insights into development.How about you● Strong understanding of software engineering principles and the software development lifecycle (SDLC).● Hands-on experience with test design, test management, observability, and data analysis.● Proficiency in Python (or another scripting language) for automating evaluations.● Familiarity with AI Agent evaluation methods (faithfulness, answer relevancy, contextual accuracy, tool correctness).● Excellent analytical and problem-solving skills.● Strong communication and collaboration abilities, able to work with cross-functional teams and stakeholders.

  • Principal QA Engineer – AI/ML

    4 days ago

    New Delhi, India CareerXperts Consulting Full time

    Job Opening: Principal Quality Assurance Engineer – AI/MLWe are looking for a visionary Principal QA Engineer to lead the end-to-end Quality Assurance strategy for advanced AI-driven applications. This role is ideal for someone who has deep experience in Agentic AI testing, strong technical leadership, and the ability to build scalable QA frameworks for...

  • Principal QA Engineer – AI/ML

    4 hours ago

    New Delhi, India CareerXperts Consulting Full time

    Job Opening: Principal Quality Assurance Engineer – AI/ML We are looking for a visionaryPrincipal QA Engineerto lead the end-to-end Quality Assurance strategy for advanced AI-driven applications. This role is ideal for someone who has deep experience inAgentic AI testing , strong technical leadership, and the ability to build scalable QA frameworks for...

  • AI Evaluator

    2 weeks ago

    Delhi, India Educatio Learning Inc. Full time

    Company Description In today’s fast-evolving educational landscape, true impact comes not just from knowledge but from the wisdom to apply it effectively and continuously improve through real-world experience. Educatio Learning helps enterprises operationalize the entire learning value chain—from data to insights, knowledge, and advanced AI...

  • Principal Engineer

    5 hours ago

    New Delhi, India Client of Solvecube Full time

    Hi,Greetings from Solvecube HCM I hope you are enjoying a great day! I am from Solvecube HCM, an AI Based global consulting firm head quartered in Singapore.Our client is a Healthcare AI Startup and looking for a great tech leader as their Principal Engineer for Bangalore. It is a permanent role with client.Your experience at Ambience Healthcare must have...

  • TinyML / Embedded AI Principal Engineer

    3 weeks ago

    New Delhi, India L&T Technology Services Full time

    Position Title: TinyML / Embedded AI Principal EngineerLocation: BangaloreExperience Required: 15+ YearsEmployment Type: Full-TimeRole Purpose:The TinyML / Embedded AI Principal Engineer will lead the design, development, optimization, and deployment of AI solutions on edge and resource-constrained embedded devices. This role is critical to enabling...

  • TinyML / Embedded AI Principal Engineer

    3 weeks ago

    New Delhi, India L&T Technology Services Full time

    Position Title:TinyML / Embedded AI Principal Engineer Location:Bangalore Experience Required:15+ Years Employment Type:Full-Time Role Purpose: The TinyML / Embedded AI Principal Engineer will lead the design, development, optimization, and deployment of AI solutions on edge and resource-constrained embedded devices. This role is critical to enabling...

  • AI Agent Engineer

    5 days ago

    New Delhi, India VoiceCare AI Full time

    Company DescriptionVoicecare AI is a Healthcare Administration General Intelligence company specializing in Revenue Cycle Management (RCM) and back-office automation. Our mission is to improve access, adherence, and outcomes for patients and the healthcare workforce through generative AI technology. VoiceCare AI operates with security and compliance at its...

  • AI Software Engineer

    5 days ago

    New Delhi, India Taskify AI Full time

    AI Software EngineerSummary We are seeking a talentedAI Software Engineerto join our engineering team and drive the design, development, and deployment of AI-powered solutions. This role is ideal for professionals passionate about artificial intelligence, machine learning, and software engineering who want to make a tangible impact on real-world...

  • Principal Engineer

    2 weeks ago

    New Delhi, India CoffeeBeans Full time

    About UsCoffeeBeans Consulting is a tech consulting firm focused on making organizations AI-ready by structuring their data efficiently across various sources and enabling AI-driven solutions. We specialize in data architecture, pipelines, governance, MLOps, and Gen AI solutions, helping clients achieve faster go-to-market and cost efficiency.We are looking...

  • Sr. AI Engineer

    2 weeks ago

    New Delhi, India AI Planet Full time

    |• Location: Hyderabad, India (Hybrid)• Experience: 4+ Years (Mandatory)• Employment Type: Full-Time• Department: AI Engineering / Product InnovationAt AI Planet, we’re building the future of human-like AI systems, from intelligent agents to cutting-edge generative tools that redefine how people create, collaborate, and innovate. Our team blends...