AI Evaluation Technical Lead
1 week ago
The Job in shortAs a a Principal AI Evaluation Engineer you will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and reporting, and make evaluation a cornerstone of release decisions. Meet the job● Define and lead the evaluation strategy and roadmap for AI-powered SDLC core product● Build and oversee evaluation pipelines and guardrails.● Build and maintain evaluation datasets (synthetic and real project data) to benchmark AI behavior.● Analyze evaluation results, identify gaps, and produce clear, actionable reports for engineering and product stakeholders.● Build a culture of innovation and excellence, encouraging continuous improvement and adoption of best practices in AI evaluation and deployment.● Collaborate with cross-functional teams to integrate evaluation insights into development. How about you● Strong understanding of software engineering principles and the software development lifecycle (SDLC).● Hands-on experience with test design, test management, observability, and data analysis.● Proficiency in Python (or another scripting language) for automating evaluations.● Familiarity with AI Agent evaluation methods (faithfulness, answer relevancy, contextual accuracy, tool correctness).● Excellent analytical and problem-solving skills.● Strong communication and collaboration abilities, able to work with cross-functional teams and stakeholders.
-
Principal AI Evaluation Engineer
6 days ago
hyderabad, India Backbase Full timeAbout BackbaseAs a a Principal AI Evaluation Engineeryou will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and reporting,...
-
Principal AI Evaluation Engineer
2 days ago
Hyderabad, Telangana, India Backbase Full time ₹ 12,00,000 - ₹ 24,00,000 per yearAbout BackbaseAs a a Principal AI Evaluation Engineeryou will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and reporting,...
-
AI and ML Engineer
2 weeks ago
Hyderabad, Telangana, India Lead Masters AI Full time ₹ 9,00,000 - ₹ 12,00,000 per yearInternship Opportunity – LeadMasters AI (Dubai-Registered AI Company)About LeadMasters AILeadMasters AI is a Dubai-registered Company operating out of Hyderabad. We build solutions in AI-driven marketing automation, social media management, CRM workflows, and intelligent lead generation. Our mission is to help businesses scale with agentic AI-powered...
-
Lead AI Engineer
6 days ago
hyderabad, India Weekday AI Full timeThis role is for Weekday's client.Role OverviewAs the Lead AI Engineer, you will be responsible for spearheading the design, development, and deployment of AI solutions. You will work with various large language models (LLMs)—both open-source and proprietary—optimizing them through fine-tuning, prompt engineering, agentic frameworks, and...
-
Principal AI Evaluation Engineer
2 weeks ago
Hyderabad, India Backbase Full timeThe Job in shortAs a a Principal AI Evaluation Engineer you will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and...
-
Principal AI Evaluation Engineer
2 weeks ago
Hyderabad, India Backbase Full timeThe Job in shortAs a a Principal AI Evaluation Engineer you will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and...
-
Principal AI Evaluation Engineer
2 weeks ago
hyderabad, India Backbase Full timeThe Job in short As a a Principal AI Evaluation Engineer you will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and...
-
Principal AI Evaluation Engineer
2 weeks ago
Hyderabad, India Backbase Full timeThe Job in short As a a Principal AI Evaluation Engineer you will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and...
-
Technical Lead
29 minutes ago
Hyderabad, India Aceolution Full timeFreelance Remote Opportunity: Tech Lead – GenAI Code InitiativesWe’re seeking an experienced Tech Lead / Senior Software Engineer to spearhead our GenAI Code Initiatives . This is a hands-on, freelancing remote role focused on evaluating, improving, and advancing AI-driven code generation systems.Key ResponsibilitiesCode Generation & Refinement: Write,...
-
Principal AI Evaluation Engineer
4 days ago
hyderabad district, India Backbase Full timeThe Job in short As a a Principal AI Evaluation Engineer you will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and...
