Ai evaluator
2 weeks ago
Company DescriptionIn today’s fast-evolving educational landscape, true impact comes not just from knowledge but from the wisdom to apply it effectively and continuously improve through real-world experience. Educatio Learning helps enterprises operationalize the entire learning value chain—from data to insights, knowledge, and advanced AI solutions—unlocking wisdom through iterative experimentation and human-centered innovation.Our diverse clients include Ed Tech platforms, academic publishers, competitive exam providers, and corporate learning teams. By blending AI with iterative human insight, Educatio Learning empowers clients worldwide to transform data into actionable knowledge and knowledge into transformative wisdom.Key Responsibilities:1. Review and Validate AI Responses:a) Evaluators will work via our online platform.b) All decisions of the Evaluators must be guided towards evaluating this central theme: usefulness of the AI response. This is the primary metric for measuring AI response quality: Is the AI response useful in addressing the question posed in the user prompt c) The goal is to assess the quality, safety, and utility of AI-generated content- particularly in use cases like literature summarization, clinical question answering, and decision support. Each AI-generated response should be scored across 10 key criteria.d) Evaluators must use a Likert scale (4=Great, 3=Good, 2=Fair, 1=Poor) on the first six criterion, and pass or fail on the remaining four.e) A commentary must be provided on the AI response, and flag critical issues (e.g., hallucinations, or ethical/safety concerns).f) Commentaries must be error-free, concise, and consistent with established medical guidelines, evidence-based practices, and current clinical standards. Rewriting the commentary and revisiting the evaluation may also be within scope based on feedback from quality checks.2. Stay Current with the latest in their domain: Stay up-to-date with the latest research, guidelines, and advancements in your area of expertise, ensuring that their validation and feedback are informed by the most current and accurate information.3. Maintain Confidentiality: Adhere to strict confidentiality of the data received for processing and comply with data protection protocols and other relevant laws and guidelines.Qualifications:1) Master’s or Ph D in a relevant field (see domains below).2) Strong command of academic writing and critical evaluation.3) Familiarity with research methodology and scholarly review preferred.4) Excellent attention to detail and analytical thinking.5) Prior experience in reviewing or teaching will be an added advantage.Domains:AccountingOperations ManagementCivil EngineeringMechanical EngineeringFinance & AccountancyPhysicsChemical EngineeringEarth ScienceEnvironmental ScienceHistoryPolitical ScienceSociologyAgricultureBusiness ManagementAnatomy and PhysiologyDentistryHealth Professions(BPT, Mpharma)NursingPharmacology and ToxicologyVeterinary ScienceBiochemistryRequirements:Minimum 6 months - 1 year of experience in the respective domain.Having your own laptop/PC.Having a stable wifi network.Why Join Us:1) Work remotely with global research and AI innovation teams.2) Influence the evolution of AI systems in your subject domain.3) Flexible work hours and intellectually stimulating projects.Hiring Process:1) Profile Screening2) Assessment Completion3) One-to-One Interview4) OnboardingInterested candidates can send their resume to
-
AI Evaluator
2 weeks ago
Delhi, India Educatio Learning Inc. Full timeCompany DescriptionIn today’s fast-evolving educational landscape, true impact comes not just from knowledge but from the wisdom to apply it effectively and continuously improve through real-world experience. Educatio Learning helps enterprises operationalize the entire learning value chain—from data to insights, knowledge, and advanced AI...
-
AI Evaluator
2 weeks ago
Delhi, India Educatio Learning Inc. Full timeCompany Description In today’s fast-evolving educational landscape, true impact comes not just from knowledge but from the wisdom to apply it effectively and continuously improve through real-world experience. Educatio Learning helps enterprises operationalize the entire learning value chain—from data to insights, knowledge, and advanced AI...
-
Principal AI Evaluation Engineer
3 weeks ago
New Delhi, India Backbase Full timeThe Job in short As a aPrincipal AI Evaluation Engineeryou will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and reporting,...
-
Principal AI Evaluation Engineer
2 weeks ago
New Delhi, India Backbase Full timeThe Job in shortAs a a Principal AI Evaluation Engineer you will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and...
-
AI Agent Evaluation Analyst
16 hours ago
Delhi, Delhi, India Mindrift Full time ₹ 2,50,000 - ₹ 12,00,000 per yearAt Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI. The Mindrift platform, launched and powered by Toloka, connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real-world...
-
AI Developer Intern
6 hours ago
Delhi, India Constructure AI Full timeAbout Constructure AI Constructure AI is building intelligent systems that make construction knowledge instantly discoverable and trusted. We turn messy project documentation such as specs, drawings, schedules, and models into structured answers that teams can rely on. If you enjoy clarity, structured thinking, and making real world data usable, you will fit...
-
AI Evaluator
3 weeks ago
Delhi, India Muenot Full timeAbout the Role:We’re seeking subject-matter experts from diverse academic and research backgrounds to join our AI Evaluation Team.Your mission: Review and Assess AI-generated responses for accuracy, depth, and clarity — and flag any factual or conceptual errors with clear, concise feedback in the comment box.If you have a sharp academic eye, love...
-
AI Intern
2 weeks ago
Greater Delhi Area, India Drema AI Full time ₹ 1,20,000 - ₹ 3,60,000 per yearAI Intern (LLM Training & Development)Location:Onsite – A-126, A Block Rd, A Block, Sector 63, Noida, Uttar Pradesh 201309Salary:₹13,000 per monthDuration:3 months (extendable or full-time based on performance)Joining:ImmediateCompany OverviewDrema AI is an innovation-driven startup building advanced AI systems and automation tools. We focus on...
-
AI Evaluator
2 weeks ago
Delhi, India Muenot Full timeAbout the Role:We're seeking subject-matter experts from diverse academic and research backgrounds to join our AI Evaluation Team.Your mission: Review and Assess AI-generated responses for accuracy, depth, and clarity — and flag any factual or conceptual errors with clear, concise feedback in the comment box.If you have a sharp academic eye, love...
-
AI Intern
2 weeks ago
Delhi, India Drema AI Full timeAI Intern (LLM Training & Development) Location: Onsite – A-126, A Block Rd, A Block, Sector 63, Noida, Uttar Pradesh Salary: ₹13,000 per month Duration: 3 months (extendable or full-time based on performance) Joining: Immediate Company Overview Drema AI is an innovation-driven startup building advanced AI systems and automation tools. We focus on...
