AI Evaluator

12 hours ago

Delhi, India Educatio Learning Inc. Full time

Company DescriptionIn today’s fast-evolving educational landscape, true impact comes not just from knowledge but from the wisdom to apply it effectively and continuously improve through real-world experience. Educatio Learning helps enterprises operationalize the entire learning value chain—from data to insights, knowledge, and advanced AI solutions—unlocking wisdom through iterative experimentation and human-centered innovation.Our diverse clients include EdTech platforms, academic publishers, competitive exam providers, and corporate learning teams. By blending AI with iterative human insight, Educatio Learning empowers clients worldwide to transform data into actionable knowledge and knowledge into transformative wisdom.Key Responsibilities:1. Review and Validate AI Responses:a) Evaluators will work via our online platform.b) All decisions of the Evaluators must be guided towards evaluating this central theme: usefulness of the AI response. This is the primary metric for measuring AI response quality: Is the AI response useful in addressing the question posed in the user prompt c) The goal is to assess the quality, safety, and utility of AI-generated content- particularly in use cases like literature summarization, clinical question answering, and decision support. Each AI-generated response should be scored across 10 key criteria.d) Evaluators must use a Likert scale (4=Great, 3=Good, 2=Fair, 1=Poor) on the first six criterion, and pass or fail on the remaining four.e) A commentary must be provided on the AI response, and flag critical issues (e.g., hallucinations, or ethical/safety concerns).f) Commentaries must be error-free, concise, and consistent with established medical guidelines, evidence-based practices, and current clinical standards. Rewriting the commentary and revisiting the evaluation may also be within scope based on feedback from quality checks.2. Stay Current with the latest in their domain: Stay up-to-date with the latest research, guidelines, and advancements in your area of expertise, ensuring that their validation and feedback are informed by the most current and accurate information.3. Maintain Confidentiality: Adhere to strict confidentiality of the data received for processing and comply with data protection protocols and other relevant laws and guidelines.Qualifications:1) Master’s or PhD in a relevant field (see domains below).2) Strong command of academic writing and critical evaluation.3) Familiarity with research methodology and scholarly review preferred.4) Excellent attention to detail and analytical thinking.5) Prior experience in reviewing or teaching will be an added advantage.Domains:AccountingOperations ManagementCivil EngineeringMechanical EngineeringFinance & AccountancyPhysicsChemical EngineeringEarth ScienceEnvironmental ScienceHistoryPolitical ScienceSociologyAgricultureBusiness ManagementAnatomy and PhysiologyDentistryHealth Professions(BPT,Mpharma)NursingPharmacology and ToxicologyVeterinary ScienceBiochemistryRequirements:Minimum 6 months - 1 year of experience in the respective domain.Having your own laptop/PC.Having a stable wifi network.Why Join Us:1) Work remotely with global research and AI innovation teams.2) Influence the evolution of AI systems in your subject domain.3) Flexible work hours and intellectually stimulating projects.Hiring Process:1) Profile Screening2) Assessment Completion3) One-to-One Interview4) OnboardingInterested candidates can send their resume to

  • Principal AI Evaluation Engineer

    4 days ago

    New Delhi, India Backbase Full time

    The Job in short As a aPrincipal AI Evaluation Engineeryou will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and reporting,...

  • AI Model Evaluation Engineer

    3 weeks ago

    Delhi Division, India Turing Full time

    Role Overview:Turing is seeking experienced Python developers to partner with a leading AI research lab in building high-quality datasets and evaluation pipelines that improve next-generation large language models.In this role, your Python expertise will directly influence how AI models understand and generate code. You’ll design and implement solutions,...

  • AI Evaluator

    2 days ago

    Delhi, India Muenot Full time

    About the Role:We’re seeking subject-matter experts from diverse academic and research backgrounds to join our AI Evaluation Team.Your mission: Review and Assess AI-generated responses for accuracy, depth, and clarity — and flag any factual or conceptual errors with clear, concise feedback in the comment box.If you have a sharp academic eye, love...

  • AI Evaluator

    12 hours ago

    Delhi, India Muenot Full time

    About the Role:We're seeking subject-matter experts from diverse academic and research backgrounds to join our AI Evaluation Team.Your mission: Review and Assess AI-generated responses for accuracy, depth, and clarity — and flag any factual or conceptual errors with clear, concise feedback in the comment box.If you have a sharp academic eye, love...

  • AI Evaluator

    3 weeks ago

    Delhi, India Muenot Full time

    About the Role:We’re seeking subject-matter experts from diverse academic and research backgrounds to join our AI Evaluation Team.Your mission: Review and Assess AI-generated responses for accuracy, depth, and clarity — and flag any factual or conceptual errors with clear, concise feedback in the comment box.If you have a sharp academic eye, love...

  • AI Evaluator

    3 weeks ago

    Delhi, India Muenot Full time

    About the Role:We’re seeking subject-matter experts from diverse academic and research backgrounds to join our AI Evaluation Team.Your mission: Review and Assess AI-generated responses for accuracy, depth, and clarity — and flag any factual or conceptual errors with clear, concise feedback in the comment box.If you have a sharp academic eye, love...

  • AI Evaluator

    3 weeks ago

    New Delhi, India Muenot Full time

    About the Role: We’re seeking subject-matter experts from diverse academic and research backgrounds to join our AI Evaluation Team.Your mission: Review and Assess AI-generated responses for accuracy, depth, and clarity — and flag any factual or conceptual errors with clear, concise feedback in the comment box.If you have a sharp academic eye, love...

  • AI Engineer

    5 days ago

    Delhi, India Amogha AI Full time

    About Amogha AI At Amogha AI, we are building India’s first voice-led conversational AI app specifically for mental health, therapy, and emotional well-being. Our mission is to provide a supportive, empathetic listener in your pocket, 24/7. We are creating a next-generation product that understands user context, provides therapy-grade support, and...

  • Evaluation Specialist

    3 days ago

    Delhi, India LawSikho Full time

    Job Description: Evaluation Associate SkillArbitrageAbout the RoleAs an Evaluations Associate, you will:Review student submissions in real-world skills:- AI Tools (ChatGPT, Canva, Wix, Jasper, Midjourney, Notion AI, etc.)- Personal Branding (LinkedIn, Twitter (X), Instagram profile optimization, content strategy, client pitching, thought leadership)- Digital...

  • Evaluation Specialist

    5 days ago

    Delhi, India LawSikho Full time

    J ob Description: Evaluation Associate SkillArbitrageAbout the RoleAs an Evaluations Associate, you will:Review student submissions in real-world skills:- AI Tools (ChatGPT, Canva, Wix, Jasper, Midjourney, Notion AI, etc.)- Personal Branding (LinkedIn, Twitter (X), Instagram profile optimization, content strategy, client pitching, thought leadership)-...