GEN AI-Evaluation Analyst

4 days ago

Remote, India iMerit Technology Services Pvt. Ltd Full time ₹ 2,70,000 - ₹ 10,80,000 per year

Job Description: Multimodal AI Evaluation Analyst

Target Language: English

Company: iMerit Scholars

Position Type: - Remote; Contract ($3/hour); ~10-40 hours per week

Position Overview:iMerit Scholars is rapidly expanding our network of skilled AI Evaluation Analysts to help advance next-generation multimodal AI systems.

This opportunity begins with a short paid pilot project (~12 hours total). Successful participants may be invited to continue with ongoing project work (typically 15–40 hours per week), depending on performance and project needs.

As a Multimodal AI Evaluation Analyst, you'll perform highly detailed evaluations of AI system outputs across text, image, video, and multimodal prompts. Your feedback will directly influence how leading AI systems understand and interact with the world.

What You'll Do

  • Evaluate and score AI-generated outputs (text, image captions, video descriptions, multimodal prompts)
  • Assess quality, correctness, coherence, style, and cultural alignment
  • Identify subtle issues like hallucinations, logical gaps, or bias
  • Provide detailed written feedback and apply complex evaluation guidelines consistently
  • Collaborate with Quality Leads and Project Managers to ensure reliability and accuracy

What You Bring

  • Excellent critical reading and evaluation skills across multiple modalities
  • Strong English comprehension (CEFR B2 or above)
  • Familiarity with LLMs, generative AI, and multimodal systems
  • Exceptional attention to detail and consistency
  • Comfort with iterative feedback, evolving workflows, and nuanced judgments
  • Prior experience in data annotation, AI evaluation, or related work (preferred)

Requirements

  • BA degree or equivalent experience (preferred)
  • Experience working with annotation tools or AI data platforms
  • Understanding of multimodal communication (text-to-image, video narratives, etc.)
  • Comfort with fast-paced projects and evolving quality standards
  • Awareness that incidental exposure to sensitive or NSFW content may occur

Why Join iMerit Scholars

  • Help shape how cutting-edge AI systems understand the world
  • Work remotely on projects at the forefront of multimodal AI
  • Gain hands-on experience in applied AI evaluation
  • Collaborate with a global, expert community of AI trainers

Next Steps

  • Start by taking the 30-minute skill assessment
  • Qualified applicants will be invited by email to a paid 12-hour pilot project (includes 2 hours paid training)

Strong results in the pilot may lead to ongoing, flexible remote work (15–40 hrs/week) with iMerit Scholars.

Job Types: Part-time, Fresher, Contractual / Temporary

Contract length: 4-6 weeks

Pay: ₹ ₹225.00 per hour

Expected hours: 40 per week

Benefits:

  • Work from home

Work Location: Remote

  • Sr. Business Development Executive

    6 days ago

    Remote, India Neural Gen-AI Networks Full time ₹ 3,60,000 - ₹ 8,40,000 per year

    About Us:Neural Gen-AI Networks is an emerging leader in the field of Artificial Intelligence and advanced digital solutions. We are driven by innovation, creativity, and a mission to empower businesses through intelligent automation and data-driven strategies.We are expanding our dynamic team and looking for a Business Development Executive who is...

  • Northcorp-Gen AI

    2 days ago

    Remote, India Nexthire Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    AI & GenAI Solutions | 10+ Years ExperienceJob Description:We are seeking an experienced Senior Full Stack Developer (10+ years) to work on AI-powered and Generative AI (GenAI) solutions. The ideal candidate must have deep expertise primarily in Python along with experience in Python-based frameworks like LangChain, LangGraph, and FastAPI. Good knowledge...

  • AI Engineer

    1 week ago

    Remote, India Serendipity Corporate Services Full time ₹ 14,05,424 - ₹ 28,20,848 per year

    Key Responsibilities Lead the day-to-day development of AI solutions, including writing production grade code and building scalable systems. Candidate should have good hands-on -Design & Development - Full SDLC in Python, .Net and Strong in Gen AI area. Interact directly with customers to understand requirements, propose solutions, and ensure successful...

  • SME – Next-Gen Skill

    1 week ago

    Remote, India Angstrom Education Pvt Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role summaryHelp us design, review, and validate innovative assessments and learning assets for next-generation skill tracks. You'll partner with our learning designers to ensure accuracy, rigor, and learner impact.What you'll doReview/edit assessments for accuracy, relevance, difficulty, and fairness.Ensure learning outcomes, rubrics, and item metadata...

  • Gen AI Platform Developer

    2 weeks ago

    Remote, India Raah Techservices Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    We are looking for a Software Developer – GenAI who combines strong Python expertise and cloud-native architecture skills with a passion for building cutting-edge AI-powered applications using the latest AWS services.This role is ideal for developers who want to take full ownership of the development lifecycle, collaborate in agile, cross-functional teams,...

  • Strategy and Operations

    1 week ago

    Remote, India Wokelo AI Full time

    **About Wokelo**: Wokelo is a Gen-AI powered investment research platform that automates complex research and analysis tasks traditionally performed by humans. Our platform is utilized by leading Private Equity, Investment Banks, Corporate Strategy, Venture Capital, and Fortune 500 firms. Our proprietary agentic technology and state-of-the-art LLMs deliver...

  • Gen AI Developer

    2 days ago

    Remote, India SBSHR Full time ₹ 20,00,000 - ₹ 28,00,000 per year

    Contact : / Job Title: GenAI Full Stack Engineer | Immediate / serving noticeLocation: RemoteExperience: Overall 6+ Years (Minimum 3–4 Years in AI/ML/ GenAI Development)About the RoleWe are seeking a highly skilled GenAI Full Stack Engineer with strong expertise in AI/ML development, agentic AI solutions, and Azure cloud services. The ideal candidate will...

  • Gen Ai and Llms

    6 days ago

    Remote, India Elfonze Technologies Full time

    **Job Information**: Date Opened - 07/03/2025 Job Type - Full time Industry - IT Services Remote Job This is a remote position. **Job Summary**: **Key Responsibilities**: **1. Solution Architecture & Design** - Design and architect GenAI solutions using **Azure AI (Azure OpenAI, Azure ML)** and/or **AWS AI/ML services (Bedrock, SageMaker, Comprehend,...

  • Gen Ai Mentor

    2 days ago

    Remote, India Aadhira charitable trust Full time

    Generative AI Trainer - FDP (Online) **Job Type**: Freelance / Contract - 5 Days (Work from Home) **Role Overview**: **Key Responsibilities**: - Engage participants with practical insights, use cases, and demonstrations - Handle Q&A sessions and encourage interactive discussions - Prepare session materials, PPTs, and hands-on...

  • Video Editor

    2 days ago

    Remote, India Buzz AI- Artificial Intelligence Full time

    **About the Role** Bizz Buzz AI is a US based Artificial Intelligence Company. We are looking for a highly skilled **Video Editor & AI Video Creator** who can craft compelling visual content using both traditional editing software and cutting-edge AI tools. If you're as comfortable with a creative brief as you are with prompt engineering—and can turn...