GEN AI-Evaluation Analyst

4 days ago

Remote, India iMerit Technology Services Pvt. Ltd Full time ₹ 2,70,000 - ₹ 10,80,000 per year

Job Description: Multimodal AI Evaluation Analyst

Target Language: English

Company: iMerit Scholars

Position Type: - Remote; Contract ($3/hour); ~10-40 hours per week

Position Overview:iMerit Scholars is rapidly expanding our network of skilled AI Evaluation Analysts to help advance next-generation multimodal AI systems.

This opportunity begins with a short paid pilot project (~12 hours total). Successful participants may be invited to continue with ongoing project work (typically 15–40 hours per week), depending on performance and project needs.

As a Multimodal AI Evaluation Analyst, you'll perform highly detailed evaluations of AI system outputs across text, image, video, and multimodal prompts. Your feedback will directly influence how leading AI systems understand and interact with the world.

What You'll Do

  • Evaluate and score AI-generated outputs (text, image captions, video descriptions, multimodal prompts)
  • Assess quality, correctness, coherence, style, and cultural alignment
  • Identify subtle issues like hallucinations, logical gaps, or bias
  • Provide detailed written feedback and apply complex evaluation guidelines consistently
  • Collaborate with Quality Leads and Project Managers to ensure reliability and accuracy

What You Bring

  • Excellent critical reading and evaluation skills across multiple modalities
  • Strong English comprehension (CEFR B2 or above)
  • Familiarity with LLMs, generative AI, and multimodal systems
  • Exceptional attention to detail and consistency
  • Comfort with iterative feedback, evolving workflows, and nuanced judgments
  • Prior experience in data annotation, AI evaluation, or related work (preferred)

Requirements

  • BA degree or equivalent experience (preferred)
  • Experience working with annotation tools or AI data platforms
  • Understanding of multimodal communication (text-to-image, video narratives, etc.)
  • Comfort with fast-paced projects and evolving quality standards
  • Awareness that incidental exposure to sensitive or NSFW content may occur

Why Join iMerit Scholars

  • Help shape how cutting-edge AI systems understand the world
  • Work remotely on projects at the forefront of multimodal AI
  • Gain hands-on experience in applied AI evaluation
  • Collaborate with a global, expert community of AI trainers

Next Steps

  • Start by taking the 30-minute skill assessment
  • Qualified applicants will be invited by email to a paid 12-hour pilot project (includes 2 hours paid training)

Strong results in the pilot may lead to ongoing, flexible remote work (15–40 hrs/week) with iMerit Scholars.

Job Types: Part-time, Fresher, Contractual / Temporary

Contract length: 4-6 weeks

Pay: ₹ ₹225.00 per hour

Expected hours: 40 per week

Benefits:

  • Work from home

Work Location: Remote

  • Gen Ai

    2 weeks ago

    Remote, India Websar Technologies LLP Full time

    **Gen AI** **Location - Remote** **Job: Contract** **Exp: 7+ years** **JD-** - Core skills needed: Amazon Bedrock, Kendra, S3, Python, ChatGPT 4 Turbo, AWS Services in Gen AI. - Working Model: Agile Methodology. experience in python, AWS services and Gen AI knowledge will be making the task easier to understand. **Job Types**: Full-time,...

  • Data Scientist Gen Ai

    7 days ago

    Remote, India Chandorkar Technologies OPC Pvt. Ltd Full time

    **Interview rounds 3 Technical rounds** **Necessary Qualifications**: - Should have good experience LLMs like OpenAI, Llama2 etc - Should have good understanding & experience of frameworks like Langchain, LlamaIndex, which are used to build Gen AI solutions - Should have good experience in the Vector database - Should have experience with transformer...

  • Sr. Business Development Executive

    7 days ago

    Remote, India Neural Gen-AI Networks Full time ₹ 3,60,000 - ₹ 8,40,000 per year

    About Us:Neural Gen-AI Networks is an emerging leader in the field of Artificial Intelligence and advanced digital solutions. We are driven by innovation, creativity, and a mission to empower businesses through intelligent automation and data-driven strategies.We are expanding our dynamic team and looking for a Business Development Executive who is...

  • Northcorp-Gen AI

    2 days ago

    Remote, India Nexthire Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    AI & GenAI Solutions | 10+ Years ExperienceJob Description:We are seeking an experienced Senior Full Stack Developer (10+ years) to work on AI-powered and Generative AI (GenAI) solutions. The ideal candidate must have deep expertise primarily in Python along with experience in Python-based frameworks like LangChain, LangGraph, and FastAPI. Good knowledge...

  • Gen AI Platform Developer

    2 weeks ago

    Remote, India Raah Techservices Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    We are looking for a Software Developer – GenAI who combines strong Python expertise and cloud-native architecture skills with a passion for building cutting-edge AI-powered applications using the latest AWS services.This role is ideal for developers who want to take full ownership of the development lifecycle, collaborate in agile, cross-functional teams,...

  • Strategy and Operations

    4 days ago

    Remote, India Wokelo AI Full time

    **About Wokelo**: Wokelo is a Gen-AI powered investment research platform that automates complex research and analysis tasks traditionally performed by humans. Our platform is utilized by leading Private Equity, Investment Banks, Corporate Strategy, Venture Capital, and Fortune 500 firms. Our proprietary agentic technology and state-of-the-art LLMs deliver...

  • AI Engineer

    2 weeks ago

    Remote, India Beroe Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are looking for an experienced AI Engineer to join our growing AI team. You'll play a key role in developing intelligent, agentic AI systems using cutting-edge large language models (LLMs), multi-agent orchestration, and retrieval-augmented generation (RAG). This is a hands-on role combining software engineering, ML/NLP expertise, and a passion for...

  • Full Stack Gen AI Developer

    5 days ago

    Remote, India Nao Medical Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    About the CompanyNao Medical is a rapidly growing multispecialty, multi-site, and AI-infused practice dedicated to providing exceptional and affordable care. We operate through five main business segments: Primary Care, Urgent Care, Occupational Health, Mental Health, and Nutrition.Our AI-first operating model has led to the development of proprietary...

  • Azure Data Scientist

    4 days ago

    Remote, India Jeevan Technologies Full time

    **JOB LOCATION**: **Remote (All over India)** **POSITIONS**: **2** **EXPERIENCE**: **6** **Immediate joiners** **PRIMARY SKILLS**: Summary of skills required: - Azure Databricks, Azure ML, Azure Data factory, Azure wasp services, Azure AI services, python - Open AI, Gen AI LLMs, Machine learning, Deep learning, NLP Key Responsibilities: - Design and...

  • AI Developer

    1 week ago

    Remote, India Innovacio Technologies Pvt Ltd Full time ₹ 3,15,000 - ₹ 4,20,000 per year

    About Us:We are an innovative technology company focused on leveraging Artificial Intelligence to solve complex business problems. Our team is currently building advanced solutions in predictive analytics and sales optimization, and we are looking for a talented AI Developer to join us.Key Responsibilities:Design, develop, and deploy Sales Forecasting Models...