AI Evaluation Expert

1 week ago

Jamnagar, India beBeeTesting Full time

Job OverviewWe seek a seasoned Automated Testing Specialist with expertise in testing AI-powered systems, particularly Large Language Model-driven applications. The ideal candidate will have a strong background in QA automation, a growing understanding of probabilistic AI outputs, and enthusiasm for building and improving AI evaluation frameworks in real-world environments.Develop and execute automated test cases using Python and Pytest for AI-driven applications.Validate LLM integrations, APIs, and multi-agent workflows through functional, regression, and smoke testing.Perform intent classification, semantic similarity, and response consistency testing for conversational AI systems.Conduct hallucination detection and factual accuracy checks using automated and semi-automated methods.Implement response quality scoring using LLM-as-a-Judge evaluation patterns.Use LLM observability and tracing tools such as LangFuse or LangSmith to monitor and validate model behavior.Test conversational AI applications including chatbots and virtual assistants across use cases.Support Kubernetes-based application health checks and basic smoke testing.Integrate automated tests into CI/CD pipelines using GitHub Actions.Document test cases, evaluation criteria, defects, and QA findings clearly and concisely.Collaborate with engineering and AI teams to improve evaluation pipelines and testing strategies.

  • Expert AI Content Evaluator

    1 week ago

    jamnagar, India beBeeContentEvaluator Full time

    Key ResponsibilitiesWe are seeking skilled professionals with proficiency in one or more Indian languages to assess, review, and compare AI-generated responses. This role involves identifying toxic or harmful content, understanding linguistic nuances, and ensuring high-quality model performance across multiple datasets.Evaluate AI-generated outputs across...

  • AI Content Evaluator

    6 days ago

    jamnagar, India beBeeEvaluation Full time

    Job Opportunity: We are seeking a skilled AI content evaluator who is a Telugu language specialist to assess, review, and compare AI-generated responses.Evaluate AI-generated outputs specifically in Telugu (both native script and transliterated formats).Identify and flag harmful or toxic content, including subtle or context-dependent cases.Strong command of...

  • Kannada Language AI Content Evaluator

    7 days ago

    jamnagar, India beBeeContent Full time

    Kannada Language AI Content SpecialistJob Overview:We are seeking skilled professionals to assess, review and compare AI-generated content in Kannada. The role involves evaluating the quality of model responses, identifying toxic or harmful content and ensuring high-performance linguistic evaluations.Evaluate AI outputs in Kannada (native script and...

  • Content Evaluator

    7 days ago

    Jamnagar, India beBeeToxicity Full time

    Job Title: Content SpecialistWe are seeking a highly skilled evaluator to review and compare AI-generated responses in the Marathi language.This role involves identifying toxic or harmful content across native scripts and transliterated text, as well as assessing model performance across multiple datasets.Key Responsibilities:Evaluate AI model outputs in...

  • AI Quality Assurance Expert

    2 weeks ago

    Jamnagar, India beBeeAutomated Full time

    Job OpportunityWe are seeking a high-caliber professional to lead quality assurance initiatives for cutting-edge Generative AI and Multi-Agent Systems. The ideal candidate will have 6+ years of experience in test automation using Python, PyTest, and DeepEval, LLM evaluation using G-Eval, custom evaluators, and LLM-as-a-Judge, RAG evaluation using RAGAS and...

  • AI Technical Expert

    7 days ago

    jamnagar, India beBeeArtificial Full time

    About the JobWe're seeking a technical expert to join our team and drive meaningful change through AI-powered solutions.As a key member of our team, you'll work closely with enterprise customers to implement and customize cutting-edge technology, delivering innovative solutions that transform employee experiences.You will receive comprehensive on-the-job...

  • AI Engineer

    2 days ago

    Jamnagar, India MightyBot Full time

    Join our team as an AI Engineer, where we're focused on graduating AI from interesting demos to indispensable products. You will build reliable, self-improving systems that empower subject matter experts to automate their most complex, high-value work. This is a role for engineers who want to solve real-world business challenges and create AI tools that are...

  • AI and Machine Learning Expert

    1 week ago

    jamnagar, India beBeeMachineLearning Full time

    AI and Machine Learning ExpertTalentXM is a next-generation AI-driven talent orchestration platform redefining how talent and opportunities connect by combining AI innovation, orchestration, and future-of-work intelligence.Key Responsibilities:We aim to expose you to Research emerging trends, tools, and architecturesCollaborate on projects and...

  • Advanced Coding Expert Wanted

    2 weeks ago

    Jamnagar, India beBeePython Full time

    Expert PHP Developer RoleWe're seeking a skilled expert in PHP development to join our team. As a key member, you'll be responsible for creating high-quality code, curating datasets, and evaluating code outputs for fine-tuning models.This is a remote contract role that involves collaborating with domain experts to ensure accurate annotations and...

  • Visual Design Expert: For AI Innovation

    2 weeks ago

    Jamnagar, India beBeeDesign Full time

    Visual Design Expert: For AI InnovationWe seek a skilled Visual Design professional who can bridge the gap between human aesthetics and artificial intelligence. This role involves developing real-world prompts to train AI models in design principles, visual problem-solving, and creativity.Evaluate AI-generated outputs, focusing on visual responses,...