AI Testing Specialist

5 days ago

Bengaluru, Karnataka, India beBeeConversational Full time ₹ 6,00,000 - ₹ 10,00,000

We are looking for a meticulous Conversational AI Evaluator to join our team. In this role, you will meticulously test and evaluate the performance of conversational bots.

This is a non-technical position that doesn't require coding, but does require a foundational understanding of how conversational bots work, how workflows are structured, and how to methodically test and evaluate them.

Key Responsibilities:
  • Evaluate Conversational Bots: Systematically test chat and voice assistant outputs for coherence, relevance, tone, factual accuracy, and alignment with business objectives.
  • Test Workflows: Assess conversation flows and user journeys for edge cases, logical consistency, and usability across different platforms (web, mobile, IVR).
  • First-Principles Problem Solving: Analyze and break down complex issues using first-principles thinking; question assumptions and propose better alternatives.
  • Scenario Creation: Develop diverse test scenarios and real-world prompts to validate conversational bot performance across varied contexts and user intents.
  • Feedback & Reporting: Document bugs, inconsistencies, and improvement opportunities; work closely with product, design, and engineering teams.
  • Alignment Audits: Ensure conversational bot responses align with user intent, brand tone, and ethical guidelines.
  • Collaborate on Evaluation Frameworks: Help define or refine qualitative and quantitative criteria for conversational bot performance and user experience.

Requirements include 2-4 years of experience in conversational bot testing, strong analytical skills, and a problem-solving mindset grounded in first-principles thinking.

Qualifications:
  • Prior experience in QA, user research, product evaluation, or content moderation is a plus.
  • Familiarity with generative AI technologies (like ChatGPT, Bard, or Claude) and LLMs.
  • Basic understanding of conversation design, chat/voice interfaces, and user journey mapping.
  • Ability to write clear, concise test documentation and reports.
  • Excellent communication skills and attention to detail.
  • Curious, user-centric mindset with a drive for continuous improvement.

  • AI Testing Specialist

    23 hours ago

    Bengaluru, Karnataka, India beBeeGenerative Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Overview\Software Quality Assurance Engineer III Role:\We are seeking a highly skilled and experienced Software Quality Assurance (SQA) Engineer III to join our innovative team. As a key member of our AI testing group, you will play a crucial role in evaluating the performance, accuracy, and reliability of our generative AI models.\Your Key...

  • AI Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Skyfall AI Full time

    Position Overview We are seeking an experienced AI Data Engineer to lead the development and scaling of synthetic data generation systems that support multiple enterprise platforms. This role combines deep technical expertise in AI/ML data pipelines with practical experience in system integration and orchestration frameworks. Position Overview We are...

  • Test Specialist

    2 weeks ago

    Bengaluru, Karnataka, India Orbit Core Tech Full time

    Company Description Orbit Tech Hub is a technology company that delivers AI-powered digital libraries, custom applications, crypto exchanges, ERP systems, hotel management solutions, and data center deployments. With a flexible team of experts, we assist clients in innovating, digitizing, and scaling their businesses across India, GCC, and the Mena...

  • Senior AI Testing Specialist

    1 week ago

    Bengaluru, Karnataka, India beBeeGenerativeAi Full time

    Software Quality Assurance Engineer III Job DescriptionWe are seeking an exceptional candidate to join our innovative team as a Software Quality Assurance Engineer III. The ideal candidate will play a crucial role in evaluating the performance, accuracy, and reliability of our generative AI models.The primary objective of this position is to design and...

  • SEO & GEO Specialist

    2 days ago

    Bengaluru, Karnataka, India Astro247 AI Full time

    About Astro 247: Astro 247 is an AI-powered astrology app that provides users with instant insights into their future. Our mission is to blend technology with ancient wisdom, offering a unique and engaging astrology experience. We're growing fast, and as search evolves with AI-driven engines , we need a specialist who can help us dominate both ...

  • SEO & GEO Specialist

    2 days ago

    Bengaluru, Karnataka, India Astro247 AI Full time

    About Astro 247:Astro 247 is an AI-powered astrology app that provides users with instant insights into their future. Our mission is to blend technology with ancient wisdom, offering a unique and engaging astrology experience.We're growing fast, and as search evolves with AI-driven engines, we need a specialist who can help us dominate both traditional...

  • ai specialist

    3 weeks ago

    Bengaluru, Karnataka, India SAANVI CLOTHING PRIVATE LIMITED Full time

    Job Description: AI SPECIALISTWe're seeking a talented artificial intelligence specialist to join our team at Envie.As an AI specialist, you will develop and implement AI models and algorithms to address complex business challenges. You will also design, implement, and optimize AI systems to solve existing problems and enhance our business operations. The...

  • AI/ML Test Engineering Specialist

    10 hours ago

    Bengaluru, Karnataka, India beBeeMachinelearning Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    AI/ML Test Engineer Job OpportunityWe are seeking a skilled professional to join our team as an AI/ML Test Engineer. This role involves designing and executing comprehensive test plans for AI and ML-based software systems.This is an excellent opportunity for someone who is passionate about testing and automation, with strong skills in Python programming and...

  • ai specialist

    1 day ago

    Bengaluru, Karnataka, India SAANVI CLOTHING PRIVATE LIMITED Full time

    Job Description: AI SPECIALISTWe're seeking a talented artificial intelligence specialist to join our team at Envie. As an AI specialist, you will develop and implement AI models and algorithms to address complex business challenges. You will also design, implement, and optimize AI systems to solve existing problems and enhance our business operations. The...

  • AI Test Automation Developer

    4 weeks ago

    Bengaluru, Karnataka, India MSRcosmos LLC Full time

    Position: AI Test Automation DeveloperLocation : Bangalore hybrid Responsibilities:Develop AI-driven test agents to automate routine testing tasks such as test case generation, documentation, and scripting.Implementing automation frameworks for testing AI models and software components.Integrate test automation solutionsinto existing automation...