Senior AI Quality Assurance Specialist

5 days ago

vellore, India beBeeValidation Full time

Lead AI Validation EngineerWe are seeking a highly skilled engineer to lead our quality assurance initiatives for cutting-edge Generative AI and Multi-Agent Systems.The ideal candidate will have strong expertise in Python automation, LLM evaluation, RAG pipelines, observability, adversarial testing, and Azure monitoring.This role will involve designing and executing LLM evaluation frameworks using G-Eval, custom evaluators, and LLM-as-a-Judge. The successful candidate will also be responsible for implementing RAG evaluation frameworks, building Python-based automation frameworks using PyTest & DeepEval, and integrating automation into CI/CD pipelines using GitHub Actions.Key Responsibilities:Design and execute LLM evaluation frameworks using G-Eval, custom evaluators, and LLM-as-a-JudgeImplement RAG evaluation frameworksBuild Python-based automation frameworks using PyTest & DeepEvalIntegrate automation into CI/CD pipelines using GitHub ActionsPerform adversarial and red-team testing: prompt injection, jailbreak attacks, bias and toxicity detectionConduct API testing for microservices (REST, async workflows)Monitor applications using Azure Application Insights & Log AnalyticsDefine automated scoring systems for GenAI outputsManage synthetic datasets and golden datasets for AI validationImplement observability and trace monitoring using LangFuse, LangSmith, or similar toolsRequired Skills and Qualifications:Test Automation – Python, PyTest, DeepEvalLLM Evaluation – G-Eval, Custom Evaluators, LLM-as-a-JudgeRAG Evaluation – RAGAS, Retrieval MetricsEvaluation Metrics – Hallucination, Faithfulness, Relevance, Precision/RecallObservability & Monitoring – LangFuse, LangSmithCI/CD – GitHub ActionsMulti-Agent Testing – Reasoning & Tool ValidationAdversarial/Red Team Testing – Prompt Injection, Jailbreak, Bias/ToxicityAPI Testing – REST & Async WorkflowsAzure Monitoring – App Insights, Log AnalyticsSynthetic & Golden Dataset ManagementAutomated Scoring System Design for GenAI OutputsBenefits:This role offers the opportunity to work on cutting-edge technologies and contribute to the development of innovative AI solutions.Others:Please note that this is not an exhaustive list of responsibilities and may evolve over time.

  • AI Systems Quality Assurance Specialist

    5 days ago

    vellore, India beBeeAnalytical Full time

    AI Systems Quality Assurance SpecialistWe are seeking an experienced and skilled professional to contribute to our cutting-edge AI systems. As an AI Systems Quality Assurance Specialist, you will be responsible for ensuring the accuracy and reliability of AI-generated outputs.Key Responsibilities:Evaluate AI-generated responses by ranking, reviewing, and...

  • Quality Assurance Specialist

    5 days ago

    vellore, India beBeeQuality Full time

    Job Title: Quality Assurance SpecialistThe quality assurance specialist is responsible for conducting technical assessments of recorded interviews, scrutinizing assigned recordings from start to finish, identifying and annotating any issues such as premature termination, audio/video glitches, or other anomalies.Additionally, they will perform investigations...

  • Data Quality and Assurance Specialist

    7 days ago

    vellore, India beBeeData Full time

    Job Title:We're looking for an accomplished Data Quality and Assurance Specialist to join our team.

  • Quality Assurance Professional

    7 days ago

    vellore, India beBeeQuality Full time

    Job OverviewWe are seeking a highly skilled and experienced Quality Assurance Specialist to join our team.The successful candidate will be responsible for ensuring the quality of our processes and services, identifying areas for improvement, and implementing corrective actions.Key ResponsibilitiesPerform quality assurance calls and process reviews for all...

  • Senior Quality Assurance Specialist

    3 days ago

    vellore, India beBeeInspector Full time

    **Job Title:** Senior InspectorAs a key member of our team, you will be responsible for performing inspections on renewable energy equipment such as solar modules, energy storage systems, wind turbines, and inverters. Your role will involve coordinating with suppliers, verifying site conditions, and analyzing data to ensure that all equipment meets our...

  • AI Quality Assurance Specialist

    5 days ago

    vellore, India beBeeArtificialIntelligence Full time

    Our organization seeks a highly skilled Artificial Intelligence Tester to guarantee the quality and integrity of our AI solutions.We require you to test advanced in-house developed AI systems, focusing on Generative AI and Agentic AI.You will be responsible for functional testing for AI applications using Python, Tosca/Selenium, and Agile methodologies.A...

  • Senior Software Quality Assurance Specialist

    3 days ago

    vellore, India beBeeQuality Full time

    Job Description:Our client is seeking a seasoned quality assurance professional to join their team as a Senior QA. This opportunity presents a chance to work on the implementation of Oracle banking latest modules, including OBDX, OBTFPM, and OBTF.The ideal candidate will have extensive experience in testing Oracle Banking Application, with a strong focus on...

  • Senior Quality Assurance Specialist

    7 days ago

    vellore, India beBeeQuality Full time

    Job OverviewBackgammon Galaxy is a leading platform for playing backgammon, boasting a community of 150,000+ players worldwide.Lead and mentor the QA team in defining quality standards and best practices.Design test automation strategies for regression and performance testing.Establish and maintain QA processes across Web (React), Mobile (Flutter), Backend...

  • Software Quality Assurance Specialist

    7 days ago

    vellore, India beBeeQuality Full time

    Job Title: Software Quality Assurance EngineerAbout the RoleMagna seeks a skilled QA Engineer to ensure software applications meet high-quality standards before release.This role bridges technical development and business needs, focusing on quality assurance testing.Key Responsibilities:Analyze business requirements and user stories to ensure testability and...

  • Head of Software Quality Assurance

    7 days ago

    vellore, India beBeeAutomation Full time

    Job Title: Senior Test Automation EngineerJob DescriptionThis role is for a senior test automation engineer who will be responsible for developing and executing automated tests, creating test cases, and performing manual testing as needed. This position involves collaborating with development teams to ensure software quality and addressing any issues...