Senior Quality Assurance Engineer

1 day ago

bhavnagar, India beBeeAutomation Full time

Job Title: Senior Quality Assurance Engineer Description:We are seeking a highly skilled Senior Quality Assurance Engineer to lead our quality assurance initiatives for cutting-edge Generative AI and Multi-Agent Systems.This role will be responsible for designing and executing test automation initiatives, implementing RAG evaluation frameworks, and building Python-based automation frameworks using PyTest & DeepEval. The ideal candidate will have expertise in LLM evaluation, observability, adversarial testing, and Azure monitoring.Key Responsibilities:Lead and scale test automation initiatives with a focus on GenAI and AI systemsDesign and execute LLM evaluation frameworks using G-Eval, custom evaluators, and LLM-as-a-JudgeImplement RAG evaluation frameworks (RAGAS or similar)Build Python-based automation frameworks using PyTest & DeepEvalIntegrate automation into CI/CD pipelines using GitHub ActionsDesign and validate multi-agent evaluation pipelines (tool usage, collaboration, reasoning chains)Perform adversarial and red-team testing: prompt injection, jailbreak attacks, bias, and toxicity detectionConduct API testing for microservices (REST, async workflows)Monitor applications using Azure Application Insights & Log AnalyticsDefine automated scoring systems for GenAI outputsManage synthetic datasets and golden datasets for AI validationImplement observability and trace monitoring using LangFuse, LangSmith, or similar toolsMandatory Skills:Test Automation – Python, PyTest, DeepEvalLLM Evaluation – G-Eval, Custom Evaluators, LLM-as-a-JudgeRAG Evaluation – RAGAS, Retrieval MetricsEvaluation Metrics – Hallucination, Faithfulness, Relevance, Precision/RecallObservability & Monitoring – LangFuse, LangSmithCI/CD – GitHub ActionsMulti-Agent Testing – Reasoning & Tool ValidationAdversarial/Red Team Testing – Prompt Injection, Jailbreak, Bias/ToxicityAPI Testing – REST & Async WorkflowsAzure Monitoring – App Insights, Log AnalyticsSynthetic & Golden Dataset ManagementAutomated Scoring System Design for GenAI Outputs],

  • Automation Quality Assurance Leader

    5 days ago

    bhavnagar, India beBeeSenior Full time

    Job DescriptionWe are seeking a seasoned professional to spearhead our quality assurance efforts. As a Senior Quality Assurance Automation Engineer, you will be responsible for ensuring the highest standards of product quality through meticulous planning and execution.Key Responsibilities:Develop and manage comprehensive testing strategies to ensure seamless...

  • Senior Quality Assurance Specialist

    5 days ago

    bhavnagar, India beBeeQuality Full time

    Quality Assurance ExpertWe are seeking a meticulous Quality Assurance Expert to join our team. In this role, you will be responsible for ensuring the high quality of software applications by planning, executing, and documenting tests.Key Responsibilities: Analyze business requirements and technical specifications. Develop detailed test plans, test cases, and...

  • Senior Quality Assurance Automation Engineer

    3 days ago

    Bhavnagar, India Veltris Full time

    We’re looking for a Senior QA Automation Engineer to join our dynamic team! You’ll play a key role in building robust automation frameworks, driving test quality, and collaborating with development teams in India and clients in the US. Experience Required- 4 to 10 years Location- Hyderabad or Remote. Roles & Responsibility: - Design, create, and maintain...

  • Senior Quality Engineer

    2 weeks ago

    Bhavnagar, India Whatjobs IN C2 Full time

    Job Title: Quality Assurance Engineer (SIEM / SaaS experience preferred) Location: Remote (India) About Company It is an early-stage cybersecurity startup building a next-generation platform for security data analytics. We process and query large-scale event and log data, leveraging modern open-source technologies and a hybrid SaaS model. Our team is small,...

  • Leading AI Quality Assurance Specialist

    3 days ago

    bhavnagar, India beBeeResponsibility Full time

    Senior Quality Assurance EngineerWe are seeking a highly skilled and experienced Artificial Intelligence Quality Assurance Test Engineer to join our dynamic team.GenAI, Prompt Testing & Agentic AI ValidationResponsible AI, Model Quality & Data ValidationFunctional, Integration & System Testing for AI ApplicationsTechnical QA Skills – Python,...

  • Senior Quality Assurance Specialist

    1 day ago

    bhavnagar, India beBeeSoftware Full time

    Mobile Software Quality Assurance RoleOur organization is seeking a seasoned software quality assurance specialist to ensure the reliability and consistency of mobile SDKs, web applications, and analytics services.Conduct thorough testing for mobile SDKs, web applications, and analytics services, including UI validation and API assessments.Review...

  • Quality Assurance Expert

    3 days ago

    bhavnagar, India beBeeQualityAssurance Full time

    Job Title: Quality Assurance ExpertAs a Quality Assurance Expert, you will play a vital role in ensuring the quality and reliability of our software applications. Your primary responsibility will be to design and execute comprehensive testing plans to identify defects and improve overall product quality.Key Responsibilities:Test Planning &...

  • Ecommerce Quality Assurance Expert

    5 days ago

    bhavnagar, India beBeeLeadership Full time

    Quality Assurance ManagerWe are seeking an experienced Quality Assurance Manager to lead our distributed QA team in ensuring exceptional quality and support for our high-traffic e-commerce platform.About the Role:This role focuses on defining quality assurance strategies, creating test plans, and overseeing execution for web and mobile applications. You will...

  • Senior Quality Assurance Tester

    5 days ago

    bhavnagar, India beBeeQuality Full time

    Job Title: Senior QA EngineerWe are seeking a skilled Senior QA Engineer to ensure the quality and reliability of our mature inspection management platform. This is an exciting opportunity for you to utilize your technical expertise in software testing, set up test automation frameworks from scratch, configure test environments, and build robust testing...

  • Quality Assurance Automation Specialist

    5 days ago

    bhavnagar, India beBeeAutomation Full time

    Establishing Quality Assurance ProcessesWe are seeking a Senior Automation Engineer to spearhead our quality assurance efforts. The ideal candidate will have extensive experience in developing and implementing automation testing strategies for complex systems.The primary responsibilities of this role include:Collaborating with developers to identify and...