Senior Quality Assurance Specialist
3 days ago
Job Description:We are seeking a skilled professional to lead quality assurance initiatives for cutting-edge Generative AI and Multi-Agent Systems. This role will focus on test automation, LLM evaluation, RAG pipelines, observability, adversarial testing, and Azure monitoring.The ideal candidate will have expertise in Python automation, LLM evaluation, RAG pipelines, observability, and adversarial testing. They should be able to design and execute LLM evaluation frameworks using G-Eval, custom evaluators, and LLM-as-a-Judge.Key Responsibilities:Lead and scale test automation initiatives with a focus on GenAI and AI systemsDesign and execute LLM evaluation frameworks using G-Eval, custom evaluators, and LLM-as-a-JudgeImplement RAG evaluation frameworks (RAGAS or similar)Build Python-based automation frameworks using PyTest & DeepEvalIntegrate automation into CI/CD pipelines using GitHub ActionsDesign and validate multi-agent evaluation pipelines (tool usage, collaboration, reasoning chains)Perform adversarial and red-team testing: prompt injection, jailbreak attacks, bias and toxicity detectionConduct API testing for microservices (REST, async workflows)Monitor applications using Azure Application Insights & Log AnalyticsDefine automated scoring systems for GenAI outputsManage synthetic datasets and golden datasets for AI validationImplement observability and trace monitoring using LangFuse, LangSmith, or similar toolsRequired Skills and Qualifications:Test Automation – Python, PyTest, DeepEvalLLM Evaluation – G-Eval, Custom Evaluators, LLM-as-a-JudgeRAG Evaluation – RAGAS, Retrieval MetricsEvaluation Metrics – Hallucination, Faithfulness, Relevance, Precision/RecallObservability & Monitoring – LangFuse, LangSmithCI/CD – GitHub ActionsMulti-Agent Testing – Reasoning & Tool ValidationAdversarial/Red Team Testing – Prompt Injection, Jailbreak, Bias/ToxicityAPI Testing – REST & Async WorkflowsAzure Monitoring – App Insights, Log AnalyticsSynthetic & Golden Dataset ManagementAutomated Scoring System Design for GenAI Outputs
-
Quality Assurance Specialist
4 days ago
amravati, India beBeeQuality Full timeJob Opportunity: Quality Assurance ProfessionalWe are seeking a skilled professional to fill the role of Quality Assurance (QA) Specialist, focusing on the testing and validation of web applications built with Drupal and React.js.Key Responsibilities:Test and validate web applications for functionality, usability, and performance.Collaborate with development...
-
Quality Assurance Specialist
3 days ago
amravati, India beBeeAssurance Full timeQuality Assurance SpecialistWe are seeking a highly motivated and driven individual to join our team as a Quality Assurance Specialist. In this role, you will have the opportunity to make a significant impact in an organization that is rapidly growing.You will be responsible for developing test cases, testing features and functionality, and creating...
-
Pega Quality Assurance Professional
5 days ago
amravati, India beBeeQuality Full timeSenior Pega Quality Assurance SpecialistWe are seeking a seasoned professional to join our team as a Senior Pega Quality Assurance Specialist. In this role, you will be responsible for ensuring the quality and effectiveness of software systems and processes.Evaluate software systems to ensure they meet required quality standards through rigorous testing and...
-
Senior Quality Assurance Engineer
3 days ago
amravati, India beBeeQuality Full timeSenior Quality Assurance EngineerWe are seeking a skilled and experienced Senior Quality Assurance Engineer to join our team.This is a senior-level position that requires a deep understanding of software development life cycles, quality assurance principles, and testing methodologies.The ideal candidate will have a strong background in manual testing,...
-
Senior Quality Assurance Professional
5 days ago
amravati, India beBeeAutomation Full timeLead Quality Assurance We are seeking a seasoned Software Testing Specialist to join our agile development team and take ownership of delivering high-quality software through advanced testing strategies. This is a hands-on role where you will design and implement automation frameworks, execute complex test plans, and drive an AI-first approach to quality...
-
amravati, India beBeeDataPipeline Full timeQuality Assurance Specialist for Data PipelinesWe are seeking a skilled Quality Assurance Specialist to oversee the testing of our data pipelines, ensuring seamless integration across cloud and on-prem environments.The ideal candidate will possess strong hands-on experience in ETL/Data Pipeline Testing, API Testing, and MDM Testing. A good understanding of...
-
Chief Quality Assurance Officer
5 days ago
amravati, India beBeeQuality Full timeSenior Quality Assurance ManagerJob Summary:We are seeking an experienced Senior Quality Assurance Manager to lead our quality assurance efforts. The successful candidate will be responsible for developing and implementing a comprehensive quality assurance strategy that ensures the highest level of product quality.Key Responsibilities:Develop and implement...
-
amravati, India beBeeAutomation Full timeJob Title: Senior Quality Assurance Automation EngineerWe are seeking a highly skilled and experienced Quality Assurance Automation Engineer to join our team. As a key member of our quality engineering group, you will play a critical role in ensuring the quality of our products through the planning and execution of product testing, and compliance with...
-
Senior Quality Assurance Professional
1 day ago
amravati, India beBeeLeadership Full timeQuality Assurance Leader OpportunityJob Overview:We are seeking a Quality Assurance Leader to spearhead our efforts in ensuring the quality and reliability of our product. As a key member of our team, you will be responsible for developing and implementing strategies to ensure that our product meets the highest standards of quality.This is an exciting...
-
Quality Assurance Expert
4 days ago
amravati, India beBeeQuality Full timeSenior Quality Assurance SpecialistJob Description:Execute comprehensive testing for large, complex deployments on ServiceNow platforms.Develop and refine functional test automation frameworks.Promote quality assurance best practices in continuous integration and delivery pipelines.Evaluate and review unit tests to ensure excellence.Analyze and improve test...
