Lead QA – Automated Testing

2 weeks ago

Kolkata, India RapidBrains Full time

Lead QA – Automated Testing & AI Validation Experience: 6+ Years Employment Type: Contract Notice Period: Immediate Role Overview We are seeking a Lead QA – Automated Testing & AI Validation with strong expertise in Python automation, LLM evaluation, RAG pipelines, observability, adversarial testing, and Azure monitoring. This role will lead quality assurance initiatives for cutting-edge Generative AI and Multi-Agent Systems. Key Responsibilities Lead and scale test automation initiatives with a focus on GenAI and AI systems Design and execute LLM evaluation frameworks using: LLM-as-a-Judge (G-Eval, custom evaluators) Hallucination detection, faithfulness, relevance, precision/recall Implement RAG evaluation frameworks (RAGAS or similar) Build Python-based automation frameworks using PyTest & DeepEval Integrate automation into CI/CD pipelines using GitHub Actions Design and validate multi-agent evaluation pipelines (tool usage, collaboration, reasoning chains) Perform adversarial and red-team testing: Prompt injection Jailbreak attacks Bias and toxicity detection Conduct API testing for microservices (REST, async workflows) Monitor applications using Azure Application Insights & Log Analytics Define automated scoring systems for GenAI outputs Manage synthetic datasets and golden datasets for AI validation Implement observability and trace monitoring using LangFuse, LangSmith, or similar tools Mandatory Skills Test Automation – Python, PyTest, DeepEval LLM Evaluation – G-Eval, Custom Evaluators, LLM-as-a-Judge RAG Evaluation – RAGAS, Retrieval Metrics Evaluation Metrics – Hallucination, Faithfulness, Relevance, Precision/Recall Observability & Monitoring – LangFuse, LangSmith CI/CD – GitHub Actions Multi-Agent Testing – Reasoning & Tool Validation Adversarial/Red Team Testing – Prompt Injection, Jailbreak, Bias/Toxicity API Testing – REST & Async Workflows Azure Monitoring – App Insights, Log Analytics Synthetic & Golden Dataset Management Automated Scoring System Design for GenAI Outputs

  • Senior QA Test Automation Specialist

    2 weeks ago

    kolkata, India beBeeAutomation Full time

    Lead QA Test Automation Engineer SDETWe are seeking an exceptional Lead QA Test Automation Engineer to join our team.This is a full-time remote role for a Lead QA Test Automation Engineer SDET. The successful candidate will be responsible for developing and executing automated tests, creating test cases, and performing manual testing as needed.The role...

  • Pursuit Software

    4 weeks ago

    Kolkata, India Pursuit Software Development Full time

    Job Role : Quality Engineer LeadYears of Experience : 5+ YearsJob Location : Kolkata, Indore, ShillongEducation : Bachelors Degree in Computer Science & Engineering or Similar FieldJob Description : Responsibilities : 1. Lead and drive test automation and SW QA for different components.2. Assist with manual testing for those components that are not easily...

  • Mobile automation Testing

    1 week ago

    Kolkata, India People Prime Worldwide Full time

    Mobile Automation Testing EngineerLocation: Kolkata Experience:P3: 5–8 YearsP4: 8–12 Years Employment Type: Permanent Notice Period: Up to 90 DaysJob DescriptionMust-Have SkillsStrong hands-on experience in Mobile Automation TestingExpertise in Android and iOS application testingProficiency in Appium for mobile test automationSolid programming skills in...

  • Mobile automation Testing

    1 week ago

    Kolkata, India People Prime Worldwide Full time

    Mobile Automation Testing Engineer Location: Kolkata Experience: - P3: 5–8 Years - P4: 8–12 Years - Employment Type: Permanent - Notice Period: Up to 90 Days Job Description Must-Have Skills - Strong hands-on experience in Mobile Automation Testing - Expertise in Android and iOS application testing - Proficiency in Appium for mobile test automation -...

  • Automated Testing Lead

    4 days ago

    kolkata, India beBeeAutomationTestEngineer Full time

    Senior Automation Test EngineerAs a seasoned test automation professional, we are seeking an experienced Senior Automation Test Engineer to lead our test automation efforts. The ideal candidate will have expertise in creating and maintaining automated end-to-end test suites using Playwright with TypeScript.

  • Cloud-Based QA Automation Specialist

    2 days ago

    kolkata, India beBeeAutomation Full time

    Job DescriptionWe are seeking an ambitious Senior QA Automation Engineer to spearhead the migration of a legacy system from the mainframe to a cloud-based Java environment for one of the largest banks in the USA.The successful candidate will work closely with the QA Lead and another QA to develop and implement test strategies, execute testing activities...

  • QA Engineer with Automated Testing Expertise

    6 days ago

    kolkata, India beBeeQuality Full time

    We're seeking a Quality Assurance Engineer with expertise in automated testing, collaboration with developers, and defect documentation.About the RoleAs a key member of our engineering team, you will execute manual and automated testing for web/product features to ensure high-quality assurance. Your experience with tools like Selenium, REST Assured, or...

  • QA Automation Engineer

    3 weeks ago

    Kolkata, India Xped pvt Ltd Full time

    Key Responsibilities : - Primary responsibilities include working with Dev/PM/Test to design and implement test and validation strategies throughout the development life cycle- Design and implement robust, scalable, and high quality test automation suites- Write, execute and debug automated test cases, troubleshoot and debug complex systems- Able to mentor...

  • Senior Test Automation Specialist

    2 weeks ago

    kolkata, India beBeeAutomation Full time

    Automation EngineerAs a key member of our team, you will play a pivotal role in driving the success of our automation initiatives. With a strong background in UFT and VBScript, you will be responsible for designing, developing, and executing automated test scripts for web and desktop applications.Key ResponsibilitiesDesign and develop automated test scripts...

  • Senior Test Automation Specialist

    7 days ago

    kolkata, India beBeeAutomation Full time

    Automation Test LeadWe are looking for an experienced Automation QA Test Lead with expertise in Java selenium, Rest API testing. Prior experience in handling a team as well.Design and implement automated tests for web and mobile applications using Selenium and BDD frameworks.Create and maintain test automation frameworks for web, mobile, and API projects.API...