AI Test Automation Specialist

2 days ago

nashik, India beBeeQuality Full time

Job OpportunityWe are seeking a Mid QA professional with hands-on experience in testing AI-powered systems, particularly LLM-driven applications. Key responsibilities include designing and developing automated test cases using Python and Pytest for AI-driven applications; validating integrations, APIs, and multi-agent workflows through functional, regression, and smoke testing; performing intent classification, semantic similarity, and response consistency testing for conversational AI systems; conducting hallucination detection and factual accuracy checks using automated and semi-automated methods; implementing response quality scoring using evaluation patterns; monitoring and validating model behavior using tools such as LangFuse or LangSmith; testing conversational AI applications including chatbots and virtual assistants across use cases; supporting application health checks and basic smoke testing using Kubernetes; integrating tests into CI/CD pipelines using GitHub Actions; documenting test cases, evaluation criteria, defects, and findings clearly and concisely; collaborating with engineering and AI teams to improve evaluation pipelines and strategies.Key SkillsHands-on experience in testing AI-powered systemsAutomated testing using Python and PytestLLM integration and validationIntent classification and semantic similarity testingConversational AI system testingKubernetes-based application health checksCICD pipeline management using GitHub ActionsAbout the RoleThis is an exciting opportunity to work on advanced AI evaluation methodologies and contribute to the development of cutting-edge technologies.

  • Automated Testing Specialist

    2 weeks ago

    nashik, India beBeeAutomation Full time

    Automated Testing SpecialistWe are seeking a highly skilled Automated Testing Specialist to join our team. As an Automated Testing Specialist, you will be responsible for creating and implementing Business-Driven Development (BDD) test scenarios based on business requirements.You will design and implement automation layers for BDD tests that contribute to...

  • AI Automation Specialist

    2 weeks ago

    nashik, India beBeeAutomation Full time

    Senior AI & Automation ExpertOur goal is to hire a highly motivated Senior Software Test Engineer who takes complete ownership of product quality—from concept to customer delivery.Lead end-to-end testing for complex enterprise applications—UI, API, data, and integration layers.Own and drive the complete QA lifecycle, from planning to production...

  • AI Automation Expert

    12 hours ago

    nashik, India beBeeAutomation Full time

    Artificial Intelligence Automation SpecialistWe are seeking a skilled and experienced Artificial Intelligence (AI) automation specialist to support the development of innovative solutions.This role involves implementing encryption for proprietary prompts and agent knowledge files, creating synthetic datasets to support OT/IT test scenarios, supporting...

  • Senior Test Automation Specialist

    7 days ago

    nashik, India beBeeAutomation Full time

    Job Summary: As a senior test automation specialist, you will be responsible for designing and developing automated test frameworks and scripts using C# to test web-based and API-based applications across different projects.Key Responsibilities:Develop and maintain automated test suites using C#, Selenium/PlaywrightIntegrate tests into CI/CD pipelines using...

  • AI Workflow Automation Specialist

    2 weeks ago

    nashik, India beBeeAutomation Full time

    No-code AI Solutions InternshipJob OverviewZentalX is a pioneer in no-code AI solutions, making AI accessible to users. We empower individuals to create custom AI agents, automate workflows, and access personalized learning tools without programming skills.This internship role involves creating and deploying no-code AI agents, troubleshooting automated...

  • Test Automation Specialist

    2 days ago

    nashik, India beBeeAutomation Full time

    Automation Test LeadWe are seeking a seasoned Automation QA Test Lead to spearhead our test automation strategy. The ideal candidate will possess hands-on experience in designing and implementing automated tests for web and mobile applications using Selenium and BDD frameworks.Develop and execute automated tests for web and mobile applications utilizing...

  • Automation Specialist

    2 weeks ago

    nashik, India beBeeAutomation Full time

    Automation Specialist (Remote)We're seeking an enthusiastic individual to collaborate with our product and engineering teams on automation projects. Key responsibilities include designing and developing automation workflows, writing, testing, and debugging scripts.ResponsibilitiesDesign and develop chat and voice automation workflowsWrite, test, and debug...

  • Test Automation Specialist

    1 week ago

    Nashik, India Peoplefy Full time

    We're Hiring Automation Test Engineer (3-6 Years) Location: Pune Work Mode: Hybrid Shift Timing: 2:00 PM - 11:00 PM Joining: 30-60 Days We are looking for a skilled Automation Test Engineer with hands-on experience in modern automation tools to join our growing QA team. Key Skills Required: - Automation Testing using Selenium - Experience with qTest OR...

  • Automation Specialist

    3 days ago

    nashik, India beBeeAutomation Full time

    Automation SpecialistIn today's fast-paced digital landscape, companies require robust automation solutions to perfect their engineering cycles. Our team seeks an Automation Specialist to develop cutting-edge automation code for mobile applications using Python or other preferred languages.Key Responsibilities:Write automation code for mobile applications,...

  • Highly Skilled Test Automation Specialist Wanted

    2 weeks ago

    nashik, India beBeeAutomation Full time

    We're seeking a skilled Test Automation Specialist to join our team.In this role, you'll design, execute, and maintain automated tests for our internal systems—covering backend (APIs) and frontend—fully integrated into our GitLab pipelines.Key Responsibilities:Develop and automate functional tests for REST APIs using Postman/Newman and end-to-end tests...