QA Engineer AI Agent System

2 weeks ago

Hyderabad, India Everite Solutions Full time

We're looking for a meticulous QA Engineer to lead testing efforts for our multi-agent AI system that automates crew member matching, onboarding, and job tracking. This is a cross-functional QA role where youll validate everything from AI-generated decisions to backend APIs, UI components, and third-party integrations (voice cloning, Twilio).

Youll create and run test plans for 8+ autonomous agents, including LLM-backed agents, and help ensure every part of the system behaves reliably and ethically before, during, and after deployment.

Key Responsibilities:

Manual & Exploratory Testing

  • Understand agent workflows (Job Matcher, Onboarding Agent).
  • Perform manual testing of workflows across:
  • Web UI interfaces
  • LLM responses and validation triggers
  • Communications (email, SMS, voice)
  • Backend API changes
  • Identify issues in agent decision logic (false positives, incorrect matching, onboarding gaps, etc.).

Automated Testing

  • Design and implement automated test scripts for:
  • API endpoints
  • Event/job creation flows
  • Onboarding and contract handling
  • SMS/Email/Voice triggers and responses
  • Use tools like Postman, Selenium, Playwright, or PyTest for scripting.

Test Planning & Documentation

  • Write and manage:
  • Test cases and scenarios for each AI agent.
  • Test Data and Synthetic data preparation (with support from ML team)
  • Regression, functional, and smoke test plans.
  • Edge-case tests for fallbacks, failed calls, no-shows, etc.
  • Document known issues, test coverage, and agent-specific behavior.

QA Reporting

  • Track bugs and raise detailed Jira tickets for:
  • UI/UX inconsistencies
  • API response mismatches
  • Communication errors
  • Model prediction issues (with support from ML team)
  • Report test progress and bug severity regularly to the engineering team.

Collaboration & Feedback Loops

  • Collaborate closely with:
  • AI Engineers to validate LLM and ML behavior.
  • Backend Devs for API consistency and error handling.
  • Voice & Communication Engineers to validate cloned call quality and integrations.
  • Help define agent success metrics from a QA lens (e.g., no incorrect crew matches, no duplicate invites, etc.).

Required Skills & Qualifications:

  • 3-5+ years in QA Engineering or SDET roles
  • Solid understanding of API and integration testing
  • Experience with test automation tools:
  • Postman / Swagger
  • Selenium
  • Python-based testing frameworks (PyTest preferred)
  • Knowledge of version control (Git/Bitbucket) and issue tracking (Jira Issues)

Preferred (Bonus) Experience:

  • Testing systems with AI/ML components or LLM-based applications
  • Familiarity with voice API testing (e.g., Twilio Voice, ElevenLabs output validation)
  • Worked on event-based or on-demand staffing platforms
  • Understanding of CI/CD and QA automation pipelines

Tools You May Work With:

  • Automation & API Tools: Postman, PyTest, Playwright, Selenium
  • Bug Tracking: GitHub Projects, Jira
  • Monitoring Feedback: Azure Monitor / Log Analytics
  • Voice & Message Validation: Call transcripts, email/SMS message logs
  • Other: Google Sheets (Test tracking), Teams (Internal team comms)

  • QA Engineer AI Agent System

    2 days ago

    Hyderabad, Telangana, India Everite Solutions Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    We're looking for a meticulous QA Engineer to lead testing efforts for our multi-agent AI system that automates crew member matching, onboarding, and job tracking. This is a cross-functional QA role where youll validate everything from AI-generated decisions to backend APIs, UI components, and third-party integrations (voice cloning, Twilio).Youll create and...

  • Agentic AI Intern

    2 weeks ago

    Hyderabad, India Centella AI Therapeutics Full time

    Centella AI is an AI-first organization addressing the critical challenges of drug discovery such as cost and time. Our AI platform, CASCADE utilizes Large Language Models (LLMs) to design drug-like molecules with improved safety and efficacy while significantly reducing costs. We are looking for passionate Agentic AI intern interested to work at the...

  • Agentic AI Intern

    2 days ago

    Hyderabad, Telangana, India Centella AI Therapeutics Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Centella AI is an AI-first organization addressing the critical challenges of drug discovery such as cost and time. Our AI platform, CASCADE utilizes Large Language Models (LLMs) to design drug-like molecules with improved safety and efficacy while significantly reducing costs. We are looking for passionate Agentic AI intern interested to work at the...

  • AI Engineer

    2 days ago

    Hyderabad, Telangana, India Deccan AI Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Location:Hyderabad (On-site/Hybrid)Company:Deccan AIEmployment Type:Full-TimeExperience:1–3 YearsAbout Deccan AIDeccan AI is at the forefront of building advanced AI solutions for real-world applications. We're passionate about pushing the boundaries of artificial intelligence, natural language processing, and intelligent agent systems. Join us and work...

  • AI Engineer

    2 weeks ago

    Hyderabad, India Deccan AI Full time

    Location:Hyderabad (On-site/Hybrid) Company:Deccan AI Employment Type:Full-Time Experience:1–3 Years About Deccan AIDeccan AI is at the forefront of building advanced AI solutions for real-world applications. We're passionate about pushing the boundaries of artificial intelligence, natural language processing, and intelligent agent systems. Join us and...

  • Test (QA) Engineer - Python & Agentic AI Tools

    9 hours ago

    Bengaluru, Hyderabad, Pune, India E Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Description: We are looking for an experienced QA Engineer with strong expertise in Python and hands-on exposure to Agentic AI tools. The ideal candidate should have a proven track record in test automation, framework development, and AI-driven testing approaches.Key Responsibilities:Design, develop, and maintain test automation frameworks using...

  • ServiceNow AI Agent Engineer

    1 week ago

    Hyderabad, India Accelyst, LLC Full time

    Accelyst is an innovative AI Consultancy that leverages a unique catalog of industry specific Agents and leading-edge AI platforms to deliver tangible, integrated, secure and ROI-optimized solutions. We combine deep industry and technical expertise to enable rapid deployment of innovative AI-driven capabilities to augment and automate client workflows for...

  • ServiceNow AI Agent Engineer

    7 days ago

    Hyderabad, India Accelyst, LLC Full time

    Accelyst is an innovative AI Consultancy that leverages a unique catalog of industry specific Agents and leading-edge AI platforms to deliver tangible, integrated, secure and ROI-optimized solutions. We combine deep industry and technical expertise to enable rapid deployment of innovative AI-driven capabilities to augment and automate client workflows for...

  • Agentic AI Engineer

    1 week ago

    Hyderabad, India Innosential Full time

    Agentic AI Engineer (Lead Role)About the RoleWe are looking for a highly skilled AI Data Scientist / Engineer (Lead Role) to drive the development of an AI-driven code generation system for Microsoft Dynamics (X++). This role will focus on LLM engineering, benchmark design, and pipeline automation, ensuring that AI-generated code is both functional and...

  • Agentic AI Developer

    2 weeks ago

    Hyderabad, Bengaluru, Chennai, India ValueLabs LLP Full time

    Job Description Job Description: Agentic AI Developer Position Overview Seeking an AI Developer with a software development background, expertise in AI tools, multi-agent orchestration frameworks, RAG techniques, and Model Context Protocol (MCP). Proficiency in Python, .NET, and hands-on experience with coding agents and research agents is...