Agentic AI Quality Assurance Engineer

2 days ago

Bengaluru, Karnataka, India Aritone Global Ventures Ltd Full time ₹ 5,00,000 - ₹ 15,00,000 per year

Description :


- Location : Bangalore (Work from Office)

- Experience : 1- 4 Years

Key Responsibilities :


- Test and validate AI agents and LLM workflows to ensure correctness and stability.

- Design and execute test cases to evaluate agentic behavior, prompt responses, and task accuracy.

- Use DeepEval to create and automate evaluation frameworks for LLM and agent testing.

- Identify, document, and track bugs, performance issues, or logic errors.

- Collaborate with developers, prompt engineers, and data scientists to improve system quality.

- Develop and maintain test documentation, including plans, cases, and reports.

- Ensure AI agents handle contexts, prompts, and tasks as intended in real-world scenarios.

- Contribute to building automated QA pipelines for continuous testing and monitoring of agents.


Required Skills & Qualifications :

- 1- 4 years of experience in QA, software testing, or AI testing.

- Strong understanding of Generative AI, LLM agents, and autonomous workflows.

- Experience with manual testing and familiarity with automation testing frameworks.

- Hands-on experience or working knowledge of DeepEval for evaluating LLM outputs.

- Good analytical, debugging, and documentation skills.

- Familiarity with Java or Python for writing scripts or automation utilities.

- Excellent collaboration and communication abilities.


Good to Have :


- Experience with LangChain, CrewAI, AutoGen, or similar agentic frameworks.

- Understanding of AI evaluation metrics (accuracy, coherence, relevance, hallucination detection).

- Exposure to CI/CD, test automation, or model monitoring tools.

  • Quality Assurance and Automation Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Forage Ai Llc Full time ₹ 40,00,000 - ₹ 1,20,00,000 per year

    Seeking a QA Automation Engineer to test and automate data and AI systems using Selenium,Python,and JavaScript.Ensure quality, accuracy,and performance across web crawling,GenAI,and agentic workflows with strong analytical and problem-solving skills. Required Candidate profile3+ yrs in QA automation/data testing. Skilled in Selenium,Python,JS, HTML, and...

  • Quality Assurance

    1 week ago

    Bengaluru, Karnataka, India Weekday AI Full time ₹ 5,00,000 - ₹ 11,00,000 per year

    This role is for one of the Weekday's clientsSalary range: Rs Rs ie INR 5-11 LPA)Min Experience: 3 yearsLocation: BANGALOREJobType: full-timeWe are seeking a skilled QA Automation Engineer with proven expertise in building robust automation frameworks, executing comprehensive test strategies, and ensuring high-quality releases across web, mobile, and API...

  • Quality Assurance

    1 week ago

    Bengaluru, Karnataka, India Weekday AI Full time ₹ 5,00,000 - ₹ 11,00,000

    This role is for one of the Weekday's clientsSalary range: Rs Rs ie INR 5-11 LPA)Min Experience: 3 yearsLocation: BANGALOREJobType: full-timeWe are seeking a skilled QA Automation Engineer with proven expertise in building robust automation frameworks, executing comprehensive test strategies, and ensuring high-quality releases across web, mobile, and API...

  • Senior Quality Assurance Engineer

    6 days ago

    Bengaluru, Karnataka, India PROADSW3 Relations pvt. ltd. Full time ₹ 15,000 - ₹ 30,000 per year

    Hiring: Senior QA Engineer – Multimedia & AI PlatformsLocation:Bangalore (Hybrid / On-site)Employment Type:Full-TimeBudget:₹1.9 LPMExperience:7+ Years (Min. 5+ Years in Automation)About the RoleWe're looking for an experiencedSenior QA Engineerto take complete ownership of quality assurance acrossAI-driven multimedia platforms, including web...

  • Agentic QA Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Bespoke Technologies Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    About the RoleWe are looking for an Agentic QA Engineer with 1 - 5 years of experience to test and validate AI agents and automation workflows. The ideal candidate should understand GenAI-based agents, testing methodologies, and quality assurance processes to ensure AI agents perform as expected.Key ResponsibilitiesTest and validate AI agents to ensure they...

  • agentic ai engineer

    1 week ago

    Bengaluru, Karnataka, India Apexon Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description About Apexon: Apexon is a digital-first technology services firm specializing in accelerating business transformation and delivering human-centric digital experiences. We have been meeting customers wherever they are in the digital lifecycle and helping them outperform their competition through speed and innovation. Apexon brings together...

  • AI Agent Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Observe Full time ₹ 8,00,000 - ₹ 16,00,000 per year

    About UsObserve.AI is the leading AI agent platform for customer experience. It enables enterprises to deploy AI agents that automate customer interactions, delivering natural conversations for customers with predictable outcomes for the business.Observe.AI combines advanced speech understanding, workflow automation, and enterprise-grade governance to...

  • AI Agent Engineer

    6 days ago

    Bengaluru, Karnataka, India GE VERNOVA Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    **Summary****We are seeking a highly motivated and talented Prompt Engineer to join our dynamic team. The successful candidate will be responsible for creating and optimizing prompts for LLMs, collaborating with domain experts and end users to develop software solutions that address both internal and external needs.This position offers a great opportunity to...

  • Clinical Quality Assurance

    2 weeks ago

    Bengaluru, Karnataka, India Weekday AI Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    This role is for one of the Weekday's clientsMin Experience: 8 yearsLocation: Bengaluru, KarnatakaJobType: full-timeWe are seeking an experienced Clinical Quality Assurance (QA) Specialist with a strong background in Teleradiology, CT, MRI, and Imaging Quality Management. The ideal candidate will ensure that all radiology reports, imaging processes, and...

  • AI Agent Engineer

    1 week ago

    Bengaluru, Karnataka, India Observe Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    About UsObserve.AIis the leading AI agent platform for customer experience.It enables enterprises to deploy AI agents that automate customer interactions, delivering natural conversations for customers with predictable outcomes for the business.Observe.AIcombines advanced speech understanding, workflow automation, and enterprise-grade governance to execute...