QA Automation Engineer

1 day ago

Vikhroli, India Techstalwarts Software Development LLP Full time

Designation - QA/Automation

Exp - 2yr - 3yrs

Key Responsibilities

Platform & Workflow QA

Build and maintain automation frameworks (Playwright, Postman, PyTest) for

frontend, API, and workflows

Validate workflows (e.g. in n8n) that use agents, knowledgebases, and

external tools

Ensure business-critical flows are error-free, with auto-healing & test

coverage in CI/CD

Agent Behavior & Prompt Evaluation

Design eval suites to validate prompt reliability, response safety, hallucination

rates, and fallback logic

Align with prompt engineers and agent designers to fine-tune behavior across

business verticals

Create test rubrics for accuracy, tone, response time, and user experience

LLM & Retrieval Evaluation

Test performance of retrieval chains (RAG) across agents and use cases

Design regression tests for memory consistency, tool usage, and voice

readiness

QA Automation, AI Evaluation & Solution Engineer 2

Contribute to eval scoring mechanisms (agent correctness, confidence, intent

coverage, etc.)

Client Enablement & Delivery QA

Collaborate with onboarding team to define and enforce QA checkpoints

during rollout

QA client workflows (lead qualification, support agents, surveys, etc.)

Mentor junior team members executing client-level QA and automation tasks

Strategic Quality Oversight

Report platform-wide quality metrics (pass rates, model failure modes,

workflow issues)

Ensure QA is deeply integrated into Dev, Product, and GTM motions

Continuously evolve quality strategy for an agent-first, AI-native platform

What We're Looking For

Must-Haves

6+ years in QA/Automation, with at least 2 years in a Lead or Senior capacity

Strong hands-on skills in Python, Playwright, Postman, and CI/CD pipelines

Experience with n8n, Zapier, or similar workflow/RPA tools

Familiarity with LLM APIs (OpenAI, Anthropic, etc.) and prompt engineering

basics

Clear understanding of Conversational AI, Voice Agents, or Agent-based

workflows

Ownership mindset with the ability to define structure in a fast-moving product

environment

Bonus

QA Automation, AI Evaluation & Solution Engineer 3

Experience testing AI agents with RAG, vector databases, and

LangChain/LLM orchestration

GitHub profile showcasing automation/test projects, prompt eval frameworks,

or agent logic

Familiarity with STT/TTS, telephony, or voice orchestration frameworks like

LiveKit or Twilio

Job Types: Full-time, Permanent

Pay: ₹60, ₹100,000.00 per month

Benefits:

  • Cell phone reimbursement
  • Health insurance
  • Provident Fund

Work Location: In person

  • QA Automation Engineer

    2 weeks ago

    Vikhroli, Mumbai, Maharashtra, India Techstalwarts Software Development LLP Full time ₹ 10,00,000 - ₹ 12,00,000 per year

    Designation - QA/AutomationExp - 2yr - 3yrsKey ResponsibilitiesPlatform & Workflow QABuild and maintain automation frameworks (Playwright, Postman, PyTest) forfrontend, API, and workflowsValidate workflows (e.g. in n8n) that use agents, knowledgebases, andexternal toolsEnsure business-critical flows are error-free, with auto-healing & testcoverage in...

  • Operations Co-ordinator

    2 weeks ago

    Vikhroli, Mumbai, Maharashtra, India White Globe Private Limited Full time

    Location: Vikhroli, Mumbai The language QC leads main responsibility is to facilitate/manage all activities related to translation engagements. Starting from evaluation and estimating the translation scope based on project requirement. Should be able to train new QC resources before and during validation of the project and monitoring bug management....

  • Project Coordinator

    2 weeks ago

    Vikhroli, Mumbai, Maharashtra, India White Globe Pvt. Ltd. Full time

    JD - QC / PROJECT COORDINATOR The language QC leads main responsibility is to facilitate/manage all activities related to translation engagements. Starting from evaluation and estimating the translation scope based on project requirement. Should be able to train new QC resources before and during validation of the project and monitoring bug management....

  • E3D BIM Coordinator

    1 day ago

    Vikhroli, India Black & Veatch Full time

    We believe real value is powered by the unique skills and experiences of our professionals. The interchange of ideas from a diverse group of people gives our teams an expanded perspective and the ability to find better solutions for our clients. Req Id : Job Title : E3D BIM Coordinator Business Unit sector : CPL-BVOPS-ENGR & DEV SVCS Department: BVCPL BV...

  • CQV Manager

    2 days ago

    Vikhroli, India IPS-Integrated Project Services Full time

    Job Description:Directly responsible for the successful delivery of compliance projects to IPS' clients Works with IPS CQV/Project management team to develop and apply quality practices for all project deliverables and business practices. In conjunction with the Project management team, performs project audits to ensure all project goals are being met and...

  • Team Member

    2 weeks ago

    Vikhroli, Mumbai, Maharashtra, India Cipla Full time

    **Division**: - Integrated Product Development**Department**: - Global Respiratory**Sub Department 1**: - Respiratory -Formulations R&D - DPI**Job Purpose**: This role leads in the development, validation, engineering, prototyping, and demonstrating a working, compact and compliant device product which is ready for scale up and tech transfer. Primary...

  • E3D BIM Coordinator

    2 weeks ago

    Vikhroli, Mumbai, Maharashtra, India Black & Veatch Full time US$ 80,000 - US$ 1,20,000 per year

    We believe real value is powered by the unique skills and experiences of our professionals. The interchange of ideas from a diverse group of people gives our teams an expanded perspective and the ability to find better solutions for our clients.Req Id : 110234Job Title : E3D BIM CoordinatorBusiness Unit sector : CPL-BVOPS-ENGR & DEV SVCSDepartment: BVCPL BV...

  • CQV Manager

    2 weeks ago

    Vikhroli, Mumbai, Maharashtra, India IPS-Integrated Project Services Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Job Description:Directly responsible for the successful delivery of compliance projects to IPS' clientsWorks with IPS CQV/Project management team to develop and apply quality practices for all projectdeliverables and business practices. In conjunction with the Project management team, performs projectaudits to ensure all project goals are being met and...

  • Associate, Payment Analytics, Consumer Banking Group

    7 days ago

    Vikhroli, Mumbai, Maharashtra, India DBS Bank Full time

    Business Function As the leading bank in Asia, DBS Consumer Banking Group is in a unique position to help our customers realise their dreams and ambitions. As a market leader in the consumer banking business, DBS has a full spectrum of products and services, including deposits, investments, insurance, mortgages, credit cards and personal loans, to help our...