AI QA Automation Engineer

1 week ago

Delhi, Delhi, India Aidewiser Soltek Full time

Title : AI QA Automation Engineer

Location : Remote

Job Summary :

We are seeking an AI Quality Engineer with a strong automation skillset to ensure the robustness, performance, and reliability of our AI systems and services. The ideal candidate is tech-savvy, proactive, and passionate about quality at every stepfrom initial design through deployment and ongoing monitoring. You will play a key role in building and maintaining a highly automated testing infrastructure to support fast, reliable model and pipeline delivery as the company scales.

Key Responsibilities :

Testing Expertise :


- Conduct comprehensive testing across all layers, including server load, integration points, and output quality.

- Apply Test Driven Development (TDD) principlesanticipate, design, and define all necessary tests before the start of feature development.

- Identify what needs to be tested and proactively communicate requirements before build phases.

Automation-First Approach :


- Develop, maintain, and extend a fully automated testing suite that covers unit, integration, performance, and end-to-end testing.

- Emphasize automation to minimize manual intervention and maximize test coverage, reliability, and repeatability.

DevOps & CI/CD Integration :


- Collaborate closely with DevOps to ensure all tests (including those for model deployment and data pipelines) are tightly integrated with modern CI/CD workflows.

- Streamline rapid yet safe releases through automation and timely feedback.

Automated Testing Frameworks :


- Extensive hands-on experience with frameworks such as Pytest (Python testing), Playwright (end-to-end browser testing), Postman (API testing), and Langfuse (LLM output tracking/testing).

- Implement and maintain robust API contract testing to ensure reliable interactions between services.

Manual & LLM Testing :


- Execute manual test cases with strong attention to detail, especially for evaluating Large Language Model (LLM) output quality.

- Flag issues such as hallucinations, factual inaccuracies, or unexpected edge case responses.

- Continuously update manual testing strategies to adapt to evolving model behaviors and business requirements.

Monitoring, Observability & Post-Deploy Quality :


- Configure, deploy, and interpret dashboards from monitoring tools like Prometheus, Grafana, and CloudWatch.

- Track model health, pipeline performance, error rates, and system anomalies after deployment.

- Proactively investigate and triage quality issues uncovered in production.

Core Abilities and Technical Skills :


- Deep practical knowledge in test automation, performance, and reliability engineering.

- In-depth experience integrating tests into CI/CD pipelines, especially for machine learning and AI model workflows.

- Hands-on proficiency in automated QA tools : Pytest, Playwright, Postman, Langfuse, and similar.

- Solid foundation in manual exploratory testing, particularly for complex and evolving outputs such as those from LLMs.

- Expertise in monitoring, APM, and observability tools (e.g., Prometheus, Grafana, CloudWatch).

- Demonstrated strong problem-solving skillsanticipate, identify, and resolve issues early.

- Strong communication skills to clearly articulate requirements, quality risks, and advocate for automation-driven quality throughout the organization.

Mindset :

- Automation-First : Relentless emphasis on driving automation over manual effort.

- Proactive : Anticipates issues and testing needs; does not wait to be told what to test.

- Quality Advocate : Champions testing best practices and designs processes to catch bugs before production.

- Curious & Continuous Learner : Seeks out new tools, stays current with testing frameworks and industry best practices.

- Collaborative : Partners effectively with product, engineering, and DevOps teams to deliver high-quality models and features at scale

(ref:hirist.tech)

  • QA Engineer Test Automation

    6 days ago

    Delhi, Delhi, India MWIDM Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    We're Hiring: QA Engineer – Test Automation (Playwright) Location: PAN India Experience: 5+ YearsAre you passionate about delivering high-quality user experiences through cutting-edge automation testing? We're looking for a QA Engineer who brings deep knowledge of testing frameworks, strong automation skills, and a pragmatic approach to quality assurance....

  • QA Automation

    2 weeks ago

    Delhi, Delhi, India Mount Talent Consulting Pvt Ltd. Full time

    Hi Folks,We are looking for QA Automation Engineer for one of the leading organization on the Payroll of Mount Talent Consulting.Please find below the Job Description:Sr. QA Automation6+ years of professional experience in testing software, preferably in an Agile environment (Scrum, Kanban)Typescript, NodeJS, Playwright - mandatory.Java, Selenium - nice to...

  • Lead QA Engineer

    2 weeks ago

    Delhi, Delhi, India CES Full time

    We are looking for a seasoned Principal QA Engineer to lead our QA strategy and raise the bar on product quality. This role offers the chance to work on diverse projects, driving both manual and automated testing in a fast-paced environment.What you'll do:Lead test strategy, automation, and performance testing efforts.Mentor engineers and champion best...

  • QA Engineer

    6 days ago

    Delhi, Delhi, India MWIDM Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    We're Hiring: QA Engineer (Playwright) Location: PAN India Experience: 5+ YearsAre you passionate about delivering high-quality user experiences through cutting-edge automation testing? We're looking for a QA Engineer who brings deep knowledge of testing frameworks, strong automation skills, and a pragmatic approach to quality assurance. What You'll...

  • Python Automation QA Engineer

    2 weeks ago

    Delhi, Delhi, India ITC Infotech Full time

    Note : Need immediate to 30 days joiners.Urgent requirement for Testing - Python sfdcThe ideal candidate will be strong in "python testing ,pytest, with SFDC /Salesforce".Role 1 – QA Engineer (Python & AI Agentic Solutions)ResponsibilitiesDesign, code, test and manage various applicationsCollaborate with engineering team and product team to establish best...

  • QA Engineer

    4 days ago

    Delhi, Delhi, India NikahForever Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Company DescriptionNikahForever is India's most trusted Muslim Matrimony platform dedicated to making marriages convenient and fostering community building. Since our founding, we have achieved over 55,000 successful marriage stories and have grown to over 2,000,000 members. Our platform uses an AI-based algorithm to simplify the process of searching and...

  • QA Automation Lead

    1 week ago

    Delhi, Delhi, India beBeeAutomation Full time ₹ 18,00,000 - ₹ 22,00,000

    Senior QA Automation EngineerWe are seeking a highly skilled Senior QA Automation Engineer to join our quality engineering team. The ideal candidate will bring strong expertise in Salesforce and nCino, with hands-on automation experience and deep knowledge of testing frameworks.Key ResponsibilitiesDesign and implement test automation frameworks for...

  • QA Automation Engineer

    7 days ago

    Delhi, Delhi, India Addendum Full time US$ 90,000 - US$ 1,20,000 per year

    ABOUT THE COMPANY: Join ADDENDUM, a global software development and IT team augmentation firm focusing on fintech, banking, and telco industries. Experience tailored benefits, diverse projects with various tech stacks, and opportunities for growth, freedom, and responsibility you desire. ABOUT THE ROLE: Were seeking a skilled QA Automation Engineer to...

  • AI Engineer

    3 weeks ago

    Delhi, Delhi, India Darwix AI Full time

    Job descriptionJob Title: AI EngineerCompany : Darwix AILocation : Gurgaon (On-site)Type : Full-TimeExperience : 2-6 YearsLevel : Senior LevelAbout Darwix AIDarwix AI is one of India's fastest-growing GenAI startups, revolutionizing the future of enterprise sales and customer engagement with real-time conversational intelligence. We are building a...

  • Automation Test Engineer

    7 days ago

    Delhi, Delhi, India One Click AI Full time

    Job Description : Automation Test Engineer (Frontend & Backend)Location : Bengaluru (Fully Remote)Experience : 13 yearsNotice Period : ImmediateEmployment Type : 6-month Contract (extendable based on performance & project requirements)Compensation : 6 -10 LPARole OverviewWe are seeking a skilled and detail-oriented Software Development Engineer in Test...