AI QA Automation Engineer

4 weeks ago

Delhi, Delhi, India Aidewiser Soltek Full time

Title : AI QA Automation Engineer

Location : Remote

Job Summary :

We are seeking an AI Quality Engineer with a strong automation skillset to ensure the robustness, performance, and reliability of our AI systems and services. The ideal candidate is tech-savvy, proactive, and passionate about quality at every stepfrom initial design through deployment and ongoing monitoring. You will play a key role in building and maintaining a highly automated testing infrastructure to support fast, reliable model and pipeline delivery as the company scales.

Key Responsibilities :

Testing Expertise :


- Conduct comprehensive testing across all layers, including server load, integration points, and output quality.

- Apply Test Driven Development (TDD) principlesanticipate, design, and define all necessary tests before the start of feature development.

- Identify what needs to be tested and proactively communicate requirements before build phases.

Automation-First Approach :


- Develop, maintain, and extend a fully automated testing suite that covers unit, integration, performance, and end-to-end testing.

- Emphasize automation to minimize manual intervention and maximize test coverage, reliability, and repeatability.

DevOps & CI/CD Integration :


- Collaborate closely with DevOps to ensure all tests (including those for model deployment and data pipelines) are tightly integrated with modern CI/CD workflows.

- Streamline rapid yet safe releases through automation and timely feedback.

Automated Testing Frameworks :


- Extensive hands-on experience with frameworks such as Pytest (Python testing), Playwright (end-to-end browser testing), Postman (API testing), and Langfuse (LLM output tracking/testing).

- Implement and maintain robust API contract testing to ensure reliable interactions between services.

Manual & LLM Testing :


- Execute manual test cases with strong attention to detail, especially for evaluating Large Language Model (LLM) output quality.

- Flag issues such as hallucinations, factual inaccuracies, or unexpected edge case responses.

- Continuously update manual testing strategies to adapt to evolving model behaviors and business requirements.

Monitoring, Observability & Post-Deploy Quality :


- Configure, deploy, and interpret dashboards from monitoring tools like Prometheus, Grafana, and CloudWatch.

- Track model health, pipeline performance, error rates, and system anomalies after deployment.

- Proactively investigate and triage quality issues uncovered in production.

Core Abilities and Technical Skills :


- Deep practical knowledge in test automation, performance, and reliability engineering.

- In-depth experience integrating tests into CI/CD pipelines, especially for machine learning and AI model workflows.

- Hands-on proficiency in automated QA tools : Pytest, Playwright, Postman, Langfuse, and similar.

- Solid foundation in manual exploratory testing, particularly for complex and evolving outputs such as those from LLMs.

- Expertise in monitoring, APM, and observability tools (e.g., Prometheus, Grafana, CloudWatch).

- Demonstrated strong problem-solving skillsanticipate, identify, and resolve issues early.

- Strong communication skills to clearly articulate requirements, quality risks, and advocate for automation-driven quality throughout the organization.

Mindset :

- Automation-First : Relentless emphasis on driving automation over manual effort.

- Proactive : Anticipates issues and testing needs; does not wait to be told what to test.

- Quality Advocate : Champions testing best practices and designs processes to catch bugs before production.

- Curious & Continuous Learner : Seeks out new tools, stays current with testing frameworks and industry best practices.

- Collaborative : Partners effectively with product, engineering, and DevOps teams to deliver high-quality models and features at scale

(ref:hirist.tech)

  • QA Engineer Test Automation

    6 days ago

    Delhi, Delhi, India MWIDM Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We're Hiring: QA Engineer – Test Automation (Playwright) Location: PAN India Experience: 5+ YearsAre you passionate about delivering high-quality user experiences through cutting-edge automation testing? We're looking for a QA Engineer who brings deep knowledge of testing frameworks, strong automation skills, and a pragmatic approach to quality assurance....

  • AI Automation Engineer

    2 days ago

    Delhi, Delhi, India Ensavior Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    About the RoleWe are looking for a skilled AI Automation Engineer to lead the design and implementation of automation across our business processes. Initially, the focus will be on automating proposal and quotation workflows, but over time, this role will expand to cover other areas such as HR, Finance, Operations, and Customer Support.The ideal candidate...

  • QA Engineer

    6 days ago

    Delhi, Delhi, India MWIDM Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    We're Hiring: QA Engineer (Playwright) Location: PAN India Experience: 5+ YearsAre you passionate about delivering high-quality user experiences through cutting-edge automation testing? We're looking for a QA Engineer who brings deep knowledge of testing frameworks, strong automation skills, and a pragmatic approach to quality assurance. What You'll...

  • QA Automation Engineer

    7 days ago

    Delhi, Delhi, India Addendum Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    ABOUT THE COMPANY: Join ADDENDUM, a global software development and IT team augmentation firm focusing on fintech, banking, and telco industries. Experience tailored benefits, diverse projects with various tech stacks, and opportunities for growth, freedom, and responsibility you desire. ABOUT THE ROLE: Were seeking a skilled QA Automation Engineer to...

  • QA Engineer

    4 days ago

    Delhi, Delhi, India NikahForever Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Company DescriptionNikahForever is India's most trusted Muslim Matrimony platform dedicated to making marriages convenient and fostering community building. Since our founding, we have achieved over 55,000 successful marriage stories and have grown to over 2,000,000 members. Our platform uses an AI-based algorithm to simplify the process of searching and...

  • Automation Test Engineer

    4 weeks ago

    Delhi, Delhi, India One Click AI Full time

    Job Description : Automation Test Engineer (Frontend & Backend)Location : Bengaluru (Fully Remote)Experience : 13 yearsNotice Period : ImmediateEmployment Type : 6-month Contract (extendable based on performance & project requirements)Compensation : 6 -10 LPARole OverviewWe are seeking a skilled and detail-oriented Software Development Engineer in Test...

  • Quality Assurance Automation Engineer

    3 weeks ago

    Delhi, Delhi, India TestGrid Full time

    Job Title: Quality Assurance Automation EngineerLocation: Remote (India)Employment Type: Full-timeJob Description Company Description TestGrid is a leading provider of end-to-end automation cloud and on-premise testing solutions. The company focuses on simplifying the testing process with innovative AI-powered technology to help organizations achieve time...

  • QA Automation Intern Engineer

    2 days ago

    Delhi, Delhi, India Xcrino Business Solutions Full time ₹ 96,000 - ₹ 1,20,000 per year

    Job Title: Selenium Tester InternLocation: Dwarka, Sector 7, New DelhiStipend: ₹8,000 – ₹10,000 per monthDuration: 6 MonthsInternship Type: Full-time (Work from Office)Contact No: About the Role:We are looking for a passionate Selenium Tester Intern to join our QA team. The ideal candidate should have a basic understanding of automation testing,...

  • AI Engineer

    1 week ago

    Delhi, Delhi, India Weekday AI (YC W21) Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    This role is for one of the Weekday's clientsMin Experience: 4 yearsLocation: Gurgaon, NCRJobType: full-timeWe are seeking a highly skilled and innovativeAI Engineerwith 4-8 years of hands-on experience in artificial intelligence, machine learning, and deep learning. This role requires a professional who can design, build, and deploy AI-driven solutions that...

  • QA Engineer

    2 days ago

    Delhi, Delhi, India HT Digital Streams Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Location:Delhi/NCR (Hybrid)Experience:6 months to 2 yearsEmployment Type:Full-timeAbout UsHT Digital Streams is a leading digital media company, shaping the future of content delivery and audience engagement through cutting-edge technology and innovation.Role OverviewWe are looking for a detail-oriented and enthusiasticQA Engineerto join our dynamic...