AI - QA Engineer

At Techjays, we are driving the future of artificial intelligence with a bold mission to empower businesses worldwide by helping them build AI solutions that transform industries. As an established leader in the AI space, we combine deep expertise with a collaborative, agile approach to deliver impactful technology that drives meaningful change.

Our global team consists of professionals who have honed their skills at leading companies such as Google, Akamai, NetApp, ADP, Cognizant Consulting, and Capgemini. With engineering teams across the globe, we deliver tailored AI software and services to clients ranging from startups to large-scale enterprises.

Be part of a company that's pushing the boundaries of digital transformation. At Techjays, you'll work on exciting projects that redefine industries, innovate with the latest technologies, and contribute to solutions that make a real-world impact. Join us on our journey to shape the future with AI.

We are looking for a detail-oriented and curious AI QA Engineer to join our growing QA team. You will play a critical role in ensuring the quality, safety, and reliability of our AI-powered products and features. If you're passionate about AI, testing complex systems, and driving high standards of quality—this role is for you

Primary Skills: QA Automation, Python, API Testing, AI/ML Testing, Prompt Evaluation, Adversarial Testing, Risk-Based Testing, LLM-as-a-Judge, Model Metrics Validation, Test Strategy.

Secondary Skills: CI/CD Integration, Git, Cloud Platforms (AWS/GCP/Azure ML), MLFlow, Postman, Testim, Applitools, Collaboration Tools (Jira, Confluence), Synthetic Data Generation, AI Ethics & Bias Awareness.

Experience: Years

Coimbatore/ Remote

Foundational QA Skills

Strong knowledge of test design, defect management, and QA lifecycle .

Experience with risk-based testing and QA strategy.

AI/ML Knowledge

Basic understanding of machine learning workflows , training/inference cycles.

Awareness of AI quality challenges : bias, fairness, transparency.

Familiarity with AI evaluation metrics: accuracy, precision, recall, F1-score .

Hands-on with prompt testing , synthetic data generation , and non-deterministic behavior validation.

Python programming for test automation and data validation.

Hands-on experience with API testing tools (Postman, Swagger, REST clients).

PyTest , Playwright, Selenium).

Understanding of CI/CD pipelines and integration testing.

Basic understanding of cloud-based AI platforms (AWS Sagemaker, Azure ML, GCP Vertex AI).

Passion for learning new technologies and contributing to AI quality practices.

Design, write, and execute test plans and test cases for AI/ML-based applications.

Collaborate with data scientists, ML engineers, and developers to understand model behavior and expected outcomes.

Perform functional, regression, and exploratory testing on AI components and APIs.

Implement and run adversarial testing , edge cases, and out-of-distribution data scenarios.

Conduct prompt testing and evaluation for LLM (Large Language Model)-based applications.

Use LLM-as-a-Judge and AI tools to automate evaluation of AI responses where possible.

Validate data pipelines , datasets, and ETL workflows.

Track model performance metrics such as precision, recall, F1-score , and flag potential degradation.

Document defects, inconsistencies, and raise risks proactively with the team.

Paid holidays and flexible paid time away

Casual dress code & flexible working environment

Medical Insurance covering self & family up to 4 lakhs per person.

Work in an engaging, fast-paced environment with ample opportunities for professional development.

---