AI QA Test Engineer

2 days ago

Cochin, Kerala, India ORION SYSTEMS Full time ₹ 12,00,000 - ₹ 36,00,000 per year

About the Role

Experienced AI QA Test Engineer to ensure the quality, reliability, and ethical performance of AI-driven systems and applications. This role requires a deep understanding of AI/ML workflows, data validation, model testing techniques, and automation frameworks. The ideal candidate is highly analytical, detail-oriented, and passionate about building trusted AI products.

Participates in QA activities on project assignments and ensures timely delivery on QA commitments in a fast-paced product development environment.

Review of functional requirements and technical design documentation and creation of test plans, test scenarios and test cases. Execution of test plans and certification of application releases

Key Responsibilities:

AI/ML Testing

  • Validate ML models, data pipelines, and inference APIs for accuracy, fairness, and robustness
  • Evaluate model behavior across diverse data sets, edge cases, and unexpected inputs
  • Monitor model drift, performance degradation, and bias over time
  • Conduct prompt testing, hallucination detection, and safety validation for LLM-based systems
  • Experience of testing implementation of AI/ML into application
  • Manual Testing with automation exposure
  • API testing with SOAP UI/ Postman
  • Agile methodology experience
  • Test & Defect management with JIRA/ Azure DevOps (ADO)
  • DB Testing Experience  - Good to Have
  • Tools - Deepeval/ Langfuse/ Pytest

Quality Engineering & Automation

  • Build and maintain automated test suites for AI features
  • Develop test scripts for ML workflows using Python, Playwright, and CI/CD tools
  • Design and execute manual and automated tests for functional, performance, and security scenarios
  • Identify defects, establish test metrics, reporting dashboards, and test traceability

Data Quality & Responsible AI

  • Validate training and inference data quality, labeling consistency, and data governance
  • Work with data science and product teams to implement responsible AI testing guidelines
  • Document and escalate risks related to bias, privacy, explainability, and compliance

Cross-Functional Collaboration

  • Partner with Data Scientists, ML Engineers, and Product Managers to define acceptance criteria
  • Participate in design reviews, model evaluations, and release readiness checkpoints
  • Contribute to QA strategy and Test Plans

Required Qualifications

  • Bachelor's degree in computer science, Engineering, or related field (or equivalent experience)
  • 4+ years of experience in software QA engineering
  • 2+ years working with AI/ML or data-centric systems
  • Strong Python programming skills
  • Proficiency in automation frameworks (e.g., Playwright, WebdriverIO, Selenium)
  • Experience with ML platforms (TensorFlow, PyTorch, Hugging Face, MLflow, etc.)
  • Hands-on experience testing APIs, data pipelines, and distributed systems
  • Exposure to RAG pipelines and agent frameworks
  • Familiarity with prompt testing and LLM evaluation techniques
  • Experience with model evaluation metrics (precision, recall, F1, ROC-AUC, BLEU, perplexity, etc.)
  • Knowledge of MLOps, CI/CD, and cloud platforms (AWS, Azure, or GCP)
  • Familiarity with Responsible AI frameworks (NIST AI RMF, ISO/IEC standards, fairness toolkits)
  • Experience testing vector databases and AI search (Pinecone, Elasticsearch, Redis Vector)

Key Skills

  • AI/ML testing & analysis
  • Automation & scripting
  • Data validation & pipelines
  • Risk, ethics, & compliance mindset
  • Strong problem-solving & debugging skills
  • Excellent documentation & communication skills
  • Working with distributed teams

  • AI Engineer

    1 week ago

    Cochin, Kerala, India Testing Mavens Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    About UsTesting Mavensis a next-generation QA & Software Testing Services provider headquartered in New Jersey, USA, with offshore facilities in India. We have deep expertise in various spheres like Finance and Advisory, EdTech, Fashion and Pharmacy, etc. We partner with massive global brands to deliver excellent testing experience.Looking for an AI Engineer...

  • QA Engineer

    2 weeks ago

    Cochin, Kerala, India Voyon Group | Voyon Folks HRMS Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    We are looking for a detail-oriented and methodicalQuality Assurance (QA) Engineerto join our team. The ideal candidate will be responsible for ensuring that our products and systems meet the highest standards of quality through rigorous testing, analysis, and collaboration with development teams.Key Responsibilities:Design, develop and execute test plans,...

  • SB-1358 QA Engineer

    2 weeks ago

    Cochin, Kerala, India Softobiz Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Job Title:QA Engineer (Automation Testing)Location:KochiJob Type:Full-TimeExperience: 6 - 9 yearsJob SummaryWe are looking for a highly skilled QA Engineer with expertise in both manual and automation testing to join our team. The ideal candidate will be responsible for ensuring the quality and reliability of our software products through rigorous testing...

  • QA Engineer

    1 week ago

    Cochin, Kerala, India VOYON Technology Services Pvt Ltd Full time ₹ 12,00,000 - ₹ 18,00,000 per year

    We are looking for a detail-oriented and methodical Quality Assurance (QA) Engineer to join our team. The ideal candidate will be responsible for ensuring that our products and systems meet the highest standards of quality through rigorous testing, analysis, and collaboration with development teams.Key Responsibilities:Design, develop and execute test plans,...

  • QA Engineer

    4 days ago

    Cochin, Kerala, India Techrish Solutions Full time ₹ 45,000 - ₹ 85,000 per year

    Job description:Job Title: Senior QA EngineerLocation: ErnakulamEmployment Type: Full-TimeExperience: 2- 4 YearsWe are seeking a highly skilled Senior QA Engineer with 2–4 years of strong hands-on experience in both manual and automation testing. In this role, you will be a key contributor in ensuring our products meet the highest standards of quality,...

  • QA Engineer

    4 days ago

    Cochin, Kerala, India ewaantech Full time ₹ 2,40,000 - ₹ 2,52,000 per year

    Position: QA EngineerDepartment: Engineering / Quality AssuranceLocation: CochinEmployment Type: Full-timeAbout EwaantechEwaantech is a top mobile app development and website development company in Saudi Arabia and UAE. We provide additional services including web apps and e-commerce development. Ewaantech is a one-stop solution for any IT related...

  • QA Engineer

    2 days ago

    Cochin, Kerala, India Saasvaap Inc. Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Company DescriptionAt Saasvaap, we specialize in delivering next-generation software solutions that help businesses simplify, scale, and succeed. We are a technology company focused on creating powerful digital ecosystems through custom SaaS products, web and mobile application development, cloud solutions, and Robotic Process Automation (RPA). With a...

  • Senior Software Development Engineer in Test

    2 weeks ago

    Cochin, Kerala, India NOV Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    DescriptionWe are seeking a highly skilled Senior Software Development Engineer in Test to join our Quality Engineering team. The successful candidate will be responsible in designing and developing scalable test automation frameworks, ensuring the quality, performance, and reliability of our software solutions. This role involves close collaboration with...

  • QA Engineer

    2 days ago

    Cochin, Kerala, India AMT ERP Full time ₹ 3,00,000 - ₹ 5,00,000 per year

    AMTERP Solutions India Pvt Ltd.Role - QA EngineerWork Location - KochiNote: Immediate Joiners OnlyQA EngineerWe are seeking a QA to join our team. The ideal candidate should possess 1 - 3 years of experience in software testing and be ISTQB certified.Please find the JD and Skills mentioned below.Key Responsibilities –Collaborate with cross-functional teams...

  • QA Engineer

    4 days ago

    Cochin, Kerala, India Saasvaap Inc. Full time ₹ 6,00,000 - ₹ 8,00,000 per year

    Company DescriptionSaasvaap Inc. specializes in creating advanced software solutions to help businesses streamline operations, scale efficiently, and achieve success. Our expertise includes SaaS product development, web and mobile application development, cloud integration, and Robotic Process Automation (RPA). We are committed to delivering tailor-made,...