AI QA Test Engineer

3 days ago

Cochin, Kerala, India ORION SYSTEMS Full time ₹ 12,00,000 - ₹ 36,00,000 per year

About the Role

Experienced AI QA Test Engineer to ensure the quality, reliability, and ethical performance of AI-driven systems and applications. This role requires a deep understanding of AI/ML workflows, data validation, model testing techniques, and automation frameworks. The ideal candidate is highly analytical, detail-oriented, and passionate about building trusted AI products.

Participates in QA activities on project assignments and ensures timely delivery on QA commitments in a fast-paced product development environment.

Review of functional requirements and technical design documentation and creation of test plans, test scenarios and test cases. Execution of test plans and certification of application releases

Key Responsibilities:

AI/ML Testing

  • Validate ML models, data pipelines, and inference APIs for accuracy, fairness, and robustness
  • Evaluate model behavior across diverse data sets, edge cases, and unexpected inputs
  • Monitor model drift, performance degradation, and bias over time
  • Conduct prompt testing, hallucination detection, and safety validation for LLM-based systems
  • Experience of testing implementation of AI/ML into application
  • Manual Testing with automation exposure
  • API testing with SOAP UI/ Postman
  • Agile methodology experience
  • Test & Defect management with JIRA/ Azure DevOps (ADO)
  • DB Testing Experience  - Good to Have
  • Tools - Deepeval/ Langfuse/ Pytest

Quality Engineering & Automation

  • Build and maintain automated test suites for AI features
  • Develop test scripts for ML workflows using Python, Playwright, and CI/CD tools
  • Design and execute manual and automated tests for functional, performance, and security scenarios
  • Identify defects, establish test metrics, reporting dashboards, and test traceability

Data Quality & Responsible AI

  • Validate training and inference data quality, labeling consistency, and data governance
  • Work with data science and product teams to implement responsible AI testing guidelines
  • Document and escalate risks related to bias, privacy, explainability, and compliance

Cross-Functional Collaboration

  • Partner with Data Scientists, ML Engineers, and Product Managers to define acceptance criteria
  • Participate in design reviews, model evaluations, and release readiness checkpoints
  • Contribute to QA strategy and Test Plans

Required Qualifications

  • Bachelor's degree in computer science, Engineering, or related field (or equivalent experience)
  • 4+ years of experience in software QA engineering
  • 2+ years working with AI/ML or data-centric systems
  • Strong Python programming skills
  • Proficiency in automation frameworks (e.g., Playwright, WebdriverIO, Selenium)
  • Experience with ML platforms (TensorFlow, PyTorch, Hugging Face, MLflow, etc.)
  • Hands-on experience testing APIs, data pipelines, and distributed systems
  • Exposure to RAG pipelines and agent frameworks
  • Familiarity with prompt testing and LLM evaluation techniques
  • Experience with model evaluation metrics (precision, recall, F1, ROC-AUC, BLEU, perplexity, etc.)
  • Knowledge of MLOps, CI/CD, and cloud platforms (AWS, Azure, or GCP)
  • Familiarity with Responsible AI frameworks (NIST AI RMF, ISO/IEC standards, fairness toolkits)
  • Experience testing vector databases and AI search (Pinecone, Elasticsearch, Redis Vector)

Key Skills

  • AI/ML testing & analysis
  • Automation & scripting
  • Data validation & pipelines
  • Risk, ethics, & compliance mindset
  • Strong problem-solving & debugging skills
  • Excellent documentation & communication skills
  • Working with distributed teams

  • QA Engineer

    2 weeks ago

    Cochin, Kerala, India VidPro Consultancy Services Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    About The RoleWe are looking for talented QA professionals at multiple levels (Lead, Sr. QA, and QA Engineer) to join our team. The ideal candidates should have strong experience inData Testing, UI Testing, and Automation, with exposure toBig Data projects (Azure DBT, MSBI, etc.). Candidates should also be open to learning and adoptingAI-driven testing...

  • Jr QA Analyst

    2 weeks ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Position Purpose This role carries out the functions of a QA functional for a POD. The purpose of this position is to be a hands-on QA engineer ensuring all functional testing requirements are met and to provide support to Test Automation Engineer. Key Responsibilities Support QA effort and direction within allocated team and offshore group Coordinate...

  • QA Lead

    2 weeks ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Role Overview:Responsible for planning, coordinating, and overseeing the testing activities within the project lifecycle. This role involves leading a team of QA engineers, ensuring quality standards are met, and collaborating closely with development, business, and product teams.Key Responsibilities:Test Planning & Strategy:Develop detailed test plans and...

  • QA Engineer

    2 weeks ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 5,00,000 - ₹ 8,00,000 per year

    Dbiz is a high-performing product and engineering company that partners with organizations to build digital solutions using the right technology at the right time. We pride ourselves on our innovative approach, leveraging technology in diverse and impactful ways.Role Overview:You will work within a platform team (10 – 14) people, responsible for new...

  • Senior QA Automation Engineer

    5 days ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 1,50,000 - ₹ 28,00,000 per year

    About the Company:Dbiz is a high-performing product & engineering company that partners up with organisations to help build out digital solutions with the right technology at the right time, whilst being innovative by using technology in different ways.Experience Level: 6+ yearsJob Description:We are seeking a highly skilled and experienced QA Automation...

  • Salesforce QA- Lead

    2 weeks ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    About the Company:Dbiz is a high-performing product and engineering company that partners with organizations to build cutting-edge digital solutions. We leverage the right technologies at the right time to drive innovation and efficiency, utilizing technology in unique ways to achieve exceptional outcomes.Experience Level:Senior (8+ years)Job Description:We...

  • QA Analyst- Inspection POD

    3 days ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Position Summary:We are seeking a skilled QA Functional Engineer to support a product delivery POD. The ideal candidate will have strong hands-on experience in functional testing, both manual and automated, with a focus on quality assurance best practices across mobile and web platforms.Key Responsibilities:Lead and support QA efforts within the team and...

  • Automation QA

    2 weeks ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    We are looking for a skilled Automation QA Engineer to join our dynamic QA team. The ideal candidate will have a strong background in automated testing, with hands-on experience in Selenium, Java, and Mobile Web Testing. You will be responsible for developing and maintaining automated test scripts, executing test plans, and ensuring the quality of both web...

  • SB-1358 QA Engineer

    3 days ago

    Cochin, Kerala, India Softobiz Full time ₹ 4,50,000 - ₹ 9,00,000 per year

    Job Title: QA Engineer (Automation Testing) Location: Kochi Job Type: Full-Time Experience: 6 - 9 years Job Summary: We are looking for a highly skilled QA Engineer with expertise in both manual and automation testing to join our team. The ideal candidate will be responsible for ensuring the quality and reliability of our software products through rigorous...

  • Lead QA Tester

    2 weeks ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    About the Company:Dbiz is a high-performing product & engineering company that partners up with organisations to help build out digital solutions with the right technology at the right time, whilst being innovative by using technology in different ways.Experience Level: Intermediate (6+ years)Job Description:We are seeking a highly skilled and experienced QA...