Software Engineer_AI Tester

9 hours ago

Bhubaneshwar, Odisha, India bebo Technologies Full time

Job Requirement

  • BE, B.Tech, M.Tech, MCA or equivalent degree in Computer Science (or related field).
  • 3-5 years of QA experience, with at least 2 years focused on AI/ML testing (LLMs, RAG, AI agents).
  • Proficiency in Artificial Intelligence, Machine Learning, AI Agents, and Large Language Models (LLMs).
  • Strong knowledge of QA methodologies, test design, functional/non-functional testing, and defect lifecycle management.
  • Solid understanding of LLM evaluation, hallucination types, prompt behavior, response scoring, and quality metrics.
  • Good understanding of RAG concepts including vector databases, embeddings, and retrieval relevance.
  • Coding proficiency in Python or Java (both preferred).
  • Hands-on experience in Web/API automation using frameworks like Playwright, Selenium, or REST Assured.
  • Experience in writing automation scripts, maintaining test frameworks, and integrating test suites into CI/CD pipelines.
  • Ability to analyze large sets of AI outputs for patterns and systemic issues.
  • Excellent analytical reasoning, problem-solving, and communication skills.

Job Responsibilitie

  • Test and validate LLM outputs, ensuring accuracy, correctness, completeness, consistency, usability, and hallucination analysis
  • Evaluate RAG systems, including retrieval accuracy, document relevance, context construction, and full response generation flows
  • Test AI agents and autonomous workflows, validating decision-making, task execution, and error handling
  • Design and execute AI-specific test strategies: dataset creation, edge-case testing, adversarial testing, pattern-based testing, and regression validation
  • Develop evaluation frameworks, scoring rubrics, and benchmarking models for AI quality assessment
  • Analyze large volumes of AI-generated responses to identify patterns, root causes, and issue clusters, instead of isolated defects
  • Knowledge of testing conversational AI, workflows, or agent-based systems
  • Exposure to vector search tools and embedding quality analysis
  • Validate fixes using new examples from the same pattern category, ensuring true model improvement
  • Collaborate closely with AI/ML engineers, QA teams, and product managers to improve AI accuracy and performance
  • Contribute to continuous improvement of AI QA practices, automation, tools, and evaluation datasets

.

  • Qa Tester

    1 week ago

    Bhubaneshwar, Odisha, India 3 Point Human Capital Full time

    We are seeking a highly motivated and detail-oriented Software Tester with a strong background in Quality Assurance (QA) to join our dynamic team. The ideal candidate will be responsible for ensuring the quality of our software applications through thorough testing processes. This role requires a deep understanding of software testing methodologies,...

  • Human Resources

    2 weeks ago

    Bhubaneshwar, Odisha, India Intellizo Global Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Job Title: Human Resources (HR) Executive – Software Industry)Location: Bhubaneswar, OdishaExperience: Minimum 4+ years (Mandatory in Software / IT Industry)Employment Type: Full-timeAbout Intellizo:Intellizois a forward thinking software and AI start up company focused on delivering scalable and impactful digital system to the global clients.As we are...

  • Quality Analysis And Manual Tester

    4 days ago

    Bhubaneshwar, Odisha, India Nekss Full time

    Responsibilities:* Drop Your CV at * Report defects and suggest improvements* Collaborate with development team for resolution* Ensure software quality through testing methods

  • UFT Architect- Bhubaneswar

    1 week ago

    Bhubaneshwar, Odisha, India AQM Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Greetings from AQM TechnologiesWe are looking for experienced senior resources inUnified Functional Testing (UFT)forBhubaneswar location.Please find the Job Description as provided below -Role - UFT Test AnalystEducational Qualifications - B.E / B. Tech / M. Tech / M.Sc.IT / Computers / MBAExperience level - 10+ years and aboveLocation - Bhubaneswar,...

  • Software Tester

    1 week ago

    Bhubaneshwar, India Tata Consultancy Services Full time

    Experience - 4 to 10 yearsWalkin Location - BhubaneshwarSkill - ETL TestingShould have experience in database/ETL concepts & testing Should have experience in understanding business requirements and translating them (writing) and resolving complex sql queries. Should be able to write new code (SQL/java/python/excel macro) to compare two tables, create test...

  • Software Tester

    1 week ago

    Bhubaneshwar, India Tata Consultancy Services Full time

    Experience - 4 to 10 yearsWalkin Location - BhubaneshwarSkill - ETL TestingShould have experience in database/ETL concepts & testing Should have experience in understanding business requirements and translating them (writing) and resolving complex sql queries. Should be able to write new code (SQL/java/python/excel macro) to compare two tables, create test...

  • Software Tester

    7 days ago

    Bhubaneshwar, India Tata Consultancy Services Full time

    Experience - 4 to 10 years Walkin Location - Bhubaneshwar Skill - ETL Testing Should have experience in database/ETL concepts & testing Should have experience in understanding business requirements and translating them (writing) and resolving complex sql queries. Should be able to write new code (SQL/java/python/excel macro) to compare two tables, create...

  • UFT - Senior Test Analyst - Bhubaneswar- (WFO)

    2 weeks ago

    Bhubaneshwar, India AQM Technologies Full time

    Greetings from AQM Technologies!We are looking for experienced senior resources in Unified Functional Testing (UFT) for Bhubaneswar location.Please find the Job Description as provided below -Role - UFT Test AnalystEducational Qualifications - B.E / B. Tech / M. Tech / M.Sc.IT / Computers / MBAExperience level - 10+ years and aboveLocation - Bhubaneswar,...