AI Testing Engineer

2 weeks ago

Noida, Uttar Pradesh, India Dailoqa Full time ₹ 8,00,000 - ₹ 24,00,000 per year

Job Title: Functional Tester – (Banking & Wealth Management)

Location
: Noida

Experience Level
: 4–6 years

Reports To
: QA Lead / Product Manager

We are looking for an experienced 
Functional Tester
 to validate the business workflows, compliance logic, and customer-facing features of an 
Agentic AI solution
 built for the 
banking and wealth management domain
. This solution leverages 
LLMs, retrieval-augmented generation (RAG)
, and 
domain ontologies
 to provide intelligent data retrieval, client interaction, and regulatory compliance automation.

Key Responsibilities

Functional Testing

  • Understand 
    functional and non-functional requirements
     for Agentic AI features (e.g., document Q&A, investment summarization, client onboarding, compliance tracking)
  • Validate 
    end-to-end user journeys
     across banking and wealth advisory use cases
  • Test conversational workflows and 
    agentic logic flows
     (multi-step LLM tasks)
  • Verify outputs from LLMs for 
    relevance, consistency, and context-awareness
  • Ensure correct 
    role-based access
     across advisors, relationship managers, and compliance officers

Test Planning & Execution

  • Prepare 
    detailed test plans, test cases
    , and 
    traceability matrices
     from EPICs, features, and user stories
  • Execute 
    manual testing
     for user interfaces, APIs, and chat/agent interactions
  • Perform 
    regression testing
     across AI pipelines after prompt or model updates
  • Log, track, and retest defects using tools like 
    Azure DevOps
    , 
    Jira
    , or 
    TestRail

AI Model & RAG Testing

  • Validate grounding of LLM responses from 
    vector databases (e.g., Azure AI Search, Pinecone)
  • Check performance of 
    RAG pipelines
     including hallucination mitigation, context window boundaries, and fallback logic
  • Test interaction of 
    agent orchestration flows
     (LangChain/Autogen/semantic kernel)
  • Verify prompt-based test cases for accuracy and coverage

Integration & Data Testing

  • Validate 
    PDF/statement extraction logic
     from custodians (e.g., Morgan Stanley, Pershing)
  • Check structured data output (JSON, tables) from Azure Form Recognizer or OpenAI pipelines
  • Ensure 
    data integrity and lineage
     from ingestion → processing → user response

Must-Have Skills & Tools

Category

Tools/Tech

Test Management

Azure / AWA DevOps, Jira, TestRail, Azure Board

Test Types

Functional, UI, Integration, AI behavior

Domain Knowledge

Wealth Mgmt workflows, KYC, portfolio views, regulatory rules

AI Concepts

LLM output validation, RAG testing, prompt engineering basics

Data

JSON validation, SQL for test data setup

Tools (Optional)

Postman, Python (for test automation scripts), Excel-based test mapping

Preferred Qualifications

  • Bachelor's degree in Engineering, Finance, or Computer Science
  • Experience testing AI/ML solutions or conversational AI products
  • Familiarity with 
    Azure AI stack
    , OpenAI, or AWS Bedrock
  • Exposure to 
    compliance-heavy domains
     like banking, wealth, or insurance
  • Ability to work with cross-functional Agile teams (product, engineering, AI/ML, compliance)

Nice-to-Have Skills

  • Experience writing 
    prompt-based test cases
     for LLMs

  • AI Testing Engineer

    4 days ago

    Noida, Uttar Pradesh, India Crowe Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Your Journey at Crowe Starts Here:At Crowe, you can build a meaningful and rewarding career. With real flexibility to balance work with life moments, you're trusted to deliver results and make an impact. We embrace you for who you are, care for your well-being, and nurture your career. Everyone has equitable access to opportunities for career growth and...

  • AI Analyst

    1 week ago

    Noida, Uttar Pradesh, India Level AI Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Level AI was founded in 2019 and is a Series C startup headquartered in Mountain View, California. Level AI revolutionises customer engagement by transforming contact centres into strategic assets. Our AI-native platform leverages advanced technologies such as Large Language Models to extract deep insights from customer interactions. By providing actionable...

  • Senior Backend Engineer

    4 days ago

    Noida, Uttar Pradesh, India Level AI Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Level AI was founded in 2019 and is a Series C startup headquartered in Mountain View, California. Level AI revolutionizes customer engagement by transforming contact centers into strategic assets. Our AI-native platform leverages advanced technologies such as Large Language Models to extract deep insights from customer interactions. By providing actionable...

  • Senior Backend Engineer

    4 days ago

    Noida, Uttar Pradesh, India Level AI Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Level AI was founded in 2019 and is a Series C startup headquartered in Mountain View, California. Level AI revolutionizes customer engagement by transforming contact centers into strategic assets. Our AI-native platform leverages advanced technologies such as Large Language Models to extract deep insights from customer interactions. By providing actionable...

  • AI developer intern

    2 weeks ago

    Noida, Uttar Pradesh, India Ropeli AI Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    Company DescriptionRopeli AIis an intelligent solutions platform powered by prompts, but what makes us unique is our ability to 'untie the ropes' around problems just like a tied rope that restricts, we unravel complex challenges and deliver precise, actionable solutions.Role DescriptionThis is an AI Developer Intern, will be responsible for tasks such as...

  • Sr Test Automation Engineer

    2 days ago

    Noida, Uttar Pradesh, India Weekday AI Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    This role is for one of the Weekday's clientsMin Experience: 8 yearsLocation: NCR, Gurgaon, Delh, New Delhi, NoidaJobType: full-timeWe are seeking a highly skilled Senior Test Automation Engineer with strong expertise in Python, ETL testing, cloud technologies, and automation frameworks. The ideal candidate should excel in hands-on testing, data validation,...

  • AI Engineer

    1 week ago

    Noida, Uttar Pradesh, India FCS Software Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role: AI EngineerLocation: remoteTerm: Long termWorking times: US EST hoursRole PurposeEnsure the AI models powering the platform remain accurate, aligned, and continuously improved to meet business and compliance requirements, while leveraging data science methods to drive insights, performance improvements, and informed decision-making.Key...

  • Senior Frontend Engineer

    4 days ago

    Noida, Uttar Pradesh, India Level AI Full time ₹ 10,00,000 - ₹ 12,00,000 per year

    Level AI was founded in 2019 and is a Series C startup headquartered in Mountain View, California. Level AI revolutionizes customer engagement by transforming contact centers into strategic assets. Our AI-native platform leverages advanced technologies such as Large Language Models to extract deep insights from customer interactions. By providing actionable...

  • Etl Test Engineer

    9 hours ago

    Greater Noida, Uttar Pradesh, India Coforge Full time ₹ 1 - ₹ 2 per year

    Role Overview:We are seeking an experienced ETL Test Lead with strong expertise in Quality Engineering and advanced AI/ML capabilities. The ideal candidate will lead testing strategies for complex data pipelines and AI-driven solutions, ensuring high-quality, scalable, and secure systems.Key Responsibilities:Design and implement ETL testing strategies for...

  • AI Engineer

    6 days ago

    Noida, Uttar Pradesh, India InvarSys Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title:AI EngineerLocation:Noida, U.PJob Type:Full-timeExperience:4+ YearsAbout the RoleWe are seeking a highly motivated and curious AI Engineer to join our team and contribute to building cutting-edge AI-driven solutions. In this role, you will work on generative AI models, customize and fine-tune LLMs, develop scalable data pipelines, and help deploy...