AI Validation Lead

2 weeks ago

bareilly, India beBeeAIValidator Full time

Lead QA & AI ValidatorWe're seeking a seasoned professional to lead quality assurance initiatives for cutting-edge Generative AI and Multi-Agent Systems. The ideal candidate will have expertise in Python automation, LLM evaluation, RAG pipelines, observability, adversarial testing, and Azure monitoring.Scale test automation initiatives with a focus on GenAI and AI systemsDesign and execute LLM evaluation frameworks using G-Eval, custom evaluators, and LLM-as-a-JudgeImplement RAG evaluation frameworks (RAGAS or similar)Build Python-based automation frameworks using PyTest and DeepEvalIntegrate automation into CI/CD pipelines using GitHub ActionsDesign and validate multi-agent evaluation pipelines (tool usage, collaboration, reasoning chains)Perform adversarial and red-team testing: prompt injection, jailbreak attacks, bias, and toxicity detectionConduct API testing for microservices (REST, async workflows)Monitor applications using Azure Application Insights and Log AnalyticsDefine automated scoring systems for GenAI outputsManage synthetic datasets and golden datasets for AI validationImplement observability and trace monitoring using LangFuse, LangSmith, or similar toolsKey Responsibilities:Develop and maintain test automation frameworksCollaborate with cross-functional teams to design and implement AI validation pipelinesDesign and execute experiments to evaluate AI system performanceAnalyze results and provide actionable insights to improve AI system qualityStay up-to-date with emerging trends and technologies in AI validation and testingRequired Skills:Python programming languageTest automation frameworks (PyTest, DeepEval)LLM evaluation frameworks (G-Eval, custom evaluators, LLM-as-a-Judge)RAG evaluation frameworks (RAGAS or similar)CI/CD pipelines (GitHub Actions)Observability and trace monitoring (LangFuse, LangSmith)API testing (REST, async workflows)Azure monitoring (App Insights, Log Analytics)Automated scoring systems for GenAI outputsSynthetic dataset managementGolden dataset managementWhat We Offer:A challenging role in a dynamic and innovative environment with opportunities for growth and development.

  • Mid QA

    4 days ago

    Bareilly, India RapidBrains Full time

    Job Title: Mid QA – Automated Testing & AI Validation Experience: 3+ Years Employment Type: Contract (Short Term) Overview We are seeking a Mid-Level QA Engineer – Automated Testing & AI Validation with hands-on experience in testing AI-powered systems, particularly LLM-driven applications. The ideal candidate will have a strong QA automation background,...

  • AI Technology Innovation Lead

    2 weeks ago

    bareilly, India beBeeArtificialIntelligence Full time

    Job DescriptionLead an innovative team in advancing AI technology, driving technological advancements in Collateral Generation and Product Life Cycle management. Collaborate with top VCs and semiconductor pioneers to develop cutting-edge solutions.Develop and lead technology development in Gen AI and Agentic AI.Innovate and build high-performing solutions...

  • AI-Powered Psychometrics Specialist

    2 weeks ago

    bareilly, India beBeePsychometrics Full time

    AI-Powered Psychometrics SpecialistWe are seeking an expert in AI-driven innovation and psychometric expertise to design and improve our assessment tools.This hybrid role combines the development of innovative test items, leveraging AI tools for item generation and quality assurance, with collaboration across departments to ensure test content validity,...

  • Senior Risk Model Validation Specialist

    2 weeks ago

    bareilly, India beBeeAnalytics Full time

    Job Opportunity:The position of Senior Quantitative Analytics Associate involves leading independent validations and reviews of risk models to ensure they function as intended and comply with regulatory requirements.Carry out hands-on quantitative model validation/review tasks.Engage in a rigorous challenge throughout the model validation/review...

  • Senior Quantitative Analytics Lead

    2 weeks ago

    bareilly, India beBeeQuantitative Full time

    Job Description:As a Senior Quantitative Analytics Associate, you will be responsible for leading independent validations and reviews of the bank's various risk models.This role ensures that models are functioning as intended, comply with regulatory requirements, and that their risks are accurately identified, measured, and reported to senior management.You...

  • AI Model Developer

    2 weeks ago

    bareilly, India beBeeMachineLearning Full time

    We are seeking a highly skilled Data Scientist to lead our analytics and automation department. This role will involve designing, developing, and deploying scalable Machine Learning and AI models.Key Responsibilities:Design and develop scalable Machine Learning and AI models for data-driven insights and business process optimization.Analyze and interpret...

  • Senior Quality Assurance and AI Testing Leader

    2 weeks ago

    bareilly, India beBeeDevelopment Full time

    Job Opportunity: Director – QAThis is a senior leadership role within the AI/Data Center of Excellence (CoE). The position involves leading the Quality Assurance and Testing organization to ensure delivery of reliable, high-performing, and compliant AI and data-driven systems across the enterprise.Key Responsibilities:Quality & Testing Leadership:...

  • AI Transformation Specialist

    2 weeks ago

    bareilly, India beBeeArtificial Full time

    AI Role DescriptionThe role bridges business stakeholders and technical AI teams, ensuring innovation translates into measurable business value.As an AI professional, you will partner with global product, engineering, and data science teams to capture requirements, design workflows, and ensure AI-driven platforms meet customer needs.You will be embedded in...

  • AI Automation Expert

    2 weeks ago

    bareilly, India beBeeAutomation Full time

    Job TitleWe are seeking a detail-oriented and analytical AI Automation Intern to join our team. In this role, you will assist in creating and deploying no-code AI agents, troubleshooting and refining automated workflows, conducting process automation, generating test cases, and ensuring systems' reliability and efficiency through quality assurance.The ideal...

  • AI-Driven Storage Solution Engineer

    2 weeks ago

    bareilly, India beBeeStorage Full time

    Unlocking the Power of AI-Driven Storage SolutionsWe are seeking an experienced SDET to join our team and contribute to the development of transformative AI storage solutions. Our cutting-edge data intelligence platform accelerates AI workloads, enabling organizations to extract maximum value from their data.With a proven track record of performance,...