Chief AI Quality Engineer

4 days ago

bikaner, India beBeeAutomation Full time

Lead Quality Assurance Specialist – Automated TestingWe are seeking a highly skilled Lead Quality Assurance Specialist with expertise in Python automation, LLM evaluation, RAG pipelines, observability, adversarial testing, and Azure monitoring.This role will lead quality assurance initiatives for cutting-edge Generative AI and Multi-Agent Systems, focusing on test automation, LLM evaluation, and RAG framework development.Key Responsibilities:Design and execute LLM evaluation frameworks using G-Eval, custom evaluators, and Hallucination detection, faithfulness, relevance, precision/recall.Implement RAG evaluation frameworks (RAGAS or similar) and build Python-based automation frameworks using PyTest & DeepEval.Integrate automation into CI/CD pipelines using GitHub Actions and monitor applications using Azure Application Insights & Log Analytics.Define automated scoring systems for GenAI outputs and manage synthetic datasets and golden datasets for AI validation.Implement observability and trace monitoring using LangFuse, LangSmith, or similar tools.Required Skills and Qualifications:Mandatory Skills:Test Automation – Python, PyTest, DeepEvalLLM Evaluation – G-Eval, Custom Evaluators, LLM-as-a-JudgeRAG Evaluation – RAGAS, Retrieval MetricsEvaluation Metrics – Hallucination, Faithfulness, Relevance, Precision/RecallObservability & Monitoring – LangFuse, LangSmithCI/CD – GitHub ActionsMulti-Agent Testing – Reasoning & Tool ValidationAdversarial/Red Team Testing – Prompt Injection, Jailbreak, Bias/ToxicityAPI Testing – REST & Async WorkflowsAzure Monitoring – App Insights, Log AnalyticsSynthetic & Golden Dataset ManagementAutomated Scoring System Design for GenAI OutputsBenefits:This role offers the opportunity to work on cutting-edge technologies and contribute to the development of innovative solutions.Others:Please note that this job description is not an exhaustive list of responsibilities and may be subject to change based on business needs.

  • Chief AI Innovation Manager

    5 days ago

    bikaner, India beBeeInnovation Full time

    About the JobWe are seeking a highly skilled and innovative Chief AI Innovation Manager to spearhead our company's AI strategy.Key ResponsibilitiesDive deep into client requirements to develop tailored solutions.Craft compelling proposals that meet customer needs and expectations.Oversee the development and deployment of cutting-edge AI models for content...

  • Chief AI Innovation Officer

    2 days ago

    bikaner, India beBeeArtificialIntelligence Full time

    Job Title: Chief AI Innovation OfficerAbout the Role:We are seeking a visionary leader to spearhead our cutting-edge AI initiatives, driving innovation and excellence in artificial intelligence technology. As a key member of our team, you will play a pivotal role in shaping the future of AI.Key Responsibilities:Develop and lead advanced AI technologies,...

  • AI Quality Assurance Engineer

    6 days ago

    bikaner, India beBeeQuality Full time

    Conversational AI Developer Job DescriptionAs a key member of our engineering team, you will be responsible for driving the quality strategy for our cutting-edge Conversational AI platform.The successful candidate will have a strong background in software testing and development, with experience in building automation frameworks in a product or platform...

  • AI Quality Assurance Specialist

    3 days ago

    bikaner, India beBeeQuality Full time

    Artificial Intelligence Quality Assurance EngineerWe are seeking an experienced professional to join our team as a Senior AI Quality Assurance Test Engineer.Ensure the quality, reliability, and integrity of advanced in-house developed AI solutions.Work on production-grade systems, delivering robust and client-ready products.Key Responsibilities:Test...

  • Chief AI Architect

    3 days ago

    bikaner, India beBeeDataScience Full time

    Job Title: Lead EngineerWe are seeking a highly skilled Lead AI/ML Engineer to spearhead the design, development and implementation of advanced AI and machine learning models.Key Responsibilities:Develop project proposals in response to client requirements.Deliver solution/product demonstrations and workshops.Oversight of AI model deployment.Implement AI...

  • Chief Generative AI Architect

    1 day ago

    bikaner, India beBeeGenerative Full time

    Generative AI Engineer RoleIn this key position, you will design, develop and implement state-of-the-art generative AI models and systems. Your expertise will contribute to creating AI-powered solutions that generate high-quality clinically relevant texts while pushing the boundaries of creativity and innovation.This is an opportunity to collaborate closely...

  • AI Engineer

    4 weeks ago

    Bikaner, India Sutra.AI Full time

    Role: AI EngineerAbout Sutra.AISutra.AI is a rapidly growing AI Enterprise SaaS Platform company focused on building data-to-decision automation at scale.Our mission is to help enterprises transform raw data into intelligent, actionable insights through AI, automation, and decision intelligence.Role SummaryWe’re seeking an AI Engineer who is passionate...

  • AI Associate

    3 weeks ago

    Bikaner, India Synaptyx AI Full time

    🧑‍💻 𝗥𝗼𝗹𝗲: AI Associate (Full Stack Front-End Engineer) 📍 𝗟𝗼𝗰𝗮𝘁𝗶𝗼𝗻: Remote (India) 🕒 𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲: 0–1 years (React / Next.js / TypeScript / AI / ML Engineering)🧩 𝗔𝗜 𝗔𝘀𝘀𝗼𝗰𝗶𝗮𝘁𝗲 (𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸...

  • AI Associate

    2 weeks ago

    Bikaner, India Synaptyx AI Full time

    🧑‍💻 𝗥𝗼𝗹𝗲: AI Associate (Full Stack Front-End Engineer) 📍 𝗟𝗼𝗰𝗮𝘁𝗶𝗼𝗻: Remote (India) 🕒 𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲: 0–1 years (React / Next.js / TypeScript / AI / ML Engineering)🧩 𝗔𝗜 𝗔𝘀𝘀𝗼𝗰𝗶𝗮𝘁𝗲 (𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸...

  • Chief AI Innovation Officer

    5 days ago

    bikaner, India beBeeArtificialIntelligence Full time

    Transform business challenges into scalable artificial intelligence solutions from the ground up.Collaborate with cross-functional teams to develop cutting-edge AI applications using modern frameworks and libraries.Analyze complex business challenges and design innovative AI-driven solutionsDevelop and deploy production-ready AI solutions that drive...