Gen AI QA Engineering Lead

Join the Corporate AI Incubator team within Corporate Banking as a Gen AI QA Engineering Lead at Barclays. This role is pivotal in ensuring the quality, reliability, and compliance of cutting-edge generative AI solutions, including LLMs and retrieval-augmented generation (RAG) pipelines. You will lead QA efforts across multiple AI PODs, collaborating with data scientists, ML engineers, Data engineers and governance teams to build robust testing frameworks and automation pipelines that validate model performance, safety, and enterprise readiness.

To be successful as a Gen AI QA Engineering Lead, you should have experience with:

• Designing and implementing QA frameworks for GenAI models, including prompt validation, output consistency, hallucination detection, and bias mitigation.
• Building automated test suites for GenAI pipelines using Python, PyTest, and tools like MLflow, SageMaker, or similar.
• Integrating QA checks into CI/CD pipelines (e.g., Jenkins, GitHub Actions, GitLab CI/CD) for continuous validation.
• Monitoring model outputs for anomalies and ensuring high-quality data ingestion and refresh cycles in collaboration with Data Enablement and DevOps teams.
• Applying DevOps principles to QA workflows, especially in cloud-native environments using AWS (e.g., S3, Glue, Athena, Step Functions, Lambda, CloudFormation).

Some other highly valued skills may include:

• Experience with LangChain, Vector DBs (e.g., OpenSearch, PG vector, ChromaDB), and RAG pipelines.
• Understanding of model interpretability, responsible AI, and governance frameworks.
• Strong leadership, communication, and cross-functional collaboration skills.
• Knowledge of data engineering, data privacy, model governance, and regulatory compliance in AI systems.

You may be assessed on the key critical skills relevant for success in role, such as risk and controls, change and transformation, business acumen strategic thinking and digital and technology, as well as job-specific technical skills.

This role is for Pune location.

Purpose of the role

To manage the efficient delivery of large-scale technical projects and capabilities across the bank and collaborate with internal and external stakeholders to understand their needs and expectations throughout the software product lifecycle, adhering to agreed time, budget and quality requirements.

Accountabilities

• Management of the delivery, resource allocation and improvement of complex technical project capabilities across the bank using Agile/Scrum methodologies for planning, stand-ups, demos, retrospectives and the execution of sprints.
• Collaboration with customers and business teams to manage and implement customer trails to support the proposal of technically feasible solutions, and effort and timeline expectations.
• Collaboration with software engineers, quality teams, product managers and other engineering teams to deliver high-quality products and features through the software project lifecycle aligned to the banks objectives, regulation and security policies.
• Identification, assessment, and mitigation of risks associated with technical projects to prepare the bank for potential challenges.
• Management of change requests and communication with stakeholders throughout the project lifecycle.
• Management of vendor relations involved in technical projects to ensure they deliver according to agreed terms.
• Stay abreast of the latest industry technology trends and technologies, to evaluate and adopt new approaches to improving deliver outcomes and to foster a culture of continuous learning, technical excellence and growth.

Vice President Expectations

• To contribute or set strategy, drive requirements and make recommendations for change. Plan resources, budgets, and policies; manage and maintain policies/ processes; deliver continuous improvements and escalate breaches of policies/procedures..
• If managing a team, they define jobs and responsibilities, planning for the department's future needs and operations, counselling employees on performance and contributing to employee pay decisions/changes. They may also lead a number of specialists to influence the operations of a department, in alignment with strategic as well as tactical priorities, while balancing short and long term goals and ensuring that budgets and schedules meet corporate requirements..
• If the position has leadership responsibilities, People Leaders are expected to demonstrate a clear set of leadership behaviours to create an environment for colleagues to thrive and deliver to a consistently excellent standard. The four LEAD behaviours are: L – Listen and be authentic, E – Energise and inspire, A – Align across the enterprise, D – Develop others..
• OR for an individual contributor, they will be a subject matter expert within own discipline and will guide technical direction. They will lead collaborative, multi-year assignments and guide team members through structured assignments, identify the need for the inclusion of other areas of specialisation to complete assignments. They will train, guide and coach less experienced specialists and provide information affecting long term profits, organisational risks and strategic decisions..
• Advise key stakeholders, including functional leadership teams and senior management on functional and cross functional areas of impact and alignment.
• Manage and mitigate risks through assessment, in support of the control and governance agenda.
• Demonstrate leadership and accountability for managing risk and strengthening controls in relation to the work your team does.
• Demonstrate comprehensive understanding of the organisation functions to contribute to achieving the goals of the business.
• Collaborate with other areas of work, for business aligned support areas to keep up to speed with business activity and the business strategies.
• Create solutions based on sophisticated analytical thought comparing and selecting complex alternatives. In-depth analysis with interpretative thinking will be required to define problems and develop innovative solutions.
• Adopt and include the outcomes of extensive research in problem solving processes.
• Seek out, build and maintain trusting relationships and partnerships with internal and external stakeholders in order to accomplish key business objectives, using influencing and negotiating skills to achieve outcomes.

All colleagues will be expected to demonstrate the Barclays Values of Respect, Integrity, Service, Excellence and Stewardship – our moral compass, helping us do what we believe is right. They will also be expected to demonstrate the Barclays Mindset – to Empower, Challenge and Drive – the operating manual for how we behave.