Quality Lead for LLM Response Evaluations IRC277492

Description
The requirment is for content engineering team focusing on working on multiple AI / ML / LLM use cases. The team currently work on millions of transactions in a month and contributing to some of the most fascinating products and features.

Requirements
We are seeking a highly motivated and experienced Quality Lead to oversee our team of LLM (Large Language Model) response evaluators. The ideal candidate will be responsible for ensuring the accuracy, consistency, and quality of human evaluations of LLM outputs. This role requires a strong understanding of LLM response evaluation, excellent leadership skills, and a keen eye for detail.

Qualifications
Bachelor's degree in Linguistics, Data Science, or a related field.

3-4 years of experience in quality assurance

1-2 years proven experience leading and managing a QA team for LLM response evaluation operations

Strong understanding of LLM concepts and LLM response evaluations rubrics

Excellent analytical and problem-solving skills.

Exceptional written and verbal communication skills.

Ability to work independently and collaboratively in a very dynamic environment.

Preferred Qualifications
Experience in developing and delivering training materials

Job responsibilities

Supervise, train, and mentor a team of QAs for LLM response evaluation work, providing guidance, training, and feedback..

Develop and implement quality assurance framework, processes and guidelines for LLM response evaluations.

Create and maintain detailed and objective rubrics and guidelines for assessing response quality, including metrics for accuracy, coherence, tone, safety, and alignment with user intent.

Conduct regular audits of evaluated data to ensure adherence to quality standards and identify areas for improvement.

Collaborate with Operations and client stakeholders to refine evaluation guidelines and improve evaluation performance.

Analyze evaluation data to identify trends, patterns, and insights.

Provide detailed reports on quality metrics and team performance.

Proactively identify and recommend improvements to evaluation tools, processes, and workflows to increase efficiency and accuracy.

What we offer

Culture of caring.
At GlobalLogic, we prioritize a culture of caring. Across every region and department, at every level, we consistently put people first. From day one, you'll experience an inclusive culture of acceptance and belonging, where you'll have the chance to build meaningful connections with collaborative teammates, supportive managers, and compassionate leaders.

Learning and development.
We are committed to your continuous learning and development. You'll learn and grow daily in an environment with many opportunities to try new things, sharpen your skills, and advance your career at GlobalLogic. With our Career Navigator tool as just one example, GlobalLogic offers a rich array of programs, training curricula, and hands-on opportunities to grow personally and professionally.

Interesting & meaningful work.
GlobalLogic is known for engineering impact for and with clients around the world. As part of our team, you'll have the chance to work on projects that matter. Each is a unique opportunity to engage your curiosity and creative problem-solving skills as you help clients reimagine what's possible and bring new solutions to market. In the process, you'll have the privilege of working on some of the most cutting-edge and impactful solutions shaping the world today.

Balance and flexibility.
We believe in the importance of balance and flexibility. With many functional career areas, roles, and work arrangements, you can explore ways of achieving the perfect balance between your work and life. Your life extends beyond the office, and we always do our best to help you integrate and balance the best of work and life, having fun along the way

High-trust organization.
We are a high-trust organization where integrity is key. By joining GlobalLogic, you're placing your trust in a safe, reliable, and ethical global company. Integrity and trust are a cornerstone of our value proposition to our employees and clients. You will find truthfulness, candor, and integrity in everything we do.

About GlobalLogic
GlobalLogic, a Hitachi Group Company, is a trusted digital engineering partner to the world's largest and most forward-thinking companies. Since 2000, we've been at the forefront of the digital revolution – helping create some of the most innovative and widely used digital products and experiences. Today we continue to collaborate with clients in transforming businesses and redefining industries through intelligent products, platforms, and services.