Wordsmith Position

As a skilled wordsmith, you will play a crucial role in evaluating and enhancing large language models. With native proficiency in the English language and deep familiarity with cultural norms, you will create high-quality content, such as writing prompts, rubric definitions, and prompt evaluations.

• Develop engaging content that showcases your linguistic expertise.
• Update rubric definitions to ensure cultural and linguistic relevance by incorporating region/language-specific examples.
• Identify areas where additional rubrics are needed for specific languages or regional contexts.
• Review translated prompts, revise them where necessary, and write thoughtful prompts that test cultural awareness in LLM models.
• Ratings and justifications of prompt-response pairs using standardized evaluation templates based on rubrics.
• Document problematic outputs, annotating them with clear explanations of rubric violations or cultural insensitivities.

• Native proficiency in the target language and deep familiarity with cultural norms in the corresponding region.
• Experience in LLM evaluation, content moderation, or linguistic QA preferred.
• Strong attention to detail with the ability to identify subtle issues in language use, tone, and cultural references.
• Comfortable working in spreadsheets and evaluation templates.
• Bachelor's or Master's degree or PhD resources in relevant streams required.

We encourage candidates with prior experience in content/creative writing, prompt engineering, or LLM testing to apply. Familiarity with tools like Gemini, ChatGPT, or similar LLM platforms is also beneficial. Ability to clearly articulate reasoning behind rubric ratings or prompt edits is essential.