Data Labeling Project Manager

About The Role:

Uber AI Solutions is one of Uber's biggest bets with the ambition to build one of the world's largest data foundries for AI applications and evolve into a platform of choice for various online tasks.

Data Labelling operations is one of the core functional teams within Uber AI Solutions, responsible for overseeing the end-to-end lifecycle of the data annotation programs for B2B clients.

Our ability to build industry-leading data annotation programs is our right to win in this space.

Job Description:

As a seasoned program manager for data labeling in the Gen AI space, you will be responsible for defining the programs and their key objectives to support LLM model training.

You will drive cross-functional efforts across Operations, Product, Engineering, and Legal to define the Program-level Ops strategy, design scalable data labeling workflows leveraging internal tools, external vendors, and automation.

The role will also shape and lead Scaled Solutions task/product/feature launches and improvements by working closely with global Supply, product, and engineering teams.

You will be required to work with a geographically diverse team.

Key Responsibilities:

• Define the key objectives for data labeling projects to support generative AI initiatives.
• Partner with stakeholders (Account Executives, Data Scientists, Machine Learning Engineers, and Product Managers) to identify data requirements and success criteria.
• Represent Scaled Solutions in customer meetings along with Account Executives.
• Design scalable data labeling workflows that leverage internal tools, external vendors, and automation.
• Optimize workflows for efficiency, accuracy, and cost-effectiveness, incorporating active learning and pre-labeling techniques where appropriate.
• Engage and manage relationships with data labeling vendors, ensuring timely delivery and adherence to quality standards.
• Collaborate with cross-functional teams to align labeling efforts with broader AI model development timelines.
• Implement robust quality assurance processes to validate labeled datasets against gold standards.
• Use metrics such as inter-annotator agreement, precision/recall, and throughput to monitor quality and make improvements.
• Manage program budgets, including vendor costs and internal resources.
• Forecast resource requirements and ensure efficient allocation to meet deadlines.
• Advocate for inclusive and unbiased labeling practices to mitigate bias in AI models.

Requirements:

• 5+ years of experience managing scaled operations programs with 1+ year of experience in GenAI/model training.
• Experience working in a fast-paced, ambiguous work environment.
• Strong knowledge of machine learning concepts, particularly around supervised learning and training data needs.
• Experience working with data annotation platforms and tools.
• Proven track record of managing large-scale projects with cross-functional teams and external vendors.

Preferred Qualifications:

• Experience in Generative AI, including text, image, video, or audio data labeling.
• Familiarity with active learning, semi-supervised labeling, and human-in-the-loop systems.
• Proficiency in data annotation tools and scripting languages (Python, SQL) to analyze datasets and processes.
• Strong understanding of ethical AI and best practices for minimizing dataset bias.
• Excellent written and verbal communication skills, with the ability to influence technical and non-technical stakeholders.
• Strong understanding of the gig economy landscape, freelancer behaviors, and recruitment strategies.
• Excellent project management skills, with a proven ability to juggle multiple priorities and deadlines.
• Ability to take initiative in a constantly-changing work environment.

What We Offer:

At Uber AI Solutions, we are passionate about re-imagining the gig marketplace for skilled workers and impacting the lives of millions of skilled gig workers. We offer a top-tier team environment that fosters growth, innovation, and collaboration.