Head of Quality, AI Data Operations

About TaskUs: TaskUs is a provider of outsourced digital services and next-generation customer experience to fast-growing technology companies, helping its clients represent, protect and grow their brands. Leveraging a cloud-based infrastructure, TaskUs serves clients in the fastest-growing sectors, including social media, e-commerce, gaming, streaming media, food delivery, ride-sharing, HiTech, FinTech, and HealthTech.

The People First culture at TaskUs has enabled the company to expand its workforce to approximately 45,000 employees globally. Presently, we have a presence in twenty-three locations across twelve countries, which include the Philippines, India, and the United States.

It started with one ridiculously good idea to create a different breed of Business Processing Outsourcing (BPO)! We at TaskUs understand that achieving growth for our partners requires a culture of constant motion, exploring new technologies, being ready to handle any challenge at a moment’s notice, and mastering consistency in an ever-changing world.

What We Offer: At TaskUs, we prioritize our employees' well-being by offering competitive industry salaries and comprehensive benefits packages. Our commitment to a People First culture is reflected in the various departments we have established, including Total Rewards, Wellness, HR, and Diversity. We take pride in our inclusive environment and positive impact on the community. Moreover, we actively encourage internal mobility and professional growth at all stages of an employee's career within TaskUs. Join our team today and experience firsthand our dedication to supporting People First.

Position Summary:
As the Head of Quality, AI Data Operations, you will be responsible for ensuring that the quality of our training data, adversarial testing, and model evaluation services meets the highest standards. You will drive thought leadership on data quality, work closely with customers to define and deliver high-quality outputs, and oversee the development of training and quality protocols across projects. This role requires a blend of strategic thinking, hands-on operations, and cross-functional collaboration to ensure our services are impactful for AI and LLM projects.

The ideal candidate will have a strong entrepreneurial mindset, deep expertise in both statistics and quality assurance, and experience leading global teams. They will be comfortable working in a fast-paced, ambiguous environment and being in a constant state of learning.

Key Responsibilities:

• Lead the design and implementation of quality assurance protocols tailored to generative AI and LLM projects.

• Own the definition of what "high-quality" data means across diverse projects and ensure these standards are met across the organization.

• Drive thought leadership with customers, aligning on data quality goals, and developing innovative quality processes.

• Act as the primary point of contact for customer discussions related to data quality and model evaluation.

• Provide strategic guidance on handling edge cases, and offer solutions for the nuances and challenges involved in building high-quality training data.

• Lead external communication on data quality, positioning the company as a thought leader in AI model robustness and evaluation.

• Collaborate with researchers, linguists, and subject matter experts to stay ahead of advancements in data quality definitions and standards.

• Foster a high-performance culture, encouraging continuous learning and improvement 

• Continuously explore and implement new methodologies for improving the quality of training data and model evaluation processes.

• Drive a culture of testing, iterating, and scaling processes, ensuring that all workflows are efficient and results-driven.

Required Skills & Competencies:

• 7+ years of experience, ideally with an entrepreneurial background and a proven ability to lead teams in ambiguous, fast-moving environments.

• Deep understanding of large language models, generative AI workflows, and data structures.

• Strong interest and commitment to learning about the latest developments in LLMs and AI technologies.

• Proven ability to design and lead quality assurance processes for data annotation and AI model evaluation.

• Experience developing scalable systems like "golden datasets" for e