Head of Data Quality - RL Gyms

About Turing

Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises looking to deploy advanced AI systems. Turing accelerates frontier research with high-quality data, specialized talent, and training pipelines that advance thinking, reasoning, coding, multimodality, and STEM. For enterprises, Turing builds proprietary intelligence systems that integrate AI into mission-critical workflows, unlock transformative outcomes, and drive lasting competitive advantage.

Recognized by Forbes, The Information, and Fast Company among the world’s top innovators, Turing’s leadership team includes AI technologists from Meta, Google, Microsoft, Apple, Amazon, McKinsey, Bain, Stanford, Caltech, and MIT. Learn more at www.turing.com

Role Overview

Turing is looking for a Head of Data Quality, RL Environments to build and lead the quality function for all reinforcement learning (RL) environment and trajectory data used to train and evaluate models at frontier AI labs.

You will manage a team of Data Quality Leads who operate like researchers in a frontier AI lab—designing tasks, stress tests, and evaluation protocols for complex RL environments (simulated, real-world, and tool-based). Your role is to set the bar for what “high-quality RL environment data” means and ensure our environments, trajectories, rewards, and evaluations are robust, diverse, and aligned with cutting-edge GenAI and RL research.

You’ll bring together:

• Deep understanding of RL environments, agents, and trajectories,
• Prior experience with ML/AI / RL / GenAI systems, and
• Strong organizational and people leadership

to create a research-grade quality organization for RL environments and agent interaction data.

Key Responsibilities

1. Own the RL Environment Data Quality Vision & Strategy

• Define the end-to-end strategy for data quality across all RL environment–related projects:
• Environment/task definitions
• Reward functions and signals
• Scenario generation and curriculum
• Agent behavior trajectories and evaluations
• Translate GenAI and agent/RL trends (e.g., tool-using agents, multi-step reasoning, multi-agent systems, simulated worlds, web environments) into actionable environment and data requirements.
• Set clear quality standards, rubrics, and KPIs for environments and trajectories, calibrated to frontier AI research expectations.

2. Lead & Develop Data Quality Leads

• Hire, manage, and mentor Data Quality Leads overseeing RL environment projects and annotation streams (e.g., trajectory labeling, human evaluations of agent behavior, reward calibration).
• Build a culture where leads think and act like researchers:
• Formulating hypotheses about what environments and tasks are needed
• Designing experiments and ablations
• Iterating based on empirical evidence
• Provide technical and methodological guidance on:
• Task and environment spec design
• Reward design and evaluation frameworks
• Quality review protocols for trajectories, states, and behaviors
• Establish performance expectations, feedback loops, and growth paths for quality leads and extended quality teams.

3. Design Research-Grade Evaluation & Quality Systems for RL Environments

• Oversee the design of evaluation frameworks for RL agents and environments, covering:
• Task success metrics and reward adequacy
• Robustness and generalization tests
• Safety, constraint satisfaction, and failure mode analysis
• Ensure quality leads apply rigorous experimental design:
• Proper sampling of scenarios and environment configurations
• A/B testing of reward functions, curriculum strategies, and environment variants
• Statistically sound comparisons between agent versions and evaluation suites
• Implement continual monitoring systems:
• Environment correctness and stability (e.g., no broken tasks, bugs, or unintended shortcuts)
• Drift detection in distribution of tasks, states, and trajectories
• Root cause analysis for degradation in performance or evaluation signal quality

4. Translate AI & RL Research Trends into E