About Handshake AI

Handshake is building the career network for the AI economy. Our three-sided marketplace connects 18 million students and alumni, 1,500+ academic institutions across the U.S. and Europe, and 1 million employers to power how the next generation explores careers, builds skills, and gets hired.

Handshake AI is a human data labeling business that leverages the scale of the largest early career network. We work directly with the world’s leading AI research labs to build a new generation of human data products. From PhDs in physics to undergrads fluent in LLMs, Handshake AI is the trusted partner for domain-specific data and evaluation at scale.

This is a unique opportunity to join a fast-growing team shaping the future of AI through better data, better tools, and better systems—for experts, by experts.

Now’s a great time to join Handshake. Here’s why:

• Leading the AI Career Revolution: Be part of the team redefining work in the AI economy for millions worldwide.

• Proven Market Demand: Deep employer partnerships across Fortune 500s and the world’s leading AI research labs.

• World-Class Team: Leadership from Scale AI, Meta, xAI, Notion, Coinbase, and Palantir, just to name a few.

• Capitalized & Scaling: $3.5B valuation from top investors including Kleiner Perkins, True Ventures, Notable Capital, and more.

About the Role

As a Staff Research Scientist, you will play a pivotal role in shaping the future of large language model (LLM) alignment by leading research and development at the intersection of data quality and post-training techniques such as RLHF, preference optimization, and reward modeling.

You will operate at the forefront of model alignment, with a focus on ensuring the integrity, reliability, and strategic use of supervision data that drives post-training performance. You’ll set research direction, influence cross-functional data standards, and lead the development of scalable systems that diagnose and improve the data foundations of frontier AI.

You will:

• Lead high-impact research on data quality frameworks for post-training LLMs — including techniques for preference consistency, label reliability, annotator calibration, and dataset auditing.

• Design and implement systems for identifying noisy, low-value, or adversarial data points in human feedback and synthetic comparison datasets.

• Drive strategy for aligning data collection, curation, and filtering with post-training objectives such as helpfulness, harmlessness, and faithfulness.

• Collaborate cross-functionally with engineers, alignment researchers, and product leaders to translate research into production-ready pipelines for RLHF and DPO.

• Mentor and influence junior researchers and engineers working on data-centric evaluation, reward modeling, and benchmark creation.

• Author foundational tools and metrics that connect supervision data characteristics to downstream LLM behavior and evaluation performance.

• Publish and present research that advances the field of data quality in LLM post-training, contributing to academic and industry best practices.

Desired Capabilities

• PhD or equivalent experience in machine learning, NLP, or data-centric AI, with a track record of leadership in LLM post-training or data quality research.

• 5 years of academic or industry experience post-doc

• Deep expertise in RLHF, preference data pipelines, reward modeling, or evaluation systems.

• Demonstrated experience designing and scaling data quality infrastructure — from labeling frameworks and validation metrics to automated filtering and dataset optimization.

• Strong engineering proficiency in Python, PyTorch, and ecosystem tools for large-scale training and evaluation.

• A proven ability to define, lead, and execute complex research initiatives with clear business and technical impact.

• Strong communication and collaboration skills, with experience driving strategy across research, engineering, and product teams.

Extra Credit

• Experience with data valuation (e.g. influence functions, Shapley values), active learning, or human-in-the-loop systems.

• Contributions to open-source tools for dataset analysis, benchmarking, or reward model training.

• Familiarity with evaluation challenges such as annotation disagreement, subjective labeling, or multilingual feedback alignment.

• Interest in the long-term implications of data quality for AI safety, governa