Validations Specialist, Trust & Safety - USDS

Team Intro

The Risk & Response Team for TikTok USDS is a multi-disciplinary team that handles risk detection, prevention and handling in the United States. The team is dedicated to maintaining a safe and secure online environment for users by actively monitoring and responding to emergent threats, ensuring compliance with legal regulations, and addressing real-life implications of online content.



As a Validations Specialist, you will support USDS Risk & Response teams with content and account review, analysis, and enforcement; test and improve threat detection logic; and support testing and implementation of new tools and processes



Our Validations Team performs a critical function that supports our efforts to address objectionable or disturbing content. Content that the Validations Specialist interacts with includes images, video, and text related to every-day life, but it can also include (but is not limited to) bullying; hate speech; child abuse; sexual assault; torture; bestiality; self-harm; suicide; and murder.

It is possible that this role will be exposed to harmful content on a daily basis.



This team must be comfortable occasionally working unusual business hours (e.g., Tuesday-Saturday or Sunday-Thursday), working late night shifts, and also providing shift coverage during crisis events.



In order to enhance collaboration and cross-functional partnerships, among other things, at this time, our organization follows a hybrid work schedule that requires employees to work in the office 3 days a week, or as directed by their manager/department. We regularly review our hybrid work model, and the specific requirements may change at any time.



Responsibilities:

Expertise in TikTok Policy

- Interpret TikTok's Community Guidelines and make enforcement decisions based on them

- Stay current on key workflow changes and policy updates that are regularly cascaded

- Leverage unique vantage point to help bridge gaps between Policy, Moderation, and Trust & Safety teams in order to improve policy and process



Content & Account Reviews

- Conduct meticulous review of large volumes of potentially high harm content and accounts, and make enforcement decisions based on Community Guidelines

- Based on large scale reviews, identify and capture new intelligence signals to improve overall detection capabilities; drive real-time insights to leadership and cross-functional partners across Incident Management, Policy, Investigations, Comms, Legal, and other teams

- Partner with subject matter experts and analysts to develop methodologies for reviewing large volumes of potentially violative content; action content at scale based on high confidence reviews and analysis

- Support research content analysis through accurate labeling and annotating content--including written text, spoken language, images, audio, videos, and accounts--in order to create accurate classifiers for increased precision of models and improved recall rates for problematic content



Data-driven decision making

- Utilize data plus internal tools and systems to help Trust & Safety teams analyze trends, identify patterns, and share insights that support process improvements

- Share data and insights that illuminate policy gaps and grey areas, all with the intention of improving policy to enable clear, scalable enforcement

- Test new tools and analysts' hypotheses, and provide precise feedback and recommendations based on data



Detection Engineering Support

- Partner with Trust & Safety analysts to test and improve threat detection models

- Identify intelligence (signals) that improve analyst-built detections

- Support hypothesis testing through validating detections in the development phase



Investigate Intelligence Collection Requests

- Review, investigate, and process sensitive intelligence leads with high quality, accuracy, and speed

- Conduct deep-dive investigations leveraging subject matter expertise, policy, tooling, and data to thoroughly assess and validate potential and emerging threats on platform

- Leverage internal tools and queries to conduct "sweeps" designed to identify remove violative content at scale

- Provide summaries/reports documenting results of deep dive investigations



Project Management

- Ensure that action items identified as the result of root cause analyses are escalated to the appropriate cross-functional partners for remediation

- Trouble-shoot internal tooling and programs; support testing and iteration of new tool launches

- Manage and coordinate requests from cross-functional partners; ensure requests are completed in a timely manner

- Support Quality Assurance initiatives to assess the team's enforcement decisions