AI Quality Lead - Virtual

Our story 

At Alight, we believe a company’s success starts with its people. At our core, we Champion People, help our colleagues Grow with Purpose and true to our name we encourage colleagues to “Be Alight.” 

Our Values: 

Champion People – be empathetic and help create a place where everyone belongs. 

Grow with purpose – Be inspired by our higher calling of improving lives. 

Be Alight – act with integrity, be real and empower others. 

It’s why we’re so driven to connect passion with purpose. Alight helps clients gain a benefits advantage while building a healthy and financially secure workforce by unifying the benefits ecosystem across health, wealth, wellbeing, absence management and navigation. 

With a comprehensive total rewards package, continuing education and training, and tremendous potential with a growing global organization, Alight is the perfect place to put your passion to work. 

Join our team if you Champion People, want to Grow with Purpose through acting with integrity and if you embody the meaning of Be Alight. 

Learn more at careers.alight.com.  

As the AI Quality Lead for the Artificial Intelligence Center of Excellence (AI CoE), you will architect and drive the end-to-end AI Quality program from inception. This is a hands-on leadership role, requiring both strategic vision and technical depth to go from 0→1 in AI Quality at Alight. 

RESPONSIBILITIES 

• Design and implement the unified AI Quality strategy and SOP, scaling controls by risk tier (Tier 1–3) 

• Define and own success metrics, risk tiering, acceptance criteria, and operating thresholds for AI solutions 

• Lead the development of golden/adversarial test suites (prompt injection, misuse, safety, bias, functional automation) 

• Oversee the creation and maintenance of schema-level UI/API tests, agent contracts, and observability instrumentation 

• Own the org-wide evaluation harnesses (e.g., promptfoo, TruLens, Ragas), CI gating, and red-team playbooks 

• Manage independent spot checks, factsheets, registry entries, and evidence for governance and audit 

• Collaborate with Product Owners, Developers, SRE/Observability, and Governance to ensure compliance and operational excellence 

• Drive continuous improvement in fairness, explainability, safety, security, and operational SLOs 

• Mentor and grow the AI Quality team, including onboarding and coaching junior engineers 

• Experience with LLM, RAG, agentic, or classic ML systems 

• Familiarity with test automation frameworks, CI/CD pipelines, and evaluation tools (promptfoo, TruLens, Ragas) 

• Knowledge of fairness, explainability, safety, and security standards (e.g., OWASP Top 10, MITRE ATLAS, Fairlearn, AIF360) 

• <