Director, Responsible AI Measurement

Microsoft is committed to ensuring that we develop and deploy our Artificial Intelligence (AI) technologies in ways that uphold our AI principles and warrant people’s trust.  The Office of Responsible AI (ORA) plays a leading role in this effort as part of Microsoft’s Corporate, External, and Legal Affairs (CELA) team.  With a cross-company mandate, we establish internal policy for how our engineering and sales teams implement responsible AI practices, enable our community of Responsible AI Champs to support the rollout of our responsible AI program, and review engineering and customer projects that require additional input based on potential impact.  We also contribute to new laws, norms, and standards to advance responsible AI for the benefit of society at large. The Director, Responsible AI Measurement role is part of ORA’s Internal Policy & Practice team and would support the implementation efforts for our Responsible AI Standard.  

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Responsibilities

• Collaborate with AI research, policy, and engineering teams to conceptualize and define potential generative AI risks. 
• Consult on operationalizing policy concepts into measurable constructs, develop methods and evaluate approaches to measurement and risk mitigation, and contribute to strategies that address potential generative AI risks, working with AI platform, research, and data science teams.  
• Draft policies and implementation guidance for AI model training and AI product development and deployment. 
• Integrate industry-wide responsible AI policy and ethics insights and best practices to anticipate risks and identify technical blockers to the implementation of Microsoft’s Responsible AI Standard, which governs responsible AI policies and practices at Microsoft. 
• Leverage technical expertise to develop partnerships across disciplines to lead the development of new or modified tools, technologies, and methods to mitigate risks and overcome blockers.  
• Drives development and implementation of automation and centralized metrics to support teams across the company comply with the Responsible AI Standard.  

Embody our Culture and Values

Qualifications

Required / Minimum Qualifications 

• Bachelor’s Degree in Business, Operations, Finance or related field AND 8+ years work experience in program management, process management, process improvement 
  • OR equivalent experience 
• 2+ years of experience working with artificial intelligence (AI), machine learning (ML), and/or natural language processing (NLP)

Preferred Qualifications 

• Experience in applied research, computational social science, statistics, large-scale computing, modeling, or data science, and comfort with relevant tools, e.g., Python, R, Synapse, PyTorch, scikit-learn, in the fields of artificial intelligence (AI), machine learning (ML), and/or natural language processing (NLP).  

• OR Bachelor’s Degree in Statistics, Data Science, Mathematics, Computer Science, Computational Social Science, or related field.  

• Experience collaborating with researchers in the fields of Responsible AI (RAI) or computational social science. 
• Experience approaching RAI as a sociotechnical challenge rather than a purely technical one.  
• Experience working in or with a large, matrixed organization. 
• Highly developed written and oral communication skills. 
• Experience learning independently in new domains, including by reading academic papers. 
• Proven track record of being able to engage and influence a variety of stakeholders. A key attribute will be the ability to work across multiple disciplines and cultures with stakeholders who have a variety of technical and non-technical backgrounds. 
• Record of being an effective team player dedicated to collective successe