(USA) Director, Data Science, Testing and Measurement

Position Summary...

What you'll do...

Lead the centralization of testing and measurement at Sam’s Club by building a single experimentation and measurement platform and launching self-service tools that empower associates to run high-quality experiments independently. Starting with in-club measurement and moving to Omni you’ll establish scientific standards, governance, and guardrails that make results trustworthy and repeatable. You will translate complex data into clear, actionable decisions, mentor a high-performing team, and partner across Product, Engineering, and the business to drive measurable business impact—while reserving your team’s deep expertise for the most complex use cases. 

What You’ll Do (Performance Objectives) 

• Lead testing & measurement framework  
  Inventory current experimentation practices across teams; map data sources, metrics, and decision paths; and publish a Unified Testing & Measurement Playbook (taxonomy, KPI catalog, experiment types, power/MDE guidance, guardrails, and approval paths). Secure cross-functional alignment. 
   

• Design the experimentation platform  
  Define architecture and build-versus-buy strategy for an enterprise testing platform (assignment service, telemetry, experiment registry, metrics pipeline, results service, observability, and audit trails). Kick off MVP build with cross functional partners; integrate with access control, and data platforms. 
   

• Launch self-serve experimentation (4–6 months): 
  Ship a guided, self-service tool enabling ~80% of standard experiments without DS support (A/A checks, sizing calculators, pre-registration, guardrails, and standardized readouts). Train product/ops partners and stand up community of practice. 
  Success looks like: ≥70% of experiments initiated via self-serve; median time to test start reduced by ≥40%. 

• Deliver in-club measurement excellence (pilot → scale): 
  Stand up robust in-club measurement for prioritized initiatives (e.g., signage, layouts, checkout prompts, labor/process changes) using appropriate designs (RCTs, matched controls, diff-in-diff, synthetic control). Publish weekly readouts and executive rollups. 
   

• Institutionalize experimentation rigor (ongoing): 
  Implement power analysis, minimum detectable effect standards, sequential testing controls, causal inference, and reproducibility checks. Require pre-registered hypotheses and success criteria. 
   

• Scale impact & portfolio ROI  
  Produce a quarterly experimentation portfolio review quantifying incremental sales, margin, and member engagement; prioritize the roadmap to maximize ROI. 
  Success looks like: X% QoQ increase in experiments tied to top OKRs; clear $ impact from in-club initiatives. 

• Build and lead a high-performing team: