Staff Machine Learning Modeler, Model Risk Management

Company

Block is one company built from many blocks, all united by the same purpose of economic empowerment. The blocks that form our foundational teams — People, Finance, Counsel, Hardware, Information Security, Platform Infrastructure Engineering, and more — provide support and guidance at the corporate level. They work across business groups and around the globe, spanning time zones and disciplines to develop inclusive People policies, forecast finances, give legal counsel, safeguard systems, nurture new initiatives, and more. Every challenge creates possibilities, and we need different perspectives to see them all. Bring yours to Block.

Team

The Model Risk Management team ensures our machine learning models are safe, reliable, and compliant with regulatory requirements. We focus on building automated validation tools while maintaining high standards for model assessment. Our work helps prevent errors and bias while making sure models are used appropriately across Block. We work closely with teams throughout the company to establish practical governance frameworks, assess model performance, and share best practices. As a core part of the compliance organization, we support all key areas of Block regardless of product or market.

The Role

We're looking for a Machine Learning Modeler to help us make our model validation process both more efficient and more thorough. You'll be validating a diverse portfolio of machine learning models that support Block's financial services, with current focus on:

• Production-scale gradient boosted decision trees for financial decisioning systems
• Hybrid ML systems that leverage LLMs in the training data pipeline
• Emerging applications of LLMs in financial services
• Additional ML models supporting Block's growing suite of financial products

You'll split your time between hands-on model validation work and developing tools to automate these processes. Your experience validating ML models will be key in building practical solutions that help our team work more effectively.

A key aspect of the role will be developing validation frameworks that can handle both traditional ML models (like XGBoost) and newer AI technologies (like LLMs), ensuring robust governance and reliable performance across different model architectures and use cases.

We're looking for someone with strong technical skills in both machine learning and software engineering. You should be detail-oriented and familiar with relevant regulations and industry standards. Clear communication and the ability to work both independently and as part of a team are essential.

You Will

Primary Focus:

• Build scalable validation frameworks for tree-based models, focusing on feature importance analysis, stability, and performance metrics
• Develop validation approaches for hybrid systems where LLMs support the ML pipeline
• Create governance frameworks for LLM applications, including:
  • Reliability and consistency assessment methodologies
  • Prompt engineering validation approaches
  • Output quality control mechanisms
  • Drift detection for both traditional ML and LLM components
• Design testing frameworks that can adapt to different model types and use cases
• Create tools that help generate clear validation reports
• Set up systems to continuously monitor model performance
• Build tools that make model validation faster and more consistent
• Create validation components we can reuse across different projects
• Develop automated approaches for common tasks like:
  • Checking model performance
  • Running statistical tests
  • Verifying data quality
  • Testing model assumptions
  • Tracking performance changes over time

Immediate Responsibilities:

• Assess machine learning models using rigorous validation methodologies
• Develop automated validation tools that can scale across different model types
• Set up comprehensive model monitoring systems
• Maintain clear validation documentation aligned with regulatory requirements
• Partner with model development teams to understand validation needs
• Build effective relationships while maintaining independent assessment standards

You Have

Required:

• Advanced degree in Computer Science, Machine Learning, or related quantitative field
• 5+ years experience in model validation or risk management, with focus on machine learning models; or 3+ years and a graduate degree
• Strong software engineering practices and experience building maintainable, well-documented code
• Strong understanding of tree-based models, particularly gradient boosted decision trees and XGBoos