Summer Associate Internship (Data Scientist - Model Risk Management)

Model risk is the potential for adverse consequences (e.g., financial loss, poor business or strategic decisions, reputational damage) arising from decisions based on incorrect or misused model outputs and reports. Model risk comes down to making informed business decisions by understanding a model's weaknesses or limitations. With a variety of models designed to maintain Navy Federal’s financial stability and its ability to make impactful member-related decisions, it is more important than ever to ensure those models are tested, challenged on current model usage and practices, and assessed on the effectiveness for their intended uses.
 

The Model Risk Management (MRM) team serves as a 2nd line of defense, and our mission is to:

• Be a trusted partner in modeling and risk management
• Challenge models and deliver transparency for a common interest across the organization
• Deliver model risk as a competitive advantage that benefits our members
• Manage model risk, not eliminate it
• Collectively become a modeling “Center of Excellence” for the enterprise

 

MRM is responsible for maintaining a governance framework that standardizes model risk activities from model development through retirement. MRM uses control processes to ensure sound model risk management across all model lifecycle activities.

Core components of Model Risk Management’s role include:

• Identification 
  • Maintaining a full inventory of models including documentation, code, and datasets
• Assessment
  • Assessing the conceptual soundness of models and their risk through analysis
  • Recommending model performance enhancements and risk mitigations
• Monitoring
  • Monitoring model usage, changes, performance, data drift, and production errors
• Reporting
  • Providing an enterprise-level assessment of model risk
  • Tracking the status of individual model risk findings and approving mitigation/remediation plans

 

Project – Model Validation:

The Summer Associate will complete an end-to-end validation of one of Navy Federal’s predictive models. This validation would include analyzing the model’s data, conceptual soundness, key assumptions, limitations, estimation, calibration, and implementation as appropriate. The validation may also include developing a challenger model against the original model design. At the conclusion of the validation, the Summer Associate will have identified a list of model risk findings that will be presented to the Model Owner for remediation. The selected model for this project may be within a variety of possible subject domains, such as fraud detection, targeted marketing, underwriting, financial forecasting, and more.

 

Project Steps:

1. Review model documentation and meet with business owners as needed to gain an understanding of the model
2. Conduct a complete independent model validation, which may include, but is not limited to, assessments of the following: 
   1. Model theoretical framework, fitting methods, assumptions
   2. Data creation process, including data preparation and data quality
   3. Model specification
   4. Model development and performance testing, which may include in-sample, out-of-sample, out-of-time backtesting, sensitivity testing, stress testing, and performance monitoring results
   5. Ongoing monitoring plan, including proposed or existing ongoing monitoring metrics and thresholds
   6. Appropriateness of the implementation testing plan scope in context of associated model risk, testing metrics, and user acceptance testing (UAT)
   7. Code and calculations associated with development/estimation and/or implementation
   8. Model documentation including completeness, accuracy, level of details
3. Document and quantify the materiality of each finding
4. Develop recommendations for model developers to mitigate the risks identified

• Leverage technologies – including Python and R – to analyze and gain insights within large data sets
• Evaluate model design and performance and perform champion/challenger development. Analyze model input data, assumptions, and overall methodology. 
• Using statistical practices, analyze current and historical data to make predictions, and identify risks, and opportunities, enabling better decisions on planned/future events 
• Provide analytics insights and solutions to solve complex business problems
• Examine data from multiple sources and share insights with leadership and stakeholders
• Transform data presented in models, charts, and tables into a format that is useful to the business and aids in effective decision-making
• Develop and maintain an understanding of relevant indust