Model Risk Appetite: Measuring and Managing Risk in Financial Models

Measuring Model Risk Appetite :-

Statements of risk appetite are often based on standard types of metrics. The most common are the quality of models, compliance with MRM policy, and risk capital add-ons. Financial institutions often employ a subjective rating system to quantify model risk levels and establish a standard for disclosure. For example, in Europe, a significant portion of banks categorize the cost of capital associated with model risk under the umbrella of operational risk. Meanwhile, a notable fraction allocates these costs under margin of conservative framework. Additionally, some banks do not allocate any distinct capital reserves for model risk management.

Following could be a list of quantitative & qualitative indicators, which can be fed to a scorecard that can be used to compute model risk capital charge:

Quantitative Indicators:

• Model performance metrics (AUC, Gini coefficient, RMSE) - Example: Minimum Gini coefficient of 0.60 for retail credit scoring against comparable benchmarks
• Model uncertainty measures (confidence intervals, prediction variance) - Example: LGD models must predict within ±15% of actual loss rates for each collateral type
• Error rates and tolerance thresholds - EAD model with maximum 5% variance from observed utilization rates at default
• Count of high-risk models vs. total model inventory - All AIRB regulatory capital models automatically classified as Tier 1 (high-risk)
• Capital reserves allocated to model risk - PD model misclassification limited to 5% impact on total expected loss
• Model validation back testing results
• Number of identified model limitations or assumptions

Qualitative Indicators:

• Model tiering/risk classification (high/medium/low risk models) based on materiality and complexity
• Quality of model documentation
• Extent of expert judgment involved
• Regulatory compliance status
• Model validation coverage and frequency
• Level of independence in model validation
• Sophistication of model governance framework

All these indicators in an absolute sense may not provide any tangible insights. However, if we benchmark these against the baseline methods/models and look at them holistically through a scoring mechanism, they can provide directionality and magnitude impact on P&L and balance sheet metrics. Currently this is being followed in few FIs, and even if they do, model risk scorecard is not typically looked or assessed with same degree of scrutiny as a typical credit scorecard would be. Quite a few FIs still don't categorize model risk as material and there is no obligation to do that as well.

Role of Internal Validation:-

These indicators are thoroughly assessed by internal validation (IV) ensuring models operate within the bank's risk tolerance and comply with regulatory standards. IV findings are factored into the overall risk metrics, helping to set and adjust risk limits, and are crucial for the continuous monitoring.

Ways in which IV can help ascertain model risk through its own processes:

Quantitative measures:

• Performance threshold breaches identified during validation
• Validation override rates for manual model adjustments
• Benchmark comparison results
• Stability of model parameters as assessed by validation
• Number of open validation findings by severity level
• Age of unresolved validation issues

Qualitative measures:

• Model approval status (approved, approved with conditions, rejected)
• Rating system for model quality (satisfactory, needs improvement, unsatisfactory)
• Independent challenge results (validator's testing vs. developer's)
• Validation findings classification (critical, high, medium, low)

Governance Metrics:

• Percentage of models with current validation coverage
• Time to remediate validation findings
• Models operating under validation exceptions
• Validation resource adequacy

Validation's processes ensure accurate and unbiased model KPIs are being captured, eventually making model risk scoring process robust.

Regulatory Models : Impacts & Mitigants

There are no predefined indicators which regulators look at to understand quantum of model risk. However, they look at model development, validation and internal audit functions:

• Their policies and procedures
• Compliance status with specific regulatory standards (Basel, CCAR, CECL)
• Number of regulatory findings by severity level
• Conservatism or prudent measures applicable
• Regulatory reporting accuracy metrics
• Supervisory benchmark comparison results
• Documentation completeness per regulatory expectations

Based on above, outcome of assessments and materiality of findings, the regulator will come up with specific areas of interventions which are not publicly disclosed but discussed through close door consultations and sessions with participating bank, regulators will recommend specific measures to be taken in a time bound manner. Additionally, in interim they could impose capital floors or penalties. The quantum of capital floor applied can be based on peer benchmarking results of similar size, region, portfolios. Example: BASEL's annual benchmark study across banks/regions.

Some examples from such assessments are highlighted below:

Deutsche Bank TRIM Exercise (2019-2020)

• Issue: Systematic underestimation in corporate PD models; failure to capture cyclical industry risks; outdated calibration methodology
• Regulatory Action: €21.5 billion increase in RWA; CET1 ratio impact of approximately -70bps; mandatory full model redevelopment
• Resolution: Implementation of new models by Q4 2020; total remediation cost exceeding €100M

HSBC IRB Model Review (2019)

• Issue: LGD models underestimating downturn conditions; insufficient historical calibration data; governance deficiencies
• Regulatory Action: $35 billion increase in RWA; mandatory review of all wholesale LGD models; enhanced governance requirements
• Resolution: Model redevelopment ($50M); additional validation resources ($15M); ongoing monitoring enhancement (~$10M annually)

Although not mandated but some indicators banks can look at from a regulatory exam readiness perspective, which can minimize dollar impact are:

• Historical supervisory feedback tracking
• Mock examination results
• Gap analysis against updated regulatory guidance
• Remediation progress for previously identified issues
• Potential capital add-ons or multipliers from model deficiencies
• Reputational risk assessment from regulatory actions
• Scenario analysis of potential MRM failures

#ModelRisk #RiskManagement #Banking #FinancialRegulation #RiskAppetite