Model Risk: GARP'sApproach to Managing and Controlling Model Risk

1. What is model risk and why is it important?

Model risk is the possibility of adverse outcomes resulting from decisions based on inaccurate, inappropriate, or misused models. It is a type of operational risk that can affect various aspects of a business, such as financial performance, regulatory compliance, reputation, and customer satisfaction. Model risk can arise from errors in the design, development, implementation, validation, or use of models, as well as from changes in the model environment or assumptions. model risk management (MRM) is the process of identifying, measuring, monitoring, and mitigating model risk throughout the model lifecycle.

Model risk is important for several reasons:

1. Models are widely used in the financial industry for various purposes, such as pricing, valuation, risk measurement, capital allocation, stress testing, and strategic planning. Models can have a significant impact on the financial results and decisions of an organization. Therefore, ensuring the quality, reliability, and suitability of models is essential for sound financial management and governance.

2. Models are subject to uncertainty and limitations due to the simplifications and assumptions they involve. Models may not capture all the relevant factors or dynamics of the real world, or may not reflect the changes in the market conditions or customer behavior. Models may also contain errors or flaws in their logic, data, parameters, or implementation. These sources of model risk can lead to inaccurate or misleading outputs that can affect the model users and stakeholders.

3. Models are subject to regulatory scrutiny and expectations. Regulators have issued guidance and standards on model risk management for financial institutions, such as the SR 11-7 by the Federal Reserve and the OCC in 2011, and the BCBS 233 by the Basel Committee on Banking Supervision in 2019. These documents outline the principles and practices for effective model risk management, such as model governance, development, validation, documentation, and reporting. financial institutions are expected to comply with these requirements and demonstrate their model risk management capabilities to regulators and auditors.

4. Models are subject to reputational and legal risks. If models fail to perform as expected or cause losses or damages to the organization or its customers, it can damage the reputation and trust of the organization in the market. It can also expose the organization to legal liabilities or penalties from regulators or other parties. Therefore, managing model risk is important for protecting the reputation and value of the organization and its stakeholders.

2. From regulatory requirements to business opportunities

Model risk management is the process of identifying, measuring, monitoring, and controlling the risks associated with the use of models in decision making. Models are simplified representations of reality that are used to analyze complex phenomena, such as financial markets, customer behavior, or natural disasters. Models can provide valuable insights and support business decisions, but they can also introduce errors and uncertainties that can have significant impacts on the outcomes. Therefore, model risk management is essential to ensure that models are reliable, accurate, and fit for purpose.

The evolution of model risk management can be traced back to the regulatory requirements that emerged after the global financial crisis of 2007-2009. The crisis exposed the weaknesses and limitations of many models that were used by financial institutions to assess risks, value assets, and allocate capital. Some of the factors that contributed to the model failures were:

1. The use of inappropriate assumptions and data that did not reflect the changing market conditions and dynamics.

2. The lack of validation, testing, and documentation of the models and their results.

3. The over-reliance on models and the underestimation of their uncertainties and limitations.

4. The lack of governance, oversight, and accountability for the development, implementation, and use of models.

As a result of the crisis, regulators around the world issued new guidelines and standards for model risk management, such as the [Supervisory Guidance on Model Risk Management] by the federal Reserve board (FRB) and the Office of the Comptroller of the Currency (OCC) in 2011, or the [Principles for effective Risk data Aggregation and Risk Reporting] by the Basel Committee on Banking Supervision (BCBS) in 2013. These regulations aimed to improve the quality, transparency, and consistency of models and their outputs, as well as to enhance the governance, controls, and auditability of model risk management practices.

However, model risk management is not only a regulatory requirement, but also a business opportunity. By adopting a proactive and holistic approach to model risk management, organizations can benefit from:

- improved decision making: By ensuring that models are robust, relevant, and aligned with the business objectives and strategies, organizations can increase their confidence and trust in the models and their results, and use them to support better decisions.

- Reduced operational costs: By streamlining and standardizing the model development, implementation, and maintenance processes, organizations can reduce the complexity and duplication of models, as well as the resources and time required to manage them.

- Enhanced reputation and compliance: By demonstrating that models are compliant with the regulatory expectations and industry best practices, organizations can avoid potential fines, sanctions, or reputational damage that may arise from model failures or errors.

- Increased innovation and competitiveness: By fostering a culture of continuous improvement and learning from model performance and feedback, organizations can enhance their ability to adapt to changing market conditions and customer needs, as well as to leverage new technologies and data sources to create more advanced and sophisticated models.

Therefore, model risk management is not only a necessity, but also a competitive advantage for organizations that want to succeed in the digital age. By embracing model risk management as a strategic function that adds value to the business, organizations can unlock the full potential of their models and turn them into powerful tools for growth and transformation.

3. Governance, policies, validation, monitoring, and reporting

Model risk is the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports. It can arise from various sources, such as model assumptions, data quality, implementation errors, or inappropriate use. To manage and control model risk, organizations need to establish a robust framework that covers the following main elements: governance, policies, validation, monitoring, and reporting. These elements are essential for ensuring that models are developed, implemented, and used in a consistent, transparent, and effective manner. Let's take a closer look at each of these elements and how they contribute to good model risk management.

1. Governance: Governance refers to the roles and responsibilities of the various stakeholders involved in the model lifecycle, such as model owners, developers, users, validators, auditors, and regulators. Governance also defines the processes and standards for model development, approval, implementation, use, change, and retirement. A clear and well-defined governance structure helps to ensure accountability, oversight, and communication among the stakeholders and to align the model objectives with the business strategy and risk appetite.

2. Policies: Policies are the written documents that specify the principles, guidelines, and procedures for model risk management. Policies should cover all aspects of the model lifecycle, such as model identification, classification, inventory, documentation, validation, monitoring, reporting, and remediation. Policies should also define the roles and responsibilities of the stakeholders, the escalation and resolution mechanisms for model issues, and the frequency and scope of model reviews and audits. Policies should be reviewed and updated regularly to reflect changes in the business environment, regulatory requirements, and industry best practices.

3. Validation: Validation is the independent assessment of the soundness and performance of a model. Validation involves testing the model assumptions, logic, data, implementation, output, and limitations. Validation also evaluates the model's suitability for its intended use and its alignment with the business objectives and risk appetite. Validation provides assurance that the model is fit for purpose and identifies any potential model risks or weaknesses that need to be addressed or mitigated.

4. Monitoring: Monitoring is the ongoing process of verifying that a model is functioning as expected and remains valid over time. Monitoring involves collecting and analyzing data on model inputs, outputs, performance indicators, and risk measures. Monitoring also involves comparing the model results with alternative models or benchmarks, such as historical data, expert judgment, or market prices. Monitoring helps to detect any changes or deviations in the model behavior or performance that may indicate model deterioration or failure.

5. Reporting: reporting is the process of communicating relevant information on model risk to the stakeholders. Reporting should provide a clear and comprehensive overview of the model inventory, status, performance, issues, and actions. Reporting should also highlight any material changes or events that may affect the model risk profile or exposure. Reporting helps to inform decision-making, support risk management activities, and facilitate regulatory compliance.

These are some of the main elements of good model risk management according to GARP's approach. By implementing these elements in a consistent and effective manner, organizations can enhance their ability to manage and control model risk and reduce the potential for negative impacts on their business operations.

: [GARP - Model Risk Management]

4. Complexity, data quality, resource constraints, and stakeholder expectations

Model risk management is the process of identifying, measuring, monitoring, and controlling the risks associated with the use of models in decision making. Models are simplified representations of reality that are used to analyze complex phenomena, such as financial markets, natural disasters, or human behavior. However, models are not perfect and may contain errors, assumptions, limitations, or uncertainties that can lead to inaccurate or misleading results. Therefore, model risk management is essential to ensure that models are reliable, valid, and fit for purpose.

Some of the challenges of model risk management are:

1. Complexity: Models can be very complex and involve sophisticated mathematical techniques, algorithms, or data sources. This makes it difficult to understand how the models work, what are their assumptions and limitations, and how they may behave under different scenarios or stress tests. Moreover, models may interact with each other in complex ways, creating dependencies or feedback loops that can amplify or propagate model risk. For example, a model that predicts the default probability of a borrower may depend on another model that estimates the market value of the collateral. If the latter model is inaccurate or outdated, it may affect the accuracy of the former model and lead to wrong decisions.

2. Data quality: Models rely on data to calibrate their parameters, validate their outputs, and monitor their performance. However, data quality can be compromised by various factors, such as errors, outliers, missing values, inconsistencies, biases, or manipulation. Data quality can also vary over time and across different sources, making it challenging to ensure data consistency and comparability. For example, a model that uses historical data to forecast future events may not account for changes in market conditions, regulations, or customer behavior. Similarly, a model that uses data from different jurisdictions may not account for differences in accounting standards, reporting practices, or legal frameworks.

3. Resource constraints: Model risk management requires adequate resources in terms of time, money, expertise, and technology. However, these resources may be limited or insufficient to cope with the increasing number and complexity of models. For instance, model development and validation may require specialized skills and tools that are not readily available or affordable. Model governance and oversight may require dedicated staff and committees that are independent and accountable. Model documentation and reporting may require standardized formats and procedures that are transparent and consistent.

4. Stakeholder expectations: Model risk management involves various stakeholders who have different roles, responsibilities, interests, and expectations regarding the use of models. These stakeholders may include model developers, users, validators, auditors, regulators, customers, investors, or the public. However, these stakeholders may have conflicting or diverging views on how models should be designed, implemented, tested, reviewed, approved, used, or disclosed. For example, model developers may focus on the technical aspects of the models and overlook their business implications. Model users may rely too much on the models and ignore their limitations or uncertainties. Model validators may apply different standards or criteria to assess the models' quality or performance. Model regulators may impose different requirements or expectations on how models should be governed or controlled.

These challenges pose significant risks to the organizations that use models in their decision making. Therefore, model risk management is not only a technical issue but also a strategic one that requires a holistic and integrated approach that aligns with the organization's objectives and culture.

5. Improved decision making, reduced losses, enhanced reputation, and competitive advantage

Model risk management is the process of identifying, measuring, monitoring, and controlling the risks associated with the use of models in various domains, such as finance, engineering, health care, and others. Models are simplified representations of reality that are used to support decision making, but they also entail uncertainties and limitations that can lead to errors and losses if not properly managed. Therefore, model risk management is essential for ensuring the quality, reliability, and validity of models and their outputs.

Some of the benefits of model risk management are:

1. Improved decision making: Model risk management helps to improve the accuracy and consistency of model outputs, which in turn enhances the quality and effectiveness of decision making. By applying rigorous standards and best practices to model development, validation, implementation, and use, model risk management reduces the likelihood of errors, biases, and inconsistencies that can compromise the soundness of decisions. For example, a financial institution that uses models to assess the creditworthiness of borrowers can improve its lending decisions by implementing a robust model risk management framework that ensures the models are fit for purpose, calibrated to relevant data, tested for sensitivity and stability, and monitored for performance and changes in market conditions.

2. Reduced losses: Model risk management helps to reduce the potential losses that can result from model failures or misuse. Model failures can occur when models produce inaccurate or misleading outputs due to flaws in design, data, assumptions, or parameters. Model misuse can occur when models are used for purposes or contexts that they were not intended or suitable for. Both scenarios can expose the model users to financial, operational, reputational, or regulatory risks that can cause significant losses or damages. For example, a power company that uses models to forecast electricity demand and supply can incur losses if the models fail to account for extreme weather events or changing consumer behavior that affect the electricity market. Model risk management can help to mitigate these risks by ensuring that the models are regularly reviewed, updated, validated, and audited for accuracy and reliability.

3. Enhanced reputation: Model risk management helps to enhance the reputation and credibility of model users among their stakeholders, such as customers, investors, regulators, and competitors. By demonstrating a high level of competence and professionalism in managing model risks, model users can build trust and confidence in their models and their decisions. This can also help them to comply with regulatory requirements and standards that govern the use of models in certain domains or industries. For example, a pharmaceutical company that uses models to support drug development and testing can enhance its reputation by adhering to a rigorous model risk management framework that ensures the models are scientifically sound, ethically acceptable, and clinically relevant.

4. Competitive advantage: Model risk management helps to create a competitive advantage for model users by enabling them to leverage the full potential of models in achieving their goals and objectives. By managing model risks effectively, model users can optimize the performance and efficiency of their models and enhance their value proposition. This can also help them to gain insights and intelligence from their models that can support innovation and differentiation in their products or services. For example, a retail company that uses models to optimize its inventory management and pricing strategies can gain a competitive edge by using a sophisticated model risk management system that allows it to monitor and adjust its models dynamically based on real-time data and market conditions.

Read Other Blogs