Risk Model Validation teaser

RiskModelValidation ByPeterQuellandChristianMeyer
Risk Model
2nd Edition
BY PETER QUELL AND CHRISTIAN MEYER
Validation
Worldwide, senior executives and managers
in financial and non-financial firms are
expected to make crucial business decisions
based on the results of complex risk models.
Yet interpreting the findings, understanding
the limitations of the models and recognising
the assumptions that underpin them can
present considerable challenges for all but
those with a background in specialised
quantitative financial modelling.
Additionally, regulatory authorities are
exponentially more interested in quantifying
model risk, and this increased regulatory
focus on risk model validation and model
risk means that financial institutions have
to come up with solutions in the immediate
future. Risk Model Validation (2nd edition)
covers the failures as well as the successes of
risk models and putting them in the context
of real-world examples, you will be able to
assess the potential limits of risk models in
your organisation.
In this fully updated second edition of
Risk Model Validation, authors Christian Meyer
and Peter Quell give a holistic view of risk
models: their construction, appropriateness,
validation and why they play such an
important role in the financial markets.
Christian Meyer and Peter Quell guide the
reader through the process of risk modelling,
demonstrating how to interpret their
findings, how to understand the limitations
of certain models, and how to recognise the
assumptions that underpin them.
Readers will be able to:
• Evaluate the validity of a model;
• Judge the model’s quality, consistency
and regulatory compliance;
• Improve a framework for validation; and
• Tailor a model-risk approach for their
institution.
Risk Model Validation (2nd edition) once
again demonstrates how risk models are
constructed and why they play such an
important role in the financial markets and
the regulatory framework that surrounds
them. This book provides financial
institutions with a toolbox to raise the key
questions when it comes to integrating
the results of quantitative risk models into
business decisions.
PEFC Certified
This book has been
produced entirely from
sustainable papers that
are accredited as PEFC
compliant.
www.pefc.org
RMV-Ed2.indd 1 16/05/2016 12:06

Risk Model Validation
00 Prelims RMV.indd 1 04/05/2016 21:52
Buy your copy today with 25% off using code: RMV25. Visit www.riskbooks.com and search Validation.

00 Prelims RMV.indd 2 04/05/2016 21:52

Risk Model Validation
Second Edition
Christian Meyer and Peter Quell
00 Prelims RMV.indd 3 04/05/2016 21:52

Published by Risk Books, a Division of Incisive Media Investments Ltd
Incisive Media
Haymarket House 28–29 Haymarket
London SW1Y 4RX
Tel: +44(0) 20 7316 9000
E-mail: books@incisivemedia.com
Sites: www.riskbooks.com
www.incisivemedia.com
© 2016 Incisive Risk Information Limited
ISBN 978 1 78272 263 2
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
Publisher: Nick Carver
Commissioning Editor: Alice Levick
Managing Editor: Lewis O’Sullivan
Designer: Lisa Ling
Copyeditor: Laurie Donaldson
Typeset by Mark Heslington Ltd, Scarborough, North Yorkshire
Printed and bound in the UK by PrintonDemand-Worldwide
Conditions of sale
All rights reserved. No part of this publication may be reproduced in any material form whether
by photocopying or storing in any medium by electronic means whether or not transiently
or incidentally to some other use for this publication without the prior written consent of the
copyright owner except in accordance with the provisions of the Copyright, Designs and Patents
Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Limited of
Saffron House, 6–10 Kirby Street, London EC1N 8TS, UK.
Warning: the doing of any unauthorised act in relation to this work may result in both civil and
criminal liability.
Every effort has been made to ensure the accuracy of the text at the time of publication, this
includes efforts to contact each author to ensure the accuracy of their details at publication is
correct. However, no responsibility for loss occasioned to any person acting or refraining from
acting as a result of the material contained in this publication will be accepted by the copyright
owner, the editor, the authors or Incisive Media.
Many of the product names contained in this publication are registered trade marks, and Risk
Books has made every effort to print them with the capitalisation and punctuation used by the
trademark owner. For reasons of textual clarity, it is not our house style to use symbols such
as TM, ®, etc. However, the absence of such symbols should not be taken to indicate absence of
trademark protection; anyone wishing to use product names in the public domain should first
clear such use with the product owner.
While best efforts have been intended for the preparation of this book, neither the publisher, the
editor nor any of the potentially implicitly affiliated organisations accept responsibility for any
errors, mistakes and or omissions it may provide or for any losses howsoever arising from or in
reliance upon its information, meanings and interpretations by any parties.
00 Prelims RMV.indd 4 04/05/2016 21:52

We would like to thank our colleagues at DZ BANK AG for valuable
discussions, as well as the team at Incisive Media for guiding the
manuscript through the different stages of the production process.
00 Prelims RMV.indd 5 04/05/2016 21:52

Disclaimer
The opinions and recommendations expressed in this book are
those of the authors and are not representative of their current or
previous employers.
00 Prelims RMV.indd 6 04/05/2016 21:52

vii
About the Authors ix
Abbreviationsxi
Introduction1
PART I: QUANTITATIVE RISK MODELS5
1 Basics of Quantitative Risk Models 7
2 Usage of Statistics in Quantitative Risk Models 21
3 How Can a Risk Model Fail? 41
PART II: MODEL RISK AND RISK MODEL VALIDATION77
4 The Concepts of Model Risk and Validation 79
5 Model Risk Frameworks 103
6 Validation Tools 119
7 Regulation 161
PART III: MODEL RISK IN MARKET RISK MODELS187
8 The Short-term Perspective 189
9 A Benchmark Model for Market Risk 207
10 The Medium-term Perspective 247
Contents
00 Prelims RMV.indd 7 04/05/2016 21:52

risk model validation
viii
PART IV: MODEL RISK IN CREDIT RISK MODELS271
11 Modelling and Simulation 273
12 Data 293
13 Model Results 319
14 Conclusion 345
References 351
Index 363
00 Prelims RMV.indd 8 04/05/2016 21:52

ix
Christian Meyer is a quantitative analyst in the portfolio analytics
team for market and credit risk in the risk controlling unit of DZ
BANK AG in Frankfurt, where he is responsible for the develop-
ment of portfolio models for credit risk and spread risk in the
banking book and incremental risk in the trading book. Before
joining DZ BANK, he worked at KPMG, where he dealt with
various audit and consulting aspects of market risk, credit risk and
economic capital models in the banking industry. Christian holds a
diploma and PhD in mathematics, and is on the editorial board of
the Journal of Risk Model Validation.
Peter Quell is head of the portfolio analytics team for market and
credit risk in the risk controlling unit of DZ BANK AG in Frankfurt.
Before joining DZ BANK, he was manager at d-fine GmbH, where
he dealt with various aspects of risk management systems in the
banking industry. Peter holds an MSc in mathematical finance from
Oxford University and a PhD in mathematics, and is on the edito-
rial board of the Journal of Risk Model Validation.
About the Authors
00 Prelims RMV.indd 9 04/05/2016 21:52

00 Prelims RMV.indd 10 04/05/2016 21:52

xi
A-IRBA Advanced internal ratings-based approach
ABS Asset-backed security
ACF Autocorrelation function
AIG Accord Implementation Group
AIGV Accord Implementation Group related to Validation
AMA Advanced measurement approach
ARMA Autoregressive–moving-average
BCBS Basel Committee on Banking Supervision
BIS Bank for International Settlements
CAP Cumulative accuracy profile
CCR Counterparty credit risk
CDO Collateralised debt obligation
CDS Credit default swap
CPU Central processing unit
CRM Comprehensive risk measure
CTP Correlation trading portfolio
CVA Credit value adjustment
DMM Direct moment matching
DRC Default risk charge
EAD Exposure at default
EBA European Banking Authority
EE Expected exposure
EEPE Effective expected positive exposure
EL Expected loss
EPE Expected positive exposure
EWMA Exponentially weighted moving average
EU European Union
F-IRBA Foundation internal ratings-based approach
FRTB Fundamental Review of the Trading Book
FSA Financial Services Authority
G-10 Group of 10
Abbreviations
00 Prelims RMV.indd 11 04/05/2016 21:52

xii
GARCH Generalised autoregressive conditional
heteroscedasticity
GIGO Garbage in, garbage out
i.i.d. Independent and identically distributed
ICAAP Internal capital adequacy assessment process
IFoA Institute and Faculty of Actuaries
IMM Indirect moment matching
IoR Impact of risk
IRB Internal ratings-based
IRC Incremental risk charge
LGD Loss given default
MAD Mean absolute deviation
MRF Modellable risk factor
NMRF Non-modellable risk factor
NPL Non-performing loan
OCC Office of the Comptroller of the Currency
OTC Over-the-counter
PL Profit and loss
PD Probability of default
PDE Partial differential equation
PIT Point-in-time
PL Performing loan
PRNG Pseudo random number generator
QA Quality assurance
QQ Quantile–quantile
QRM Quantitative risk model
RBA Ratings-based approach
RNG Random number generator
ROC Receiver operating characteristic
SFT Securities financing transaction
SREP Supervisory Review and Evaluation Process
TTC Through-the-cycle
VV Verification and validation
VaR Value-at-risk
VVUQ Verification, Validation and Uncertainty
Quantification
00 Prelims RMV.indd 12 04/05/2016 21:52

1
The last decade of the 20th century and the first decade of the 21st
century saw the use of quantitative risk models (QRMs) become a
cornerstone of financial regulation. Financial institutions now have
to determine capital buffers based on increasingly complex model-
ling techniques. The computation and allocation of capital buffers
to business lines has come to share more features with the solution
of multi-dimensional optimisation problems than with classical
business model analysis. As a consequence, the Turner review
(FSA, 2009) speculated that many in top management face difficul-
ties in assessing and exercising judgement over the risks being
taken by their institutions. According to the review, these difficul-
ties contributed to one of the worst crises that our financial system
has ever experienced.
Of course, the Turner review also mentioned many other factors
that may have played a prominent role, such as the heightened
complexity of the structured credit market, the fact that financial
institutions took on too much leverage and the role of the rating
agencies. However, this global financial crisis definitely empha-
sised the need for rigorous and critical analysis of the use and
misuse of risk models, and the results they can produce.
Although the existing literature presents many contributions
from specialists into the use of risk modelling, ranging from quali-
tative aspects to sophisticated statistical validation techniques,
their highly specialised nature means they are not hugely acces-
sible to managers without training in quantitative finance. This
book, based on the authors’ practical experiences of establishing
models for both market risk and credit risk in the banking industry,
will focus on providing a holistic perspective aimed at the
“informed layperson”, with some technical aspects added where
needed. It will attempt to answer the following key questions.
Introduction
00 Prelims RMV.indd 1 04/05/2016 21:52

2
❏❏ How can we establish a practical framework for thinking about
risk?
❏❏ Were there risk models before the Basel framework?
❏❏ What are the common features of existing QRMs?
❏❏ How can risk models fail?
❏❏ What are the limits to risk modelling?
❏❏ What are the challenges when implementing risk models in
software?
In particular, the following important aspects concerning the usage
of risk models in practice will be emphasised.
❏❏ Risk models have the potential to be useful. In essence, a QRM
allows for the analysis of a quantity of interest (portfolio profit
and loss (PL), insurance claims, etc) under several potential
future scenarios. That is, it can be compared to extensive “what
if” analysis. In principle, since the scenarios are given explicitly,
there is a sound basis for comprehensible risk assessment – ie,
the recipients of risk reports can trace back why and how one
arrived at the current risk estimate. As a consequence, the
scenarios also provide a starting point for discussions about
what interventions management can take in order to actively
manage risk.
❏❏ Results produced by risk models require interpretation. There is
a growing tendency in the banking industry to obtain value-at-
risk (VaR) estimates based on very high confidence levels. For
example, in the context of economic capital it is not unusual to
encounter 99.975% (or even 99.99%) in association with loss esti-
mates covering a risk horizon of one year. Unfortunately, these
risk estimates are sometimes interpreted as “losses that will
occur only once in 4,000 (or 10,000) years”. Can we seriously
claim to know how the world (or the financial markets) will look
like even 20 years from now? In other words, as with all things
that are potentially useful (eg, cars, computers, money, food),
there is also the potential for misuse.
❏❏ There are limits to risk modelling. In a similar way to models
built in the natural sciences or engineering, risk models are
simplified representations of reality. However, compared with
such other models, risk models are supposed to reflect the
00 Prelims RMV.indd 2 04/05/2016 21:52

introduction
3
behaviour of quantities emerging in social systems. For example,
market participants may react to the way risks are measured and
interpreted and, potentially, in doing so invalidate the assump-
tions the models are based upon. Such feedback effects may
seriously contribute to the development of financial crises, and
may therefore render modelling of such crises practically
impossible.
It has become common practice to address these aspects under the
label of “model risk” in order to manage them similarly to other
types of risk. In particular, “risk model validation” aims to address
such aspects in the context of a concrete risk model implementa-
tion. Of course, there are numerous specific validation tools
available, and this book will describe these tools and their applica-
tion in practice. However, it will always focus on the holistic
perspective of model risk and validation that should be kept in
mind. To further define our terms, model risk refers to the potential
of a specific QRM implementation not being useful, while risk
model validation is the collection of all activities aimed at assessing
if a specific QRM implementation is useful.
The intention of this book is to ensure that the reader has enough
information to be aware of model risk issues and to raise topics of
validation in the context of a specific QRM, to judge the quality of
an existing model risk and validation framework, and to initiate
activities for setting up or improving such a framework. It builds
on our earlier book on risk model validation (published in 2011),
which has been heavily extended. It has been organised into four
parts, as follows.
❏❏ Part I starts by thinking about risk and risk models in the first
place, illustrating these concepts using a historical example in
Chapter 1. Chapter 2 then focuses on quantitative risk models as
used by participants in financial markets, and explains the use of
statistics within these models. Chapter 3 goes into detail about
how a risk model might fail at the stages of design, implementa-
tion, data, processes and use.
❏❏ Part II addresses the problems identified in Part I using the
concepts of model risk and risk model validation. Chapter 4
defines these terms, notes some general issues and provides an
00 Prelims RMV.indd 3 04/05/2016 21:52

4
extensive overview of the related literature. Chapter 5 discusses
model risk governance (ie, model risk management as a social
activity) and model risk mitigation (ie, model risk management
as a process). Risk model validation is one major tool for model
risk mitigation, and Chapter 6 goes into detail about possible
validation tools. Chapter 7 presents the regulatory framework
for using internal risk models, and comments on regulatory
expectations regarding risk model validation.
❏❏ Part III deals with model risk issues that are specific to market
risk models. Chapter 8 starts by discussing model risk from a
short-term perspective (ie, one trading day). Chapter 9 presents
a benchmark model for market risk that is able to adapt to
changing market conditions. Chapter 10 discusses problems that
occur when the risk horizon is extended to the medium term (eg,
to 10 trading days).
❏❏ Part IV, in an analogous way, treats model risk issues that are
specific to credit risk models. Chapter 11 focuses on model
design and issues with simulation. Chapter 12 examines the data
typically needed to feed credit risk models, while Chapter 13
brings together the previous two chapters and discusses effects
of model risk issues on model results, before presenting possibil-
ities for benchmarking.
Note that Parts III and IV are slightly more technical than the rest of
the book. Parts I and II can be read together as a general introduc-
tion to risk models, model risk and risk model validation, and Parts
III and IV can stand on their own, as well as illustrate the general
issues presented in the first two parts.
The references have been updated and extended, and the latest
regulatory developments – such as the “Fundamental Review of
the Trading Book”, the guidelines on model risk issued by the
Office of the Comptroller of the Currency, OCC, or the guidelines
for the “Supervisory Review and Evaluation Process” (SREP)
issued by the European Banking Authority (EBA) – have been
included.
00 Prelims RMV.indd 4 04/05/2016 21:52

Part I
Quantitative Risk Models
01 Chapter RMV.indd 5 04/05/2016 19:52

01 Chapter RMV.indd 6 04/05/2016 19:52

7
Before examining aspects of risk model validation in detail, this
chapter will explore the task of explaining how risk can be concep-
tualised, and how this conceptualisation can be used to design
quantitative risk models. It starts with some historical factors
concerning the evolution of quantitative risk modelling, explaining
how the concept of risk is not as clearly defined as one might think,
and how its perception and modelling contain many subjective
aspects.
This will be followed by a description of the basic elements of
QRMs, which will be relatively generic and applicable to different
kinds of risk assessment problems, such as in business planning or
risk management in insurance companies or the banking industry.
The chapter will then explore features of risk and risk diversifica-
tion that were already present in ancient times, using an historical
example to pinpoint key elements. In more modern times, the
development of probability theory and statistics from the late 17th
century onward provided an important context for how quantita-
tive risk assessment could be conceptualised. The chapter will
therefore start to explore the usage of statistics in QRMs and
present some examples where these techniques could be useful, as
well as emphasising limitations that should be kept in mind.
Finally, there will be a description of some procedures that are typi-
cally undertaken during the practical implementation of QRMs.
However, this chapter will not provide an exhaustive treatment
of all aspects of risk and QRMs, but rather will integrate numerous
references. In addition, as general background reading, the
following contributions are recommended.
1
Basics of Quantitative Risk Models
01 Chapter RMV.indd 7 04/05/2016 19:52

8
❏❏ For historical aspects of quantitative risk modelling, as well as
general reading: Bernstein (1996); Bookstaber (2007); Fox (2009);
Brown (2012); and Crouhy et al (2014).
❏❏ For problems in dealing with randomness, and fundamental
critique: Taleb (2005); Taleb (2010); Taleb (2012); and Rebonato
(2007).
❏❏ For quantitative aspects: McNeil, Frey and Embrechts (2015);
Holton (2003); and Jorion (2007).
❏❏ For scenario generation: Rebonato (2010); Ziemba and Ziemba
(2007); and Kindleberger and Aliber (2005).
THINKING ABOUT RISK
“Risk” is part of everybody’s daily experience. We face risks when
using any means of transport, and have to cope with risks when
buying a house or making other important life or investment deci-
sions. We also encounter risks when taking part in sports and a
certain risk if taking no exercise at all. Risk seems to be inherent in
so many decisions, considering the potential but unknown future
consequences these decisions could have.
The way in which terms such as risk management, risk insur-
ance, risk mitigation and risk avoidance are used can give the
impression that risk is a well-defined concept, and implies there is
a common understanding of its main characteristics. However,
when it comes to a precise definition of what is meant by risk, a
sampling of the corresponding literature offers a much more
diverse picture.
EARLY 20TH CENTURY ECONOMICS
One of the first systematic inquiries into the nature of risk is prob-
ably Frank H. Knight’s seminal book on Risk, Uncertainty and Profit
(Knight 1921). The key element proposed by Knight is the distinc-
tion between risk as a quantifiable topic and uncertainty as a
non-quantifiable topic:
But Uncertainty must be taken in a sense radically distinct from the
familiar notion of Risk, from which it has never been properly
separated. The term “risk”, as loosely used in everyday speech and
in economic discussion, really covers two things which, function-
ally at least, in their causal relations to the phenomena of economic
organisation, are categorically different.[...] The essential fact is that
“risk” means in some cases a quantity susceptible of measurement,
01 Chapter RMV.indd 8 04/05/2016 19:52

basics of quantitative risk models
9
while at other times it is something distinctly not of this character;
and there are far reaching and crucial differences in the bearings of
the phenomenon depending on which of the two is really present
and operating.[...] It will appear that a measurable uncertainty, or
“risk” proper, as we shall use the term, is so far different from an
unmeasurable one that it is not in effect an uncertainty at all. We
shall accordingly restrict the term “uncertainty” to cases of the
non-quantitative type.
In short, Knight thinks of risk as being related to (outcomes of)
events that show a certain form of regularity, cases where one can
employ an intuitive notion of characterising probabilities as
frequencies. This applies when one throws a conventional six-sided
die, where the probability of throwing the number five is 1/6 by
symmetry arguments. It also applies to statistical inferences based
on homogeneous data – for example, when statisticians analyse the
distribution of body heights of children of a given age. In contrast,
uncertainty is related to (outcomes of) events that are in a certain
way unique, irregular, without comparable precedent or without a
good theory to guide us.
Around the same time, John Maynard Keynes finished his book
A Treatise on Probability (Keynes 1920). Bernstein (1996) summarises
Keynes’ view on quantifiability as: “An objective probability of
some future event does exist[...] but our ignorance denies us the
certainty of knowing what that probability is; we can only fall back
on estimates.” Keynes also shared Knight’s scepticism, arguing
that uncertainty rather than mathematical probability is the driving
force behind economic activity.
Engineering
Up to now, this chapter has focused on early 20th century views on
risk within a framework developed by economists. However, in the
discussion around the usage of probability, an important aspect
was missing. It is of course interesting to follow the throwing of a
dice or the outcomes produced by a roulette wheel in a casino, but
if one does not have a big stake in the game there is hardly any risk
involved, no matter the definition. It might be helpful, therefore, to
consider the very different situation of a possible natural, environ-
mental or engineering disaster (Kelman 2003).
Below are some of the characterisations in which exposure
becomes evident:
01 Chapter RMV.indd 9 04/05/2016 19:52

10
❏❏ “Risk is a combination of the chance of a particular event, with
the impact that the event would cause if it occurred. Risk there-
fore has two components – the chance (or probability) of an
event occurring and the impact (or consequence) associated with
that event. The consequence of an event may be either desirable
or undesirable.[...] In some, but not all cases, therefore a conven-
ient single measure of the importance of a risk is given by: Risk =
Probability x Consequence.” (Sayers et al, 2002)
❏❏ “Risk might be defined simply as the probability of the occur-
rence of an undesired event [but] be better described as the
probability of a hazard contributing to a potential disaster.[...]
Importantly, it involves consideration of vulnerability to the
hazard.” (Stenchion, 1997)
❏❏ Risk is “Expected losses (of lives, persons injured, property
damaged and economic activity disrupted) due to a particular
hazard for a given area and reference period. Based on mathe-
matical calculations, risk is the product of hazard and
vulnerability.” (UNDHA, 1992)
Note also the occurrence of the terms “probability” and “mathe-
matical calculations” in these definitions: they clearly refer to
quantifiability, and therefore implicitly adopt the Knightian view
of risk.
Portfolio and investment theory
The engineering definitions of risk focus almost exclusively on
downside risk – ie, on just considering exposure to potential
threats. In the context of investment theory, of course, one does not
only include potentially negative outcomes in the analysis but also
the upside potential. The first major contribution to this subject
was Harry Markowitz’ work on portfolio selection (Markowitz
1952), which used the standard deviation or variance of returns as a
characteristic of an investment opportunity under consideration. In
so doing, Markowitz gave equal credit to potential gains and
potential losses. Then again, he was careful not to define risk as the
variance of returns, instead characterising variance as an undesir-
able thing and (positive) returns as a desirable thing.
Although Markowitz’ approach was not initially intended to be
a foundation for financial risk analysis, much of his (and his
01 Chapter RMV.indd 10 04/05/2016 19:52

11
successors’) assumptions, such as using normal distributions (ie,
the bell curve) and their standard deviation, linear index models
and optimisation, have found their way into the toolboxes of finan-
cial risk measurement, especially for regulatory and policy
purposes.
Sociology
Somewhat different concepts of risk that are not so heavily reliant
on quantification have been developed in the field of sociology. An
excellent reference in this context is Jakob Arnoldi’s book Risk
(Arnoldi 2009), which presents three main approaches that never-
theless do not encompass all sociological aspects of risk.
❏❏ The theory of reflexive modernisation: Threats resulting from
new technologies pose enormous problems for scientists, politi-
cians, the mass media and the public. Due to the complexity,
uncertainty and magnitude of potential consequences of the
introduction of new technologies, there is a lack of solid (theoret-
ical) foundations for decision-making, which itself creates
problems over responsibility and what needs to be done.
❏❏ The cultural theory of risk: There is a cultural logic that under-
pins the differences between what people are afraid of and what
risks they are willing (or ready) to take. For example, most indi-
viduals hold fears about their life and their health, while a fear
about global warming is not likely to be as widespread.
❏❏ The governmental tradition: The presence of risk can be
employed for government techniques to change the behaviour of
people. For example, consider public campaigns related to the
potential dangers of smoking and drug abuse, or the design of
social insurance systems and the corresponding incentives these
systems establish.
These three approaches may sound quite abstract, but they involve
an issue not previously mentioned: reflexivity (or feedback). In its
most general definition, reflexivity describes some circular rela-
tionship between cause and effect. In the sociological context,
reflexivity is considered to occur when the observations or actions
of observers in a social system affect the very situations they are
observing. An obvious example for this can be seen in the financial
01 Chapter RMV.indd 11 04/05/2016 19:52

12
markets, where the interplay between beliefs and observations
directly influences prices. If (enough) market participants believe
that prices for a certain security will fall, they will sell, potentially
driving down prices even further. The same logic can be applied to
situations where (enough) market participants believe in rising
prices, resulting in increased demand and therefore rising prices
(Soros, 1994).
Although the presence of feedback seems quite obvious, it is
possibly one of the more complicated elements of risk modelling.
The reader will have noticed the use of the word “enough” to clas-
sify the number of market participants jumping on the bandwagon.
This is an important aspect of modelling risk in financial markets
or social systems: people may react to what they think that the
decision of the majority of people will be. Here the situation resem-
bles the so-called “Keynes beauty contest” (Keynes 1936), in which
the judges in a beauty contest are asked not to pick their favourite
candidate but the one they think will attract most votes from the
whole jury.
A working definition of risk
In closing this section, it is useful to draw attention to a book
(currently only available in Italian and German) by the Italian soci-
ologist Elena Esposito (Esposito 2007) that applies the sociological
methodology to questions arising in the economy. In the context of
this book, a working definition of risk with an epistemological
flavour is proposed: that risk is a situation with an uncertain future
outcome that is of importance to us (cf Holton 2004). This brief
description includes three main elements.
❏❏ Uncertainty: This involves facing a situation with lack of knowl-
edge or, maybe even more importantly, being already aware of a
lack of knowledge. At this point, we do not differentiate between
quantifiable (eg, in terms of probabilities or frequencies of occur-
rence) and non-quantifiable uncertainties. The way this book
uses the terms risk and uncertainty will therefore be somewhat
different from the way Knight used the terminology.
❏❏ Exposure: We suspect that the unknown outcome under consid-
eration is of importance to us – ie, we would care. Here the
phrase “would care” was deliberately used, since a person is still
01 Chapter RMV.indd 12 04/05/2016 19:52

13
exposed to unknown outcomes if (temporarily) in a state of
impaired consciousness – eg, while asleep or busy with activities
other than risk assessment.
❏❏ Subjectivity: A subtle issue in the above definition is the role of
the person carrying out the risk analysis. This relates to at least
two aspects. First, a person can only reason about perceived
uncertainty and perceived exposure; for example, if there is a
new technology or a new therapy for a disease, different experts
(even after sharing their knowledge) may come to radically
different conclusions with respect to the corresponding risks.
The individual perception of uncertainty and exposure may be
based on personal experience, the outcome of laboratory experi-
ments or the careful collection of data. A second issue is
especially crucial when it comes to the task of quantifying risks.
Suppose that different experts have agreed on the expected
consequences of a new technology. Does that mean, if the
expected consequences are acceptable, the new technology
should be embraced? Or should we care about the worst-case
scenario in connection with the new technology? In other words,
when it comes to the quantification of risk, it is important to have
a risk measure, and that risk measure is almost always not
uniquely defined. Presumably, the usage of different risk meas-
ures would lead to different judgements concerning the
introduction of the new technology.
So, where does referencing economics, finance, engineering and
sociology leave us when it comes to defining risk? There is (at least
up to now) no precise, universally accepted definition of risk. The
only possibility is to look for similar structures in the reasoning
about risk. For this purpose, this book will identify three main
aspects when it comes to risk model validation:
❏❏ perceived uncertainty;
❏❏ perceived exposure; and
❏❏ the issue of subjectivity.
However, in most situations the debate around a risk assessment
being right or wrong may be superfluous. Instead, one should ask
if the risk assessment is actually useful or not.
01 Chapter RMV.indd 13 04/05/2016 19:52

14
ELEMENTS OF QUANTITATIVE RISK MODELS
Many industries have witnessed an enormous increase in efforts to
model risks. These efforts have often, but not exclusively, been
directed towards QRMs. In the banking and insurance industry,
this is in part due to the introduction of regulatory frameworks
such as Basel II, Basel III (see Chapter 7) and Solvency II (cf EU,
2009), which heavily rely on the quantification of risks. In this
context, one is confronted frequently by elements such as market
risk, credit risk, operational risk, business risk, strategic risk, insur-
ance risk, liquidity risk and concentration risk. The chemical and
steel industries, for instance, are obviously confronted with
commodity risk on the buy side as well as on the sell side. Moreover,
energy risk, and of course operational risk, is of primary concern.
Utilities and airline companies are also usually exposed to the risk
of fluctuating oil prices, not to mention their dependence on other
macroeconomic conditions.
Quantitative risk modelling is a framework that contains three
main elements.
❏❏ A quantity of interest the future value of which, referring to a
specific point in time or period of time (the risk horizon), is
uncertain. Examples include the value of a portfolio of financial
instruments 10 business days from now, the revenue of a certain
business over the next five years, and the number of new clients
at the end of the current calendar year.
❏❏ A set of potential future scenarios that describe possible values
of that quantity of interest. Examples include the potential future
value of a portfolio of financial instruments 10 business days
from now after making a specific investment decision, the
revenue of a certain business over the next five years after
changing parts of the business model, and the number of new
clients at the end of the current calendar year after the latest
advertising campaign. To render quantitative analysis possible,
each potential future scenario is equipped with a weight signal-
ling its importance relative to the other potential future scenarios.
❏❏ A statistic or risk measure to sum up the essential information
obtained from the analysis of the potential future scenarios.
Examples include the value of a portfolio of financial instru-
ments 10 business days from now in the worst-case scenario, the
01 Chapter RMV.indd 14 04/05/2016 19:52

15
revenue of a certain business over the next five years corre-
sponding to the scenario with the largest weight, or the average
number of new clients at the end of the current calendar year
over all scenarios.
To summarise, we can think of QRMs as a structured way to reflect
on the future in terms of scenarios. Of course, the performance of
this approach in practice will depend crucially on the quality of
efforts invested in each of these three elements, as well as on the
expectations management has regarding the results. These aspects
will be repeatedly emphasised throughout this book. However, a
few general words of caution are in order.
❏❏ Completeness of scenario sets: In all cases of practical relevance,
one obviously cannot say ex ante which future scenario will
manifest itself. Moreover, one can get into the situation of being
unable to claim the completeness of the set of potential future
scenarios. This applies particularly to the case of rare events that
are often of special interest in risk assessment and risk manage-
ment. Rare events are seldom reflected adequately in historical
data, and are also sometimes hard to capture in the context of
expert judgement.
❏❏ Feedback effects: The situation may become even more compli-
cated due to the presence of feedback. Since the choice of
potential future scenarios and subsequent decisions based on
these scenarios may influence other market participants to adapt
their strategies, some of the scenarios may quickly turn out to be
of a different level of importance than was anticipated. Therefore,
it is necessary to update the set of potential future scenarios, or
at least the weights attached to them, on an ongoing basis.
❏❏ Communication of results: The reporting and communication of
results of QRMs should always be complemented with a
summary of the main assumptions used in deriving the numbers,
which helps to avoid complacency. It is also worth noting that
QRMs can only capture perceived risk – ie, perceived uncer-
tainty and perceived exposure (see previous section on a
definition of risk).
❏❏ Validation: Is the risk model useful? Every model is a simplifica-
tion of reality, an observation that applies especially to the
framework of QRMs, which are based frequently on statistical
01 Chapter RMV.indd 15 04/05/2016 19:52

16
techniques and stochastic modelling. For risk model validation,
the correct question is not: “Is the risk model right?” or “Is the
risk model correct?”, but “Is the risk model useful?” (see Chapter
4 for more detail). A QRM for the same quantity of interest (eg,
future value of a specific portfolio 10 business days from now)
may look quite different for different institutions. For example,
under the Basel II and Basel III regulatory regime (see Chapter
7), banks are able to choose between the so-called standard
approach and the internal model approach for market risk.
Although the standard approach may be considered as being
derived from a QRM, a bank may still find it more useful to
construct an internal model to allow for diversification or netting
effects between financial instruments within the portfolio.
HISTORICAL EXAMPLE
In ancient Greece, people were aware of the inherent risks of mari-
time trading. Merchants were exposed to the danger of losing their
freight (and sometimes their lives) due to storms or piracy. Even if
they were able to ship their goods, they were still exposed to market
risk – ie, the uncertainty of the price at which they would be able to
sell their goods. Usually, merchants would need to take out a loan
to buy the commodities and products they wanted to sell.
Therefore, without the proceeds from sales or a large capital buffer,
they ran the risk of defaulting on their loans. For merchants with a
small- or medium-sized business, that could have been a consider-
able portion of risk.
Merchants in ancient Greece were able to access high-interest
maritime loans that had to be repaid only if the insured cargo made
it safely to its port of destination. If, on the other hand, the lender
allowed several of these maritime loans, the loss incurred from one
non-performing loan could be compensated by the high interest
earned on the performing loans. This constitutes an early form of
maritime insurance. Unfortunately, there is not much known about
how these loans were priced – ie, how the risk premium was deter-
mined by prevailing weather conditions, prevalence of piracy, etc.
As Bernstein (1996) mentions, the Roman emperor Claudius tried
to foster trade by taking personal responsibility for storm losses
that were incurred by merchants. In other words, he provided a
premium-free insurance for certain types of maritime trading.
01 Chapter RMV.indd 16 04/05/2016 19:52

17
There have also been some severe setbacks to this kind of insur-
ance business through the ages. Economic calculations of risk
conflicted with the Church’s view in mediaeval Europe (Daston,
1988; Arnoldi, 2009). Taking interest on loans was considered usury,
and the type of maritime cargo insurance discussed above was
therefore banned by papal decree in 1237. A major reason why the
papal ban did not happen earlier seems to be rooted (Franklin,
2001; Arnoldi, 2009) in the difficulty of separating taking interest
(which was forbidden) from elements of risk sharing (which was
not forbidden). In the aftermath of the papal ban, there may have
been incentives to develop new concepts of risk that avoided the
problems of earlier insurance contracts.
In the context of techniques for the mitigation of risks associated
with maritime trading, a quote from Shakespeare is appropriate. In
The Merchant of Venice, Antonio tells us: “My ventures are not on
one bottom trusted/Not to one place; nor is my whole estate/Upon
the fortune of this present year;/Therefore my merchandise makes
me not sad.” This could be interpreted as an allusion to diversifica-
tion or to risk reduction techniques. A small example may give us a
first impression how effective these risk mitigation concepts can be.
Assume for the moment that Antonio did not act according to his
own words, but did place all his goods in one bottom (ie, in one
ship), and assume that there is a 10% chance of losing that ship
upon the next journey. Finally, also assume that if the ship success-
fully completes its voyage, Antonio will receive the full proceeds P
(let us exclude market risks for the moment).
Going back to the conceptual framework of QRMs presented
earlier, one can identify the following.
❏❏ The quantity of interest: The proceeds from the next voyage (ie,
the risk horizon is chosen not to extend beyond the next voyage).
❏❏ The set of potential future scenarios: Obviously there are two
scenarios:
❍❍ scenario 1: the goods will make their way through to the desti-
nation and Antonio will receive the proceeds P; this scenario
is assigned a weight of, say, 90%; and
❍❍ scenario 2: the goods will not arrive (due to bad weather,
pirates, etc) and Antonio will lose all his goods; this scenario is
assigned a weight of 10%.
01 Chapter RMV.indd 17 04/05/2016 19:52

18
❏❏ The risk measure: Antonio could of course compute the expecta-
tion: Expected proceeds = 90% x P + 10% x 0.
However, computing expectations makes little sense in this case
because Antonio cannot necessarily repeat this experiment too
often (once the ship is lost, he may be bankrupt, for instance).
Therefore, Antonio would be wise to choose the worst-case scenario
(ie, scenario 2) as his statistic in this example. Based on this anal-
ysis, one would advise Antonio not to embark on this journey but
to conceive another strategy.
The reader may have noticed the problem with the above
strategy: statistics is of limited use if one cannot repeat the experi-
ment! Once the ship is lost (and that is assumed to happen with a
probability of 10%), Antonio cannot continue to trade. An obvious
alternative would be to distribute Antonio’s freight evenly onto
several ships. Retain the assumptions made above, and also assume
that the fate of the different ships carrying Antonio’s goods are
independent of one another, although that assumption could be
challenged – for instance, if there are concentrations of bad weather
fronts or concentrations of pirate activities. Therefore, one can then
identify the following.
❏❏ The quantity of interest: The proceeds from the next voyage (as
before).
❏❏ The set of potential future scenarios: Assume that Antonio
distributes his freight over 100 independent ships; this gives 2100
weighted possible scenarios (because every ship may get to its
port of destination or may be lost during its voyage). Instead of
discussing each of these scenarios in detail (2^100 is more than
1.2 x 10^30), picture this scenario with a histogram showing the
loss distribution of ships (see Figure 1.1). To build this histo-
gram, one can evaluate a relatively small number (eg, 10,000) of
randomly chosen scenarios of the 2^100 potential scenarios,
which will already offer a reasonable overview. On the x-axis,
one can show the number of ships lost, and on the y-axis the
number of scenarios (among 10,000) in which exactly this
number of ships were lost.
❏❏ The risk measure: In essence, one can observe many scenarios
with a loss of around 10 ships, which is due to the assumption of
01 Chapter RMV.indd 18 04/05/2016 19:52

19
a 10% chance of losing a specific ship during the next voyage. In
this situation, Antonio may consider the expected value of losing
10 ships as his risk statistic.
Another possibility would be to look at the 95% quantile of the loss
distribution – that is, the number of ships lost that will not be
exceeded with a probability of 95% in the above setting. A quick
spreadsheet computation reveals that this quantile amounts to
about 15 ships – that is, in 95% of all scenarios “only” up to 15 ships
(and therefore 15% of goods, since Antonio distributed the freight
evenly) will be lost. Note that, in the previous example where
Antonio relied on a single ship, the 95% quantile amounts to exactly
100%. A third possibility to sum up the information contained in
the loss distribution would be to look at the average loss above the
95% quantile, and to compute the average loss over all scenarios
where more than 15 ships are lost. This quantity is sometimes
called the 95% expected shortfall. In our example, the 95% expected
shortfall amounts to around 16 ships (and therefore 16% of goods).
Finally, it is important to note that in this historical example, the
analysis has been working through the basic modelling procedure
proposed. The only issue not commented on is the question of how
to arrive at the value of 10% for the chance of losing a specific ship
Figure 1.1 Loss distribution of ships
1400
1200
1000
800
600
400
200
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
No. of ships lost
No.ofscenarios
01 Chapter RMV.indd 19 04/05/2016 19:52

20
(in other words, how to arrive at the scenario weights). One might
base such an assumption on historical loss data (own loss experi-
ence, loss experience of other merchants), on risk factor mapping
(the weather, the condition of the ships, the level of experience of
the ships’ crews, pirate activities), on expert judgement (own gut
instinct) or even on market prices (cost of maritime insurance).
However, will there be a “best” alternative? If yes, how can it be
determined? And even assuming that the “best” alternative has
been found, how “good” will it be (How many historical losses
have been observed? Are weather forecasts reliable? Are reports on
pirate activities subject to (actual) survivorship bias? Do insurers
take into account similar reasoning when computing premia?)?
These questions are concerned with using statistics in risk models,
a topic that will be taken up in the following chapter and returned
to repeatedly in this book.
The above examples should have demonstrated the application
of the three elements of QRMs:
❏❏ the quantity of interest;
❏❏ the set of potential future scenarios; and
❏❏ the relevant statistic.
On the other hand, the examples showed also that even in this very
basic setting one sometimes has to make strong assumptions (eg,
the assumption of independence of the fates of different ships) to
keep things quantifiable. In addition, and even more importantly,
the second example shows that there is no unique risk measure
(expected value, quantile or expected shortfall) to guide
decision-making.
01 Chapter RMV.indd 20 04/05/2016 19:52

Buy your copy today with 25% off
using code: RMV25. Visit
www.riskbooks.com and search
Validation.

Risk Model Validation teaser

More Related Content

Similar to Risk Model Validation teaser (20)

Recently uploaded (20)

Risk Model Validation teaser