Improving risk analysis

125L.A. Cox, Jr., Improving Risk Analysis, International Series in Operations
Research & Management Science 185, DOI 10.1007/978-1-4614-6058-9_4,
© Louis Anthony Cox, Jr. 2012
Chapters 1 and 2 emphasized technical methods−causal analysis and robust
decision-making, respectively – that are especially useful for individual decision-
makers. Chapter 3 explored challenges and opportunities for improving decision-
making by treating communities, rather than individuals, as the natural units for
decision-making. This chapter, by contrast, considers an intermediate level of
decision-making entity: the organization, including business enterprises. Although
it is a fascinating challenge to understand how businesses (and other organizations)
interact with each other and the public within societies, communities, and institu-
tional frameworks, adapting to each other and to their uncertain environments over
time (Harford 2011), this chapter has a narrower, applied focus: understanding and
improving how organizations describe and respond to the risks and threats that they
perceive. It has become common practice for many organizations to explicitly iden-
tify, list, and make management priority decisions about different risks that they are
aware of facing. These can be as diverse as risks of supply chain disruption, loss of
reputation, failure of business continuity, legal liabilities, strikes, plant closures, and
market and financial risks. This chapter critically examines how well such explicitly
identified risks can be managed by the scoring, rating, and ranking systems now
widely used in practice; and whether it is possible to make simple changes to
improve the performance of these risk management systems.
Background: Traditional Versus Novel Risks and Risk
Management Principles
Forover500years,needsandopportunitiestomanageriskybusinessesmoreprofitably
have driven a succession of risk management innovations in corporate organization,
law, and governance. Limited liability, joint stock ownership, insurance, reinsurance,
Chapter 4
Improving Organizational Risk Management

126 4 Improving Organizational Risk Management
stock exchanges, financial options and derivatives, securitization and bundling of
collateralized debt obligations, and networks of cooperative and reciprocal risk-
underwriting agreements are among the developments in business and financial risk
management that helped to shape and make possible the modern world. From the
Age of Discovery through the scientific and industrial revolutions and into modern
times, ability to coordinate the activities of speculative investors to fund risky ven-
tures and business enterprises, in return for shares in resulting gains or losses, has
enabled large-scale profitable risk-taking (Bernstein 1998). Large-scale risk-taking,
in turn, has helped to power risky but enormously beneficial explorations, discover-
ies, innovations, and developments in a variety of industries.
Risk-taking in modern businesses and finance exploits a key principle: risk shar-
ing among investors allows mutually beneficial acceptance of larger-scale risks
than any investor alone would accept. In financial risk analysis, a risky prospect is
an investment opportunity that offers different sizes of potential gains or losses,
with different corresponding probabilities. A risky prospect that each investor in a
group would be unwilling to accept, because its potential losses are too large to
justify its potential gains (for a given degree of individual risk aversion), might
nonetheless be acceptable to all of them if they take shares in it.
Example: Sharing Large-Scale Risks Can Make Them Acceptable
to Risk-Averse Investors
A risk-averse decision-maker who would refuse to accept a 50–50 chance of gaining $2,000 or
losing $1,000 might nonetheless want to accept a 50–50 chance of gaining $20 or losing $10. If so,
then 100 such individuals could all benefit by taking equal shares in the large risk that returns
either $2,000 or -$1,000. This illustrates one of the basic principles that enabled investors in early
joint stock companies to fund risky exploration, exploitation, and colonization ventures: shares in
a risky prospect may be acceptable, even if the prospect as a whole would not be. (The economic
theory of syndicates (Wilson 1968) extends this insight by showing that a group of investors with
exponential individual utility functions and different degrees of risk aversion, ui
(x) = 1 - exp(-x/ci
)
for individual i, should act like one individual with utility function u(x) = 1 - exp[-x/(c1
+ c2
+ …
+ cn
)] in deciding what risky investments to accept. Each individual member maximizes expected
utility by taking a share ci
/(c1
+ c2
+ …+ cn
) in each accepted investment and perhaps participating
in side bets with other individuals. The inequality 1/(c1
+ c2
+ …+ cn
) < 1/ci
implies that the group
as a whole should be less risk-averse than its members.) Such arrangements for sharing risks,
together with diversification of investments across multiple independent prospects, creation and
management of investment portfolios of prospects (possibly with correlated returns), and hedging
of bets over time (by exploiting negatively correlated assets to reduce the variance in returns), have
become staples of financial risk management.
However, there is a widespread perception that additional principles are needed
for enterprise risk management (ERM) in today’s world, as novel risks are created
by increasingly interlinked and interdependent enterprises, new financial instru-
ments for packaging and distributing risky prospects, changing social and moral
mores and standards for acceptable (and legal) risk-taking behavior, and new

127Background: Traditional Versus Novel Risks and Risk Management Principles
risk-taking incentives created by modern compensation, liability, corporate gover-
nance, and institutional structures. The resulting nontraditional risks can threaten
the stability and viability of even the largest organizations and institutions. Initiating
events, from unanticipated droughts in remote locations, to factory fires, to loss of
reputation and public confidence in a previously trusted organization or institution,
can send repercussions spreading through networks of tightly coupled supply chains,
contractual obligations, and contingent claims, sometimes causing unexpectedly
large and systemic cascades of losses or failures in enterprises far from the original
source.
Unrecognized correlations or interdependencies can also create hidden systemic
risks in networks of tightly coupled enterprises, making them vulnerable to swiftly
cascading failures. As discussed in Chap. 3, and as emphasized in the literature on
black swan risks, the resulting heavy-tailed loss distributions, in which unprece-
dentedly large losses occur too often to be ignored, do not satisfy traditional statisti-
cal risk modeling assumptions. Such risks make previous experience an inadequate
basis for assessing, planning for, or underwriting future risks of loss. Instead, it
becomes necessary to try to anticipate and prepare for risks which, by their very
nature, are unlikely to have been seen before. Even within a single enterprise,
incomplete and private information, costly communications and transactions costs,
and organizational incentives too often undermine effective cooperation and risk
management. Many commentators on enterprise risk management (ERM) have
concluded that traditional risk management principles need to be augmented with
new ideas for managing such nontraditional risks.
New business and financial risks arise largely from uncertainty about the trust-
worthiness of partners and of agreed-to plans and commitments. Can supply chain
partners be relied on to fulfill their contractual agreements, or are they subject to
unexpected interruptions due to strikes, factory fires, unanticipated shortages, or
other causes? Can fellow employees in other divisions of a company, or within a
single division, be trusted to deliver what they have committed to, or are they likely
to be overwhelmed by unforeseen changes in market demand or competition or
regulation? Will poorly aligned incentives cause business partners or fellow employ-
ees to take less care than we might want or expect? Uncertainties about whether
agreements and internal operational procedures and systems can be trusted, together
with high transaction costs for creating, monitoring, and enforcing formal contracts,
increase the costs of starting and operating profitable businesses.
Questions about trust and trustworthiness also arise in many economic transac-
tions, for example, between employers and employees, producers and consumers,
insurers and insured, as well as among business partners. Similar questions affect
domestic political risks at multiple levels (e.g., how far can union members trust
union bosses, or voters trust those they have voted for?) and international relations
(e.g., how far can countries trust each other to abide by agreements on disarma-
ments, or free trade, or environmental emissions, or fair work practices?) A few
examples follow, to emphasize and illustrate the types of political, economic, and
organizational risks that spring from limited or uncertain trustworthiness of other
individual agents.

Example: Individual Versus Social Rationality in Games of Trust
Game theory illuminates many challenges for creating and maintaining high-trust relations in
organizations. Principles of individual rationality often conflict strongly with requirements for col-
lective rationality, especially when the incentives of a game undermine trustworthy behavior.
Perhaps most famously, temptations to free ride, or succumb to tragedies of the commons, can lead
players to make individually rational choices which leave them all worse off than would different
choices. In Prisoner’s Dilemma (often used as a model for international arms races or local free
riding) and similar games, playing always defect is a dominant strategy for every player, even
though it leads to Pareto-dominated outcomes.
Prisoner’s Dilemma
Thus, the social rationality principle “Don’t choose Pareto-dominated outcomes” conflicts with
the individual rationality principle “Don’t choose dominated strategies.” The Centipede Game and
Chain Store Paradox (discussed in most modern game theory texts and expositions, e.g., Gintis
(2000) and Rosenthal (2011)) show that social rationality also conflicts with other foundations of
individual rationality, such as backward induction (used in decision tree analysis and dynamic
programming) and dynamic consistency (or its multi-person extension, subgame perfection),
respectively. In each of these games, if players could trust each other to cooperate despite the
incentives to defect, all would all end up better off (with higher individual payoffs) than when each
applies principles of individual rationality to the incentives provided by these games (i.e., choosing
dominant strategies in Prisoner’s Dilemma, using backward induction in the Centipede Game, and
selecting the subgame perfect equilibrium in the Chain Store Paradox) (Gintis 2000). In reality,
both laboratory experiments (such as the ultimatum, trust, and dictator games) and real-world
evidence (e.g., from labor markets, participation in voting, paying honest taxes, and so forth), as
well as neuroeconomic studies of oxytocin levels and reward pathways in the brain when deciding
whether to trust and to cooperate, all show that people are predisposed to cooperate more than
game theory would predict (Rosenthal 2011; Gintis et al. 2003). Yet, with repeated play, the incen-
tives of these games start to prevail, and defection, rather than cooperation, increases unless some
form of retaliatory punishment is allowed (Gintis 2000).
Example: Incentives and Trust in Principal-Agent Relations
In organizations, employees must repeatedly decide how trustworthy to be (e.g., how hard to
work each day to achieve their employer’s goals, if level of effort is private information and not
easily monitored) and also how much to trust each other, for example, in creating shared plans
whose success requires multiple divisions to keep commitments. Economists and management
scientists have studied how to design compensation rules and other organizational incentives to
avoid providing constant temptations to free ride, cheat, lie, or otherwise defect, so that the
benefits of mutual cooperation can be more fully achieved. In simple principal-agent models, a
single agent chooses a level of effort and produces an outcome for the principal. The outcome
depends on the agent’s level of effort, and also on chance, so that higher levels of effort are
associated with more valuable outcomes, but do not guarantee them. The agent receives
Player 2 cooperates Player 2 defects
Player 1 cooperates 2, 2 0, 3
Player 1 defects 3, 0 1, 1

129Background: Traditional Versus Novel Risks and Risk Management Principles
compensation from the principal, typically according to a compensation rule or contract to
which both agree in advance. The principal can observe the outcome, but not the agent’s effort,
and hence, the agent’s compensation can depend only on the outcome, but not on his level of
effort. Analysis of such models shows that private information (here, the agent’s level of effort),
coupled with the assumption of purely rational play, leads to Pareto-inefficient levels of effort
and probability distributions for outcomes. That is, under any contract that can be designed
when only the outcome but not the agent’s effort is common knowledge (called a second-best
contract), the agent typically provides less effort and receives less compensation than if his level
of effort could be freely observed by the principal. Both the principal and the agent have lower
expected utility than could be achieved by a first-best contract based on common knowledge of
effort as well as outcome (Gintis 2000; Rosenthal 2011). Both parties could gain, if only the
principal could trust the agent to put in a first-best level of effort, and compensate him accord-
ingly. But it would be strategically irrational for them to cooperate this way, in the sense that the
principal trusting the agent and the agent being trustworthy do not constitute a Nash equilibrium
pair of mutual best (expected utility maximizing) responses to each other’s choices. However,
when multiple agents repeatedly compete to serve one or more principals, the rewards to favor-
able reputation, together with improved opportunities for the principal to gauge each agent’s
effort by comparing results across agents and over time, can induce more trustworthy, and hence
more valuable and better-rewarded, agent performance.
Example: Incentives, Trust, and Risk in Market Transactions
Similar principles hold for insurance contracts and for consumer product quality and liability, as
well as for employment contracts (Rosenthal 2011; Gintis 2000). In each case, Pareto efficiency of
enforceable agreements or contracts is reduced by the existence of private information (or asym-
metric information) that creates incentives for one or both parties to defect, compared to what they
would do if the private information could be credibly and freely shared. Both parties could gain if
each could trust the other to provide a first-best level of effort or due care (i.e., the level that would
be achieved if private information were common knowledge), but such trust would not be strategi-
cally rational.
In insurance markets, two well-known incentive effects reduce the ability of insurer and insured
to agree on mutually beneficial contracts, if the insured’s true risk level and care level are private
information that cannot be freely observed or verified by the insurer. Adverse selection occurs
when only people with above-average risks (who expect to benefit from having policies) are
willing to pay the premiums for insurance coverage. This self-selection makes the insurance con-
tract less attractive and more expensive for the insurer. If insurer solvency or regulatory constraints
require higher premiums to cover the expected higher payouts, then rates may increase, so that only
even riskier subsets of buyers are willing to pay the high premiums. In extreme cases, this cycle of
escalating costs and increasing self-selection of the riskiest individuals continues until the market
collapses, and no insurance is offered, even though many people would have been willing to buy
insurance at rates that would have benefitted both themselves and the insurer. Moral hazard arises
because those who are insured have less incentive to take care than if they were not insured. Again,
both parties could gain if the insurer could trust the insured to take more care despite having insur-
ance. Likewise, in product markets, both manufacturers and consumers might gain if the consumers
could trust the manufacturers to deliver high-quality products at market prices and if manufacturers
could trust consumers to exercise care in the use of products.
Enterprise risk management (ERM) and related practices help organizations to
think about and manage nontraditional risks. In addition to financial risks, these

include legal, reputational, and brand image risks. They include the many risks aris-
ing from complex interdependences and networks of obligations and commitments,
and from uncertainty about the willingness or ability of employees, partners, and
customers to deliver on commitments and to maintain trustworthy behaviors in the
face of temptations to defect. Successful ERM reduces the costs of uncertainty and
its adverse impacts on organizational performance. ERM typically focuses on iden-
tifying, documenting, sharing, tracking, and managing risks that could disrupt a
business or jeopardize its commitments and operations. At least in principle, mak-
ing such risk information explicit and available for scrutiny – often with the help of
periodic audits and reports – can reduce the adverse incentive effects of private
information about risks. Maintaining trust in business (and other relations) may be
less difﬁcult when risk information is tracked and disclosed. In practice, however,
those assessing the risks may not have a very precise understanding of how to assess
or express them. Efforts to assess and share risk information and risk management
plans responsibly may degenerate into compliance exercises in which boxes are
checked off and vague descriptions or summaries are produced, with little real
insight into the extent of remaining risks or what to do about them. The following
sections provide examples. A worthwhile challenge for risk analysts is therefore to
develop and apply more useful technical methods for enterprise risk analysis, bear-
ing in mind the substantial business and economic advantages of improving risk
assessment, communication, and management so that the adverse incentives created
when such information remains private can be overcome.
Top-Down ERM Risk Scoring, Rating, and Ranking
A popular current approach to ERM involves employees from the boardroom level
down in trying to think through what might go wrong, how frequent or likely these
failures are, how severe their consequences are likely to be, and what should be done
about them, if anything, both now and later. Such ERM exercises and processes
emphasize anticipation and prevention. They have the virtue of bringing together and
sharinginformationamongemployeesfromdifferentpartsofacompany(andsometimes
among partners in a supply network), perhaps helping to align organizational under-
standing of different risks and of plans to deal with them. Sharing information on risks,
uncertainties, and measures to manage their effects can help participants more fully
achieve the potential gains from well-coordinated cooperation (both inside and outside
an organization). The results of ERM processes typically include priority lists, risk
matrices, and similar devices to focus management attention and to inform deliberation
and decisions about what risks to accept and what risk management interventions to
allocate attention and resources to ﬁrst.
Despite their advantages, such popular approaches to risk management in orga-
nizations can inadvertently increase the very risks that they seek to manage; and
they too often recommend risk management interventions that could easily be

131Limitations of Risk Scoring and Ranking Systems
improved upon (Hubbard 2009). The remainder of this chapter explains why. It also
considers how to modify existing ERM systems to improve their performance.
The key issues are not restricted to ERM but apply to all uses of risk ranking, scor-
ing, and comparison systems to inform risk management deliberations and resource
allocations, whether in a corporation, a regulatory agency, the military, or the
Department of Homeland Security. The potential returns are enormous for improv-
ing risk management practices that are based on these methods.
Limitations of Risk Scoring and Ranking Systems
Many organizations practice risk management by regularly scoring, rating, or ranking
different hazards (sources of risk) or risk-reducing opportunities to identify the top-
ranked opportunities to be addressed in the current budget cycle. Use of priority
scoring and rating systems is becoming ever more widespread as they are incorpo-
rated into commercial software offerings designed to support compliance with national
and international standards (such as the ISO 31000 risk management standard), regu-
lations, and laws (such as Section 404 of the Sarbanes–Oxley Act of 2002, in the
United States). It is therefore useful to understand, and where possible overcome,
some intrinsic limitations in the performance of all possible priority-setting rules and
scoring systems, evaluated as guides to rational action (Hubbard 2009). Although
many of these limitations are already well recognized among specialists in decision
analysis and financial risk analysis, they are of great practical importance to users
seeking to understand what can and cannot be achieved using current risk-scoring
methods or seeking to develop improved approaches to risk management. In general,
risk-scoring methods are not appropriate for correlated risks. Indeed, as we will dem-
onstrate, they are not necessarily better than (or even as good as) purely random selec-
tion of which risk management activities to fund.
More constructively, when risk-reducing opportunities have correlated conse-
quences, due to uncertainties about common elements (such as carcinogenic or
toxic potencies of chemicals used in manufacturing, effectiveness of counterterror-
ism or cybersecurity countermeasures used in IT systems, and stability of currency
or solvency of banks and insurers used in financing), then methods for optimizing
selection of a portfolio (subset) of risk-reducing opportunities can often achieve
significantly greater risk reductions for resources spent than can priority-scoring
rules. In general, the best choice of a subset of risk-reducing activities cannot be
expressed by priority scores. Instead, optimization techniques that consider interde-
pendencies among the consequences of different risk-reducing activities are essen-
tial. Fortunately, such methods are easy to develop and implement. They can
substantially improve the risk-reduction return on investments in risk-reducing
activities.

The Need for Improvement: Some Motivating Examples
Examples of important applications of priority-scoring systems in diverse areas of
applied risk analysis include the following.
Example: Scoring Information Technology Vulnerabilities
The Common Vulnerability Scoring System (CVSS) for rating information technology (IT) system
vulnerabilities uses scoring formulas such as the following to help organizations set priorities for
investing in security risk reductions:
BaseScore=(.6*Impact+.4*Exploitability-1.5)*f(Impact)
Impact=10.41*(1-(1-ConfImpact)(1-IntegImpact)*(1-AvailImpact))
Exploitability=20*AccessComplexity*Authentication*AccessVector
f(Impact)=0 if Impact=0; 1.176 otherwise
AccessComplexity=case AccessComplexity of
High: 0.35
Medium: 0.61
Low: 0.71
Authentication = case Authentication of
Requires no authentication: 0.704
Requires single instance of authentication: 0.56
Requires multiple instances of authentication: 0.45
AccessVector = case AccessVector of
Requires local access: .395
Local Network accessible: .646
Network accessible: 1
(Source: http://nvd.nist.gov/cvsseq2.htm)
Such a rule base, no matter how complex, can be viewed as an algorithm that maps categorized
judgments and descriptions (such as that access complexity is high and that local access is required)
into corresponding numbers on a standard scale. Higher numbers indicate greater vulnerability and
need for remedial action. Proponents envision that “As a part of the U.S. government’s SCAP
(Security Content Automation Protocol) CVSS v2 will be used in standardizing and automating
vulnerability management for many millions of computers, eventually rising to hundreds of mil-
lions” (http://www.first.org/cvss/).
Example: Scoring Consumer Credit Risks
The practice of rank-ordering consumers based on credit scores is ubiquitous in business today. A
recent description states that “FICO® risk scores rank-order consumers according to the likelihood
that their credit obligations will be paid as expected. The recognized industry standard in consumer
credit risk assessment, FICO® risk scores play a pivotal role in billions of business decisions each
year. …[They] are widely regarded as essential building blocks for devising successful, precisely
targeted marketing, origination and customer management strategies by credit grantors, insurance
providers and telecommunications companies.” Examples include BEACON® at Equifax US and
Canada; FICO® Risk Score, Classic at TransUnion US; and Experian/Fair Isaac Risk Model at
Experian. (Source: www.fairisaac.com/fic/en/product-service/product-index/fico-score/)

133The Need for Improvement: Some Motivating Examples
Example: Scoring Superfund Sites to Determine
Funding Priorities
The State of Connecticut (www.ct.gov/dep/lib/dep/regulations/22a/22a-133f-1.pdf) published a
Superfund Priority Score method, to be used in determining funding priorities for remediation of
Superfund sites. Users must score each of many factors (reflecting exposure potential; groundwa-
ter impact; surface water impact; toxicity, persistence, mobility, and quantity of hazardous sub-
stances; impact to the environment, including Species of Special Concern; and potential air release
and fire hazards) using ordered categories. Each category carries a certain number of points.
For example, an area that contains a rare species gets a score of 4 on this factor. If it has a declining
or infrequent species, the score is 3; for a habitat-limited species, the score is 2. If this factor (species
of concern) is not applicable, the score for this factor is zero. The scores for all factors are summed.
The resulting total score determines the priority for funding of remedial action at sites on the SPL
[the State of Connecticut Superfund Priority List].
Example: Priority Scoring of Bioterrorism Agents
MacIntyre et al. (2006) proposed a risk priority-scoring system for bioterrorism agents. They
described their approach as follows:
“Disease impact criteria were as follows: infectivity of the agent (person-to-person trans-
mission potential), case fatality rate, stability in the environment and ease of decontamina-
tion, incidence of disease per 100,000 exposed persons in the worst-case release scenario,
and reports of genetic modification of the agent for increased virulence.
Probability of attack criteria was [sic] designated as: global availability and ease of pro-•
curement of the agent, ease of weaponization, and historical examples of use of the agent
for an attack.
Prevention/intervention criteria were categorized as: lack of preventability of the disease•
(such as by vaccination) and lack of treatability of the disease (such as by antibiotics).
For each of the scoring categories, a score of 0–2 was assigned for each category A agent as•
follows: 0=no, 1=some/low, and 2=yes/high. The sum of these scores (of a total possible
score of 20) was used to rank priority.”
This is similar to the Superfund scoring system, in that categorical ratings for various factors
are assigned numerical scores, and the sum of the scores is used to set priorities. In neither case did
the authors verify whether additive independence conditions hold, which are required in multiat-
tribute value and utility theory to justify additive representations of preferences (Keeney and Raiffa
1976). For example, an agent with a score of 2 for lack of preventability of disease and 0 for lack
of treatability would have the same sum for these two factors (2+0=2) as an agent with lack of
preventability of disease = 0 and lack of treatability = 2 or as an agent with lack of preventability
of disease = 1 and lack of treatability = 1. Yet, risk managers who can completely prevent a disease
(lack of preventability of disease = 0) might not care as much about whether it is treatable as they
would if the disease could not be prevented. Likewise, in Superfund site scoring, many decision-
makers might care less about the presence of a declining species near a site that creates no expo-
sure than near a site that creates a large, toxic exposure. Such interactions among factor scores are
ignored in purely additive scoring systems.

Example: Larger Input Uncertainties May Create Smaller
Output Uncertainties
Occasionally, users of risk-scoring systems are asked to rate or rank their uncertainties about different
inputs, with the idea being that larger uncertainties in inputs drive greater uncertainty about outputs,
and therefore might benefit most from further information. It may be worth noting that the assumption
that greater uncertainty in an input does not produce smaller uncertainty in the output of a model is not
necessarily mathematically valid. Consider a model Y = f(X), where X is an uncertain input and Y is
the model’s output. For concreteness, suppose that X is a scalar input, uniformly distributed over some
interval, and that f is a known, deterministic function. Now, is it true that the uncertainty about Y cor-
responding to an uncertain value of X should necessarily be a non-decreasing function of the level of
uncertainty in X? The following example suggests not. Presumably, most analysts (and all who use
variance or entropy to define and measure the uncertainty of a probability distribution) would agree
that X has smaller uncertainty if it is uniformly distributed between 98 and 100 than if it is uniformly
distributed between 0 and 198. Yet, if f is the threshold function f(X) = 1 for 99 £ X £ 100, else f(x) = 0,
then the uncertainty (e.g., variance or entropy) of Y = f(X) is greatest when X is uniformly distributed
between 98 and 100 (since there are then equal probabilities of 50% each that Y will be 0 or 1) and is
much smaller when X is uniformly distributed between 0 and 198 (since there is then a 99.5% probabil-
ity that Z=0). So, larger uncertainty about X induces smaller uncertainty about the value of output Y
caused by X. Thus, uncertainty about the output should not necessarily be assumed to be an increasing
function of input uncertainty.
Example: Threat–Vulnerability–Consequence (TVC)
Risk Scores and Risk Matrices
Many organizations use numerical priority-scoring formulas such as Risk=Threat×Vulnerability
×Consequence or Risk=Threat×Vulnerability×Criticality or Risk=Threat×Vulnerability×
Impact. The Department of Homeland Security, the Department of Defense, and the armed ser-
vices all use this approach to prioritize anti-terrorism risk-reduction efforts (Jones and Edmonds
2008; Mitchell and Decker 2004; www.ncjrs.gov/pdffiles1/bja/210680.pdf.) The formula Risk=
Threat×Vulnerability×Consequence also provides the conceptual and mathematical basis for the
RAMCAP™ (Risk Analysis and Management for Critical Asset Protection) standard and related
compliance training and software (www.ramcapplus.com/). Law enforcement officers have been
trained to use Risk=Threat×Vulnerability×Impact scoring systems to set priorities for managing
security risks at major special events (www.cops.usdoj.gov/files/ric/CDROMs/PlanningSecurity/
modules/3/module%203%20ppt.ppt). Unfortunately, when the components on the right-hand side
(e.g., Threat, Vulnerability, and Consequence) are correlated random variables – for example,
because attackers are more likely to attack facilities with high Vulnerability and Consequence or
because larger storage facilities have higher Vulnerability and Consequence than small ones – then
the product of their means differs from the mean of their product, and it is not clear what either one
has to do with risk. Correct expressions require additional terms to adjust for non-zero covariances
(Cox 2008b). Similar comments apply to widely used risk matrices based on formulas such as
Risk=Frequency×Severity, with the right-hand side variables assessed using ordered categories
(such as high, medium, and low) and Risk ratings or priorities then being determined from these
component ratings. In general, such risk matrices order some pairs of risks incorrectly and, in some
cases, can perform even worse than setting priorities randomly (Cox 2008a).

135Setting Priorities for Known Risk-Reducing Investment Opportunities
Setting Priorities for Known Risk-Reducing Investment
Opportunities
To enable formal analysis of the properties of priority-scoring systems in a reason-
ably general framework, we define a priority-setting process as consisting of the
following elements:
1. A set of items to be ranked or scored. The items may be hazards, threats, custom-
ers, interventions, assets, frequency–severity pairs, threat–vulnerability–
consequence triples, threat–vulnerability–consequence–remediation cost
quadruples, Superfund sites, construction projects, or other objects. We will refer
to them generically as items, hazards, prospects, or opportunities.
2. An ordered set of priority scores that are used to compare hazards. These may be
ordered categorical grades, such as high, medium, and low; nonnegative integers
indicating relative priority or ranking; or nonnegative real numbers, representing
values of a quantitative priority index such as Risk=Threat×Vulnerability×Con
sequence or priority index=expected benefit of remediation/expected cost of
remediation, where the italicized variables are nonnegative numbers.
3. A priority-scoring rule. A scoring rule is a mathematical function (or a procedure
or algorithm implementing it) that assigns to each hazard a unique corresponding
priority score. (This implies that any two hazards having identical attribute val-
ues, or identical joint distributions of attribute values, must have the same prior-
ity score.)
The priority-scoring rule determines a priority order in which hazards are to be
addressed (possibly with some ties). Addressing a hazard is assumed to reduce risk
and hence to be valuable to the decision-maker: it increases expected utility. For
example, it may stochastically reduce the flow of illnesses, injuries, or fatalities
resulting from a hazardous process, activity, or environment.
Although items might have multiple attributes, and value trade-offs might make
preferences among them difficult to define clearly in practice, we shall assume that
the decision-maker has perfectly clear, consistent preferences for the consequences
of addressing different hazards. For example, suppose that addressing hazard j
reduces loss, measured on a scale such as dollars (for financial risks) or quality-
adjusted life years (QALYs) (Doctor et al. 2004), for health risks, by an amount, xj
,
defined as the difference between the loss if hazard j is left unaddressed and the loss
if hazard j is addressed. Suppose that all value units (e.g., dollars or QALYs) are
considered equally intrinsically valuable, with twice as many being worth twice as
much to the decision-maker. More generally, we assume that addressing hazards
creates gains on a measurable value scale satisfying standard axioms (Dyer and
Sarin 1979) that allow preferences for changes in or differences between situations,
from before a hazard is addressed to after it is addressed, to be coherently ranked
and compared. Let xj
be the measurable value from addressing hazard j. We assume
that the value of addressing a hazard, expressed on such a measurable value scale,
depends only on its attributes, and we work directly with the measurable values,

rather than the underlying attributes. (The value scale need not be measured in
QALYs, but thinking of such a concrete example may aid intuition.) If it costs the
same amount to address any hazard, and if the resulting increases in value are known
with certainty, then, for any budget, total beneﬁts are maximized by addressing the
hazards in order of their decreasing values, xj
. This provides one useful model for
priority-based risk management decision-making.
Priorities for Independent, Normally Distributed
Risk Reductions
Next, suppose that the value achieved by addressing hazard j is uncertain. This
might happen, for example, if the quantities or potencies of hazardous chemicals
stored at different waste sites are uncertain, or if the sizes of exposed populations
and their susceptibilities to exposure are not known, or if the effectiveness of inter-
ventions in reducing risks is in doubt. To model priority-based risk management
decisions with uncertainty about the sizes of risk reduction opportunities, we
assume that their values are random variables and that the decision-maker is risk-
averse. For a risk-averse decision-maker with a smooth (twice-differentiable)
increasing von Neumann–Morgenstern utility function for the value attribute, the
conditions in Table 4.1 are all mutually equivalent, and all imply that the utility
Table 4.1 Equivalent characterizations of exponential utility functions
Let X and Y be any two risky prospects (random variables) measured on the intrinsic value
scale. They represent the uncertain values (e.g., QALYs saved) by addressing two different
hazards
• Strong Risk Independence: Adding the same constant to both X and Y leaves their prefer-
ence ordering unchanged. Thus, if X + w is preferred to X + w for some value of the constant
w, then X is preferred to Y for all values of w
• Risk Premium Independence: The decision-maker’s risk premium (amount she is willing to
pay to replace a prospect with its expected value) for any risky prospect depends only on
the prospect (Thus, it is independent of background levels of the value attribute.)
• Certainty Equivalent Independence: If a constant, w, is added to every possible outcome of
a prospect X, then the certainty equivalent of the new prospect thus formed is CE(X) + w,
where CE(X) denotes the certainty equivalent (or selling price on the intrinsic value scale)
of prospect X. (This is sometimes called the delta property, due to Pfanzagl, 1959.) Thus,
for any constant, w, CE(w + X) = CE(X) + w
• Equal Buying and Selling Prices: For any prospect X and any constant w, the decision-
maker is indifferent between w + CE(X) – X and w + X – CE(X)
• No Buying Price/Selling Price Reversals: The ranking of prospects based on their certainty
equivalents (i.e., selling prices, e.g., how many QALYs would have to be saved with
certainty to offset the loss from abandoning the opportunity to save X QALYs) never
disagrees with their ranking based on buying prices (e.g., how many QALYs a decision-
maker would give up with certainty to save X QALYs). (This assumes the decision-maker is
risk-averse; otherwise, the linear risk-neutral utility function u(x) = x would also work)
• Exponential Utility: u(x) = 1 – e–kx
Dyer and Jia (1998), Hazen and Sounderpandian (1999)

137Priority Ratings Yield Poor Risk Management Strategies for Correlated Risks
function is exponential. If one or more of these conditions is considered norma-
tively compelling, then an exponential utility function should be used to choose
among prospects with uncertain values.
The expected value of an exponential utility function for any random variable
corresponds to its moment-generating function. For example, let Xj
represent the
uncertain measurable value of addressing hazard j, modeled as a random variable on
the value axis. Let CE(Xj
) denote the certainty equivalent of Xj
, that is, the value
(such as QALYs saved) received with certainty that would have the same expected
utility as (or be indifferent to) random variable Xj
. Then if Xj
is normally distributed
with mean E(Xj
) and variance Var(Xj
), it follows (from inspection of the moment-
generating function for normal distributions) that its certainty equivalent is:
CE(Xj
) = E(Xj
) – (k/2)Var(Xj
),
where k is the coefﬁcient of risk aversion in the exponential utility function (Infanger
2006, p. 208).
A set of equally costly risk-reducing measures with independent, normally dis-
tributed values can be prioritized in order of decreasing CE(Xj
) values. For any
budget, total expected utility is maximized by funding risk-reduction opportunities
in order of decreasing priority until no more can be purchased. Moreover, even if
the risk-reducing measures do not have identical costs, an optimal (expected utility
maximizing, given the budget) policy maximizes the sum of certainty equivalents,
subject to the budget constraint. (This follows from the additivity of means and of
variancesforindependentrisks.Findinganoptimalsubsetinthiscaseisawell-studied
combinatorial optimization problem, the knapsack problem.) Thus, for any two fea-
sible portfolios of risk-reducing measures, the one with the greater sum of certainty
equivalents is preferred. Certainty equivalents therefore serve as satisfactory priority
indices for identifying optimal risk-reducing investments in this case.
Priority Ratings Yield Poor Risk Management Strategies
for Correlated Risks
Priority-based risk management successfully maximizes the risk-reduction value
(expected utility or certainty equivalent value of risk-reducing activities) of defen-
sive investments in the special cases discussed in the preceding two sections.
However, it fails to do so more generally. Selecting a best portfolio of hazards to
address (or of risk-reducing measures to implement) cannot in general be accom-
plished by priority-setting if uncertainties about the sizes of risks (or of risk-
reduction opportunities) are correlated. Unfortunately, this is the case in many
applications of practical interest. No priority rule can recommend the best portfolio
(subset) of risk-reducing opportunities when the optimal strategy requires diversify-
ing risk-reducing investments across two or more types of opportunities, or when it
requires coordinating correlated risk reductions from opportunities of different
types (having different priority scores).

Example: Priority Rules Overlook Opportunities
for Risk-Free Gains
A priority-setting rule that rates each uncertain hazard based in its own attributes only, as all the
real priority-scoring systems previously mentioned do, will in general be unable to recommend an
optimal subset of correlated risk-reducing opportunities. For example, any risk-averse decision-
maker prefers a single random draw from a normal distribution with mean 1 and variance 1,
denoted N(1, 1), to a single draw from normal distribution N(1, 2), having mean 1 but variance 2.
Therefore, a scoring rule would assign a higher priority to draws from N(1, 1) than to draws from
N(1, 2). But suppose that X and Y are two N(1, 2) random variables that are perfectly negatively
correlated, with Y = 2 – X. (This might happen, for example, if effects depend only on the sum of
X and Y, which has a known value of 2, but the relative contributions of X and Y to their sum are
uncertain.) Then, drawing once from X and once from Y (each of which is N(1, 2)) would yield a
sure gain of 2. Any risk-averse decision-maker prefers this sure gain to two draws from N(1, 1).
Unfortunately, any priority rule that ignores correlations among opportunities would miss this pos-
sibility of constructing a risk-free gain by putting X and Y in the same portfolio, as it would always
assign draws from N(1, 1) higher priority than draws from N(1, 2).
This example shows that priority-setting rules can recommend dominated portfolios, such as
allocating all resources to risk reductions drawn from N(1, 1) instead of pairing negatively corre-
lated N(1, 2) risk reductions, because they cannot describe optimal portfolios that depend on cor-
relations among risk-reducing opportunities, rather than on the attributes of the individual
opportunities. The next example shows that priority rules can, in principle, not only recommend a
dominated decision but in some cases can even recommend the worst possible decision.
Example: Priority-Setting Can Recommend the Worst Possible
Resource Allocation
Setting: Suppose that an environmental risk manager must decide how to allocate scarce resources
to remediate a large number of potentially hazardous sites. There are two main types of sites.
Hazards at type A sites arise primarily from relatively long, thin chrysotile asbestos fibers. Hazards
at type B sites arise from somewhat shorter and thicker amphibole asbestos fibers. The risk manager
is uncertain about their relative potencies, but knows that removing mixtures of approximately equal
parts of the chrysotile and amphibole fibers significantly reduces risks of lung cancer and mesothe-
lioma in surrounding populations. She believes that the following two hypotheses are plausible, but
is uncertain about their respective probabilities. (This is intended for purposes of a simple illustra-
tion only, not as a realistic risk model.)
H1: Relative risk from a type A site is 0; relative risk from a type B site is 2 (compared to the•
risk from a hypothetical site with equal mixtures of chrysotile and amphibole fibers, which we
define as 1). This hypothesis implies that all risk is from amphibole fibers.
H2: Relative risk from a type A site is 2; relative risk from a type B site is 0. This hypothesis•
implies that all risk is from the chrysotile fibers.
For purposes of illustration only, we assume that only these two hypotheses are considered
plausible, although clearly others (especially that the two types of fiber are equally potent) would
be considered in reality.
Problem: If the risk manager can afford to clean N=10 sites, then how should she allocate them
between type A and type B sites? Assume that she is risk-averse and that more than 10 sites of each
type are available.

139Priority Ratings Yield Poor Risk Management Strategies for Correlated Risks
Solution: If the risk manager cleans x type A sites and (N−x) type B sites, then the total expected
utility from cleaned sites is pu(N – x) + (1 – p)u(x). Here, p denotes the probability that hypothesis
H1 is correct, 1−p is the probability that H2 is correct, N=10 is the total number of sites that can
be cleaned, and u(x) is the utility of cleaning x sites with relative risk of 2 per site cleaned. For any
risk-averse (concave) utility function u(x), and for any value of p between 0 and 1, Jensen’s inequal-
ity implies that expected utility is maximized for some x strictly between 0 and N. For example, if
u(x) = x0.5
and p=0.5, then x=5 maximizes expected utility. The worst possible decision (minimiz-
ing expected utility) is to allocate all resources to only one type of site (either type A or type B).
Yet, this is precisely what a priority system that assigns one type a higher priority than the other
must recommend. Hence, in this case, any possible priority order (either giving type A sites prece-
dence over type B sites or vice versa, perhaps depending on whether p<0.5) will recommend a
subset of sites that has lower expected utility than even a randomly selected subset of sites. The
best subset (e.g., 5 type A sites and 5 type B sites, if p=0.5) can easily be constructed by optimiza-
tion if p is known. But even if both p and u(x) are unknown, it is clear that a priority order is the
worst possible decision rule.
Example: Priority-Setting Ignores Opportunities
for Coordinated Defenses
Setting: Suppose that an information security risk manager can purchase either of two types of security
upgrades for each of 100 web servers. Type A prevents undetected unauthorized access to a web server,
and type B prevents unauthorized execution of arbitrary code with the privileges of the web server, even
if the web server is accessed. (For examples of real-world historical vulnerabilities in an Apache web
server, see http://www.ﬁrst.org/cvss/cvss-guide.html#i1.2.) For simplicity, suppose that installing a type
A upgrade reduces the annual incidence of successful attacks via web servers from 0.03 to 0.02 per
web-server-year and that installing a type B upgrade reduces it from 0.03 to 0.025. Installing both
reduces the average annual rate of successful attacks via these machines from 0.03 to 0.
Problem: If the security risk manager can afford 100 security upgrades (of either type), what
investment strategy for reducing average annual frequency of successful attacks would be recom-
mended based on (a) priority ranking of options A and B and (b) minimization of remaining risk?
(Assume that the frequency of attempted attacks remains constant, because hackers only discover
the defenses of a web server when they attempt to compromise it.)
Solution: (a) A vulnerability-scoring system could assign top priority to installing a type A upgrade
on each of the 100 web servers, because a type A upgrade achieves a larger reduction in the
vulnerability score of each server than a type B upgrade. Following this recommendation would
leave a residual risk of 0.02*100=2 expected successful attack per year. (b) By contrast, a risk-
minimizing budget allocation installs both A and B upgrades on each of 50 machines, leaving 50
machines unprotected. The residual risk is then 0.03*50=1.5 expected successful attack per year,
less than that from giving A priority over B.
Comment: In this example, a scoring system that considers interactions among vulnerability-reduc-
ing activities could give install A and B a higher priority for each server than either install A or
install B. But most deployed scoring systems do not encourage considering interactions among
vulnerabilities or among vulnerability-reducing countermeasures. In many applications, doing so
could lead to combinatorial explosion. (For example, the guidance for Common Vulnerability
Scoring System 2.0 offers this advice: “SCORING TIP #1: Vulnerability scoring should not take
into account any interaction with other vulnerabilities. That is, each vulnerability should be scored
independently” http://www.ﬁrst.org/cvss/cvss-guide.html#i1.2.).

Example: Priority Rules Ignore Aversion to Large-Scale
Uncertainties
Setting: A bioterrorism risk manager must choose which of two defensive programs to implement
this year: (A) a prevention program (e.g., vaccination) that, if it works, will reduce the risk of fatal
infection from 10% to 0% for each affected person in the event of a bioterrorism attack with a
certain agent; or (B) a treatment program (e.g., stockpiling an antibiotic) that will reduce the risk
of mortality from 10% to 5% for each affected individual in the event of such an attack. For sim-
plicity, suppose that program A will prevent either N expected deaths (if it works) or none (if it
does not) following an attack and that its success probability is p. Program B prevents 0.5N
expected deaths with certainty, leaving 0.5N remaining expected deaths in the event of an attack.
Problem: (a) For a risk-averse decision-maker with utility function u(x) = 1 – e–kx
, where x is the
number of expected deaths prevented, which risk reduction measure, A or B, is preferable?
(Express the answer as a function of p, k, and N.) (b) How does this compare to the results of a
priority ranking system, for p=0.8 and k=1?
Solution: (a) The expected utility of risk reduction is pu(N) = p(1 – e–kN
) for program A and
u(0.5N) = 1 – e–0.5kN
for program B. Program A is preferable to program B if and only if p(1 – e–kN
)
> 1 – e–0.5kN
, or, equivalently, p > (1 – e–0.5kN
)/(1 – e–kN
). For example, if kN = 1, then p must be at
least 62.2% to make A preferable to B. If kN = 10, then p must be at least 99.3% to make A prefer-
able to B. (b) If the probability that program A will work is p=0.8 and the coefficient of absolute
risk aversion is k=1, then A is preferred to B for N=1 or 2, and B is preferred to A for N³3. In this
case, diversification is not an issue (i.e., either A or B is definitely preferable, depending on the
value of N.) However, no priority ranking of interventions A and B is best for both N=2 and N=3.
The reason is that a risk-averse decision-maker who prefers A to B for small N prefers B to A for
larger N. Any priority-scoring system that ranks one of A or B above the other, and that is not
sensitive to N, will recommend the less valuable decision for some values of N. In practice, most
scoring systems use qualitative or ordered categorical descriptions that are not sensitive to quanti-
tative details such as N. (For example, the Common Vulnerability Scoring System rates “Collateral
Damage Potential,” which scores “potential for loss of life, physical assets, productivity or reve-
nue,” as high if “A successful exploit of this vulnerability may result in catastrophic physical or
property damage and loss. Or, there may be a catastrophic loss of revenue or productivity.” http://
www.first.org/cvss/cvss-guide.html#i1.2 Such a qualitative description does not discriminate
between N=2 and N=3.)
Discussion: Precisely analogous examples hold for consumer credit risk-reducing interventions,
information security, homeland security, and other applications in which the success of some pro-
posed interventions is uncertain. Suppose that intervention A reduces the average rate of successful
attacks per target (e.g., secure facility or web server) per year from 10% to 0% if it works, while
intervention B reduces the rate from 10% to 5% with certainty. The probability that A will work
(i.e., that an attacker cannot circumvent it) is p. If the choice between A and B affects N similar
targets, then, by analogy to the above example, a risk-averse risk manager should prefer A to B for
sufficiently small N and B to A for larger values of N. Any priority system that is applied to a small
number of targets at a time (possibly only 1, by the target’s owner, operator, or security manager)
will then consistently recommend A, even though B should be preferred when the complete set of
N targets is considered. That scoring systems are blind to the total number of similar targets that
they are applied to (i.e., to the scale of application) can lead to excessively high-risk exposures
arising from large-scale application of priorities that hold for small numbers of targets, but that
should be reversed for larger numbers of targets.

141Opportunities for Improvement
Opportunities for Improvement
Applied risk analysis is in a curious state today. Highly effective optimization
methods for selecting subsets of risk-reducing investments to maximize the value of
risk reductions achieved for a given budget are readily available. They can draw on a
rich and deep set of technical methods developed in financial risk analysis and opera-
tions research over the past half century. Yet, these methods are having little or no
impact on management of some of the world’s most critical risks. Instead, extremely
simplistic priority-setting rules and scoring systems are being widely used to set
priorities and to allocate resources in important practical risk management applica-
tions. Scoring systems are being used in important real-world applications as diverse
as Superfund site cleanups, computer and IT security vulnerability assessment, coun-
terterrorism, military asset protection, and risk matrix systems (used in everything
from designing and defending federal buildings and facilities, to managing construc-
tion project and infrastructure risks, to regulating risks of financial and business
enterprises). Yet, these risk-scoring systems achieve less value-of-risk-reduction than
could easily be obtained if resources were allocated by other methods (including
randomized decision-making, in extreme cases.)
The requirements that scoring systems must meet before being adopted and rec-
ommended in standards are not very stringent. In the applications examined in earlier
sections, there appears to be no requirement that risk-scoring systems should pro-
duce effective risk management decisions (or even that they should not produce the
lowest-value decision possible) before they are standardized for widespread use. In
all of the applications mentioned, common elements found in multiple risky systems
create correlated vulnerabilities, criticalities, consequences, or threats. Priority lists
do not generally produce effective risk management decisions in such settings.
Applyinginvestmentportfoliooptimizationprinciples(suchasoptimaldiversification,
consideration of risk aversion, and exploitation of correlations among risk reductions
from different activities) can create better portfolios of risk-reducing activities in
these situations than any that can be expressed by priority scores.
In summary, risk priority-scoring systems, although widely used (and even
required in many current regulations and standards), ignore essential information
about correlations among risks. This information typically consists of noting com-
mon elements across multiple targets (e.g., common vulnerabilities). These common
features induce common, or strongly positively correlated, uncertainties about the
effectiveness of different risk-reducing measures. It is easy to use this information,
in conjunction with well-known decision analysis and optimization techniques, to
develop more valuable risk-reduction strategies, for any given risk management
budget, than can be expressed by a priority list. Thus, there appears to be abundant
opportunity to improve the productivity of current risk-reducing efforts in many
important applications using already well-understood optimization methods.
This observation will not be new or surprising to experts in decision and risk
analysis (Hubbard 2009). Techniques for optimizing investments in risk-reducing
(and/or benefit-producing) interventions have been extensively developed in opera-
tions research and management science for decades. What is perhaps startling is that

these methods are so little exploited in current risk assessment and risk management
systems. Risk priority scores can never do better (and often do much worse) than
optimization methods in identifying valuable risk-reducing strategies. Perhaps it is
time to stop using risk priority scores to manage correlated risks, recognizing that
they often produce simple but wrong answers. Optimization techniques that con-
sider dependencies among risk-reducing interventions for multiple targets should be
used instead. The following sections consider how to apply this advice in a simple
but important case where many different such interventions are available, but bud-
get constraints make it impossible to pursue all of them simultaneously.
Risk Management Software Based on Risk Indices
Despite the limitations and deficiencies of priority-setting rules and scoring systems
for managing risks (Hubbard 2009), they are widely used in ERM and other areas
of applied risk analysis. This is not only because of their simplicity and intuitive
appeal, but also because they are already embedded in risk management software
initiatives and tools used around the world to help companies follow international
risk management standards and recommendations, such as ISO 31000. For better or
worse, risk priority-scoring systems are being used to support organizational risk
management tasks ranging from ERM at Walmart (Atkinson 2003) to terrorism risk
assessment programs (Mitchell and Decker 2004). This magnifies the benefits from
any simple changes that can improve their practical value.
As previously mentioned, many deployed risk management software tools use
the following simple conceptual framework. Users estimate the values or qualitative
ratings of a few (typically, two or three) components of risk, such as probability and
impact in ERM applications; threat, vulnerability, and consequence in terrorism
applications; or exposure, probability, and consequences in occupational health and
safety risk management applications. They enter these inputs for each event or con-
dition of concern that they want to prioritize for purposes of risk management. The
software combines these inputs using simple (typically, multiplicative) formulas or
look-up tables, to produce corresponding risk numbers or ratings for each event or
condition of concern. We will refer to the resulting risk numbers (or scores or rat-
ings), in the rest of this chapter, as risk indices, since they are typically interpreted
as indicating the relative sizes, importances, or priorities of different risks that an
organization faces.
Most risk management software products display risk index outputs as risk matri-
ces (tables), with frequency and severity categories for rows and columns; or as
colorful heat maps, with cell colors indicating priorities for action or remediation of
the estimated risks. Other popular displays include bar charts comparing risk indices
and scatter plots (e.g., showing impact versus probability) showing their compo-
nents. These methods are widely employed in diverse organizations and ERM
products.

143Simulation–Evaluation of Methods for Selecting Risks to Address
Example: Simple Risk Formulas in Commercial Risk
Management Systems
Vendors now offer many risk index systems used by large organizations. For example, the
STARSYS® System (www.starys.com/html/products.html) is offered as “an Integrated Risk
Management and Document Control system developed specifically to enable organisations to
implement sound practices that comply with Occupational Health and Safety and Environmental
and Quality control requirements.” It uses three risk components, called consequences, exposure,
and probability, and provides a Risk Calculator for assigning numbers (e.g., between 0 and 6) to
each of these components. From these user-supplied ratings, it then calculates a corresponding risk
priority class.
Similarly, the SAP BusinessObjects Risk Management 3.0 software documentation (http://scn.
sap.com/docs/DOC-8488) states that “Impact levels (and if use[d] Benefit Levels) are an important
building block of any risk management model. All risks are described in terms of Likelihood and
Impact. Impact levels are used to give a real-world description to the magnitude of a risk event.
Benefit Levels give a real-world description to the magnitude of a benefit.” The documentation
also explains that “Impact Levels combined with Probability Levels are used to create a Risk Heat
Map.” More explicitly, documentation of the “Risk and Opportunity Level Matrix” explains that
“The combination of impact level×probability level should correspond to the defined risk level.”
Example: A More Sophisticated Commercial Risk
Management System
The GuardianERM system (www.guardianerm.com/RiskManagement.htm) notes that “Users
evaluate and categorise each risk, record the possible causes, rate the likelihood and consequences,
record Value at Risk and assign a financial statement assertion if required. Users attach any number
of controls to a risk and evaluate each control as to its effectiveness, record cost of control, update
control status (agreed, proposed, implemented), control type (treat, transfer, correct), key control
indicator, execution frequency, action and control responsibility.” Although the system displays
conventional-looking heat maps and bar charts as outputs to summarize and interpret the data it
records, the information that it collects, specifically on control costs and effectiveness, can poten-
tially be used to improve upon conventional risk indices. This possibility is explored below.
In light of the theoretical limitations of risk indices described in previous sections, it is important
to understand How well do real-world risk management recommendations or priorities based on the
conceptual framework of risk indices perform? If an organization uses risk indices, risk matrices, or
risk heat maps to set priorities and allocate resources, then how much better or worse off will it be
than if it used different approaches? To better understand the objective performance characteristics
of these widely deployed, but not yet well-understood systems, the following sections compare the
relative performances of several different risk indices to each other, and to an optimal approach,
using simple models with easily derived correct answers.
Simulation–Evaluation of Methods for Selecting Risks
to Address
To clearly compare different risk management approaches, this section constructs a
simple example with detailed data, for which it can be determined how resources should
be allocated. This makes it possible to quantify how well two different risk indices

perform, compared to this ideal answer. Finally, a large, randomly generated data set
will be used to further analyze the performances of these alternative approaches.
Consider a risk manager or decision-maker constrained by a limited budget to
allocate among a large number of opportunities to reduce risks. She wishes to use
risk management software, based on the risk index framework, to decide which
ones to address with this limited budget. Table 4.2 shows an example with five risks
(or opportunities for risk reduction), each represented by one row of the table.
Each risk is characterized by three attributes, here called Threat, Vulnerability,
and Consequence, shown in the left columns. Their product gives the index called
Risk (4th column). Many risk management software products stop at this point,
color-code or rank or categorize the resulting risk index values, and display the
results, with the top-ranked risks (here, the top two) displayed in a color such as red
and assigned top priority for risk management interventions.
One criticism of this method recognizes that the true values of the inputs (such
as Threat, Vulnerability, and Consequence in Table 4.2) are typically uncertain and
their uncertain values may be correlated. Considering the correlations can com-
pletely change the values for the risk index and can even reverse their relative sizes
(Cox 2008a). Risk management software tools that omit correlation information
from the inputs – as most do – produce risk rankings (and implied or explicit recom-
mendations) that might be changed or reversed if correlations were accounted for.
To avoid this difficulty, for purposes of understanding performance driven by
other factors, the input columns in Table 4.2 are populated by independent random
variables (i.e., all correlations among variables are assumed to be 0). Specifically,
each input value in Table 4.1 is independently randomly sampled from a unit uniform
distribution, U[0, 1]. This case of statistically independent input values may
artificially improve the performance of risk indices, compared to real performance,
if real performance is deteriorated by the presence of negative correlations between
input values. It has previously been found that negatively correlated input values
can cause risk indices to systematically assign higher estimated values (or levels,
ratings, etc.) of risk to smaller risks than to larger ones, making the index approach
worse than useless (i.e., worse than random selection) as a guide to effective risk
management (Cox 2008a; Hubbard 2009). However, to understand the relative
Table 4.2 Example of resource allocation problem data
1
Threat
2
Vulnerability
3
Consequence
4=3*2*1 5
6=5*4
Risk
reduction
7
Cost ($)
8=6/7
Risk (e.g.,
average
loss per
year)
Fraction
of risk
eliminated
if addressed
Risk
reduction
per unit
cost
0.64 0.44 0.22 0.063 0.55 0.034 0.83 0.04
0.28 0.92 0.90 0.231 0.42 0.097 0.40 0.25
0.07 0.73 0.15 0.008 0.80 0.006 0.35 0.02
0.44 0.75 0.04 0.014 0.82 0.012 0.37 0.03
0.70 0.01 0.34 0.003 0.76 0.003 0.16 0.02

145Simulation–Evaluation of Methods for Selecting Risks to Address
performance and limitations of different indices, even under favorable conditions,
we will make the assumption that the inputs are statistically independent.
A second criticism of index methods based on combining inputs (e.g., Threat×
Vulnerability×Consequence, Frequency×Severity, and Probability×Impact) with-
out considering costs or budgets or risk reductions achieved by alternative interven-
tions is that they leave out information that is crucial for rational risk management
decision-making. Knowing which risks are largest does not necessarily reveal which
risk management interventions will achieve the greatest risk reduction for a given
amount spent and thus may prove deceptive as screening and prioritization tools.
(Some risk index software products do consider costs and risk reductions for differ-
ent potential interventions and are not subject to this criticism.)
To evaluate the significance of this criticism for tools that omit cost consider-
ations when prioritizing risks, Table 4.2 includes four additional columns that deal
with costs and risk reductions. Fraction of Risk eliminated if addressed gives the
fraction of the Risk number in the fourth column that could be removed by spending
available budget on the most cost-effective available risk-reducing measure for the
risk in that row. Risk reduction is the product of the two columns to its left, Risk and
Fraction of Risk eliminated if addressed. Risk reduction shows the risk-reduction
benefit (measured in units such as average prevented loss per year) that would be
achieved if the risk in that row were selected to be addressed. This is another pos-
sible index that could be used to set priorities for risk management, corresponding
to changing the decision rule from “Address the largest risks first” to “Address the
largest opportunities for risk reduction first.”
The Cost column shows the assumed cost to address each risk, which would
reduce it by the factor shown in the Fraction of Risk eliminated if addressed col-
umn. The last column, Risk reduction per unit cost, shows the ratio of the Risk
reduction to Cost columns, indicating the amount of risk reduction achieved per
dollar spent if selected (i.e., if there are several alternatives for reducing a risk, we
assume that the one with the greatest value of this ratio is selected). To evaluate the
performance limitations of risk index methods under assumptions favorable for
their use, we assume that each risk (i.e., row) can be addressed independently, so
that the risk manager’s only problem is to decide which risks (i.e., which rows) to
address. Such additive independence could be realistic if the risk manager is trying
to decide how to allocate risk-reduction resources among separate, non-interacting,
geographic areas or facilities, based on attributes such as those in Table 4.2. Given
the choice of a feasible subset of rows (meaning any subset with total costs sum-
ming to no more than the available budget), the total risk-reduction benefit achieved
is assumed to be the sum of the benefits achieved (i.e., the Risk reduction numbers)
from the selected rows.
The last column, Risk reduction per unit cost (column 8), provides a possible
alternative index to the Risk and Risk reduction indices in columns 4 and 6 for set-
ting priorities and selecting a subset of risks to address. (Note that, in general, costs
and risks may be measured in different units. Costs might be measured in units such
as dollars spent or person-years of expert time allocated to problem remediation.
Benefits might be measured as lives saved or loss of critical facilities or infrastruc-
ture prevented. No effort has been made to monetize these impacts or to place them

on a common scale. Although Table 4.2 shows values less than 1 for the Risk
reduction per unit cost column, due to the simple arithmetic that Risk reduction
comes from a product of several U[0, 1] variables and cost comes from a single
U[0, 1] variable, this does not imply that the benefits of risk reductions are not worth
the costs.)
In Table 4.2, with only five risks (rows), one can easily identify the subset of
interventions that should be addressed to maximize the risk reduction achieved for
any given budget spent. For example, if the budget is less than 0.35 (on a scale nor-
malized so that 1 represents the maximum possible cost for any intervention), then
the only affordable intervention would be to select the bottommost row, which has
a cost of 0.16 and yields a risk-reduction benefit of 0.003 (on a scale normalized so
that the mean risk-reduction benefit is the mean of the product of four independent
U[0, 1] random variables, i.e., (0.5)^4 = 0.0625). If the budget is 0.37, then a larger
benefit, of 0.012 can be obtained. For budgets greater than 0.51, multiple risks can
be addressed. As the budget increases further, one must search for the feasible
(i.e., affordable) subset of risks that maximizes the risk reduction achieved. This
combinatorial optimization problem can be solved approximately or exactly using
operations research algorithms (Senju and Toyoda 1968; Martello and Toth 1990).
Either specialized knapsack algorithms (Senju and Toyoda 1968) or general-purpose
branch-and-bound algorithms (such as those implemented in the Excel Solver
add-in) can solve such problems in minutes, if the number of risks is at most a few
dozen. For larger-scale problems (e.g., with thousands or tens of thousands of risks),
special-purpose heuristics provide nearly optimal solutions within seconds (Martello
and Toth 1990); thus, there is no practical reason to use significantly less-than-
optimal approaches. This optimization identifies the maximum risk-reduction
benefit that can be achieved for each level of budget.
In summary, we consider the following increasingly demanding indices:
• Risk: This is column 4 (i.e., Risk=Threat×Vulnerability×Consequence). It is
the most basic index that we consider. Using this index to set priorities for
addressing risks corresponds to the decision rule, “Address the largest risks
first.”
• Risk reduction: This (column 6) is the product Risk Reduction=Risk×Fraction
of Risk eliminated if addressed. Using it to set priorities for addressing risks cor-
responds to the decision rule, “Address the largest risk reductions first.”
• Risk reduction/cost ratio (column 8) takes the preceding index (Risk reduction)
and divides it by the cost needed to achieve it. The corresponding decision rule
is “Address the largest risk reductions per unit cost first.”
Each of these indices is derived by refining its predecessor with additional
information – from risk, to risk reduction, to risk reduction per unit cost. We will
compare the performance of these indices to each other and also to the optimal solu-
tion (obtained by solving a knapsack problem) on a simple test set of randomly
generated budget allocation problems. Our goal is to answer the following research
questions in a simple simulation setting for which one can obtain answers easily:

147Results: Comparing Index Policies to Optimal Portfolios
1. How do the risk-reduction benefits achieved by using the Risk index in Table 4.2
to select risks to address compare to the risk-reduction benefits achieved by using
the other two indices? Is the Risk index (the product of the three inputs called
Threat, Vulnerability, and Consequence in Table 4.2) a useful surrogate for the
more indices that include bang for the buck (i.e., risk reduction and cost) informa-
tion? Or, is the Risk index significantly less useful than these more refined ratios
in setting priorities that achieve large risk-reduction benefits for dollars spent?
2. How do the benefits achieved by using these different indices to set priorities
compare to the benefits from optimal selection of which risks to address?
In short, for this simple setting, we can investigate the value of using a more
demanding index instead of a simpler one and explore how much additional benefit
(if any) could be achieved by using optimization, instead of either index, to decide
which risks to address for a given budget. Comparing these alternatives on simple
random data suggests the potential sizes of gains in risk-reduction benefits from
collecting and using more information or more sophisticated algorithms to try to
improve upon the risk management priorities suggested by the simpler Risk index.
We carry out the comparisons using a table analogous to Table 4.2 but with 100
risks instead of 5.
Results: Comparing Index Policies to Optimal Portfolios
Figure 4.1 shows the amounts of risk reduction (y-axis) that can be purchased for
different costs, if each of the three different indices – Risk, Risk reduction, or Risk
reduction per unit cost – is used to set priorities and allocate resources in the test set
of randomly generated problems. Table 4.3 shows numerical comparisons of the risk
reductions achieved by each index, for several different budget levels. The rightmost
column of Table 4.3 shows the maximum possible risk reduction that can be achieved
for each budget level (as determined by solving the combinatorial optimization
problem (knapsack problem) of selecting a subset of risks to address that will maxi-
mize the total risk reduction obtained for the specified budget. With 100 randomly
generated risks from which to choose, the solution times are on the order of about 10
min on a modern PC, using the Excel Solver’s branch-and-bound algorithm for
binary integer programs. Since no specific units have been selected for costs and
benefits, Table 4.4 presents the information from Table 4.3 normalized to make the
maximum risk reduction possible equal to 1 (from addressing all risks) and similarly
normalized to make the smallest cost needed to achieve this equal to 1.
The results exhibit the following conspicuous patterns:
• All three indices are useful. Compared to a completely uninformed (random)
approach to priority-setting for resource allocation (for which the corresponding
cumulative risk reduction versus cumulative cost curve in Fig. 4.1 appears as the
straight line shown from the origin to the leftmost point where all projects are
funded), all three curves in Fig. 4.1 show a useful degree of lift (i.e., improve-
ment, visually seen as the difference between each curve and the straight line).

Thus, in this test set of problems, even an index that does not consider cost is
valuable compared to uninformed selection (i.e., the lowest curve in Fig. 4.1
compared to the straight line).
• In this test set of randomly generated problems, the Risk reduction per unit cost
index outperforms the other two indices. The Risk index performs less well than
Table 4.3 Risk reductions achieved by using different indices to allocate budgets
Budget
Risk reduction
using Risk index
to allocate budget
Risk reduction
using Risk
reduction index
Risk reduction
using Risk reduction
per unit cost index
Optimal risk
reduction for
given budget
0.5 0 0 0.19 0.52
1 0.65 0.65 0.83 0.94
2 0.91 1.05 1.48 1.61
4 1.66 2.01 2.56 2.64
8 3.25 3.35 3.86 3.88
16 4.6 4.94 5.07 5.09
32 5.73 5.84 5.86 5.86
Inﬁnite 5.95 5.95 5.95 5.95
Cumulative Risk Reduction vs. Cumulative Cost for Three Indices
0 10 20 30 40 50 60
Cumulative cost
0
1
2
3
4
5
6
7CumulativeRiskReduction
Risk
Risk Reduction
Risk Reduction/Cost
Fig. 4.1 Comparison of risk reductions achieved using three different indices

149Results: Comparing Index Policies to Optimal Portfolios
the other indices. For example, for the same cost, the priority order generated by
the Risk index reduces risk by only 15% of the maximum possible amount, com-
pared to 25% for the Risk reduction per unit cost index. Thus, at this budget
level, the Risk index is only about 60% as efficient as the Risk reduction per unit
cost index in obtaining risk reductions for cost spent. Similarly, the Risk index
reduces risk by only 28% of the maximum possible amount, for the same cost at
which the Risk reduction per unit cost index reduces risk by 43%. This gap
between the lowest-performing (Risk) and highest-performing (Risk reduction
per unit cost) indices diminishes at budget levels high enough so that most or all
risk-reduction opportunities are taken.
• The best index (Risk reduction per unit cost) provides nearly optimal decisions
for almost all budget levels. Although this index can fail to recommend the best
subset of risks to address when the budget is too small to address more than a
very few risks (e.g., one or two), it yields decisions that are optimal or nearly so
(i.e., within about 2% of optimal, in terms of risk reduction obtained for resources
spent for this simple simulation), for all budget levels greater than about 0.02
(on a scale where 1 denotes the smallest budget needed to address all risks.)
• Diminishing returns. The risk reductions achieved by different budgets show
steeply diminishing returns, for each index. For example, more than half of the
maximum possible risk reduction can be achieved (via any of the indices) for
less than 1/6 of the budget needed to eliminate all risk; and more than 80% of the
total risk can be removed (unless the simplest index, Risk, is used) for about 1/3
of the budget needed to remove all risk. Conversely, the best index (with cost
considerations) achieves significantly higher lift than by the worst index (with no
cost considerations) only in situations where budget restrictions make careful
allocation of resources essential for achieving close-to-maximum risk-reduction
benefits, as shown in Table 4.4.
These findings for the simple test set considered indicate that for resource-con-
strained organizations faced by a large number of opportunities to invest in costly
risk reductions, using simple risk indices, such as Risk=Threat×Vulnerability×Co
nsequence or Risk=Frequency×Severity, to allocate risk management resources,
Table 4.4 Normalized risk reductions achieved by using different indices
Budget
Risk reduction
using Risk index
to allocate budget
Risk reduction
using Risk
reduction index
Risk reduction using
Risk reduction per
unit cost index
Optimal risk
reduction for
given budget
0.01 0 0 0.03 0.09
0.02 0.11 0.11 0.14 0.16
0.04 0.15 0.18 0.25 0.27
0.08 0.28 0.34 0.43 0.44
0.17 0.55 0.56 0.65 0.65
0.33 0.77 0.83 0.85 0.86
0.67 0.96 0.98 0.98 0.98
1 1 1 1 1

may be relatively inefficient. For some budget levels, these simple indices (and, a
fortiori, risk matrices or risk heat maps based on them) yield no more than about
60–65% of the risk-reduction benefits achieved by using indices that consider risk
reduction per unit cost, at least in this simple test set of randomly generated prob-
lems. Thus, organizations may gain substantial improvements (e.g., more than a
third, in this simple setting) in risk reductions achieved for dollars spent, by using
better indices.
However, investing in more sophisticated optimization algorithms produces little
further gain (except at the lowest budget levels) beyond what can be achieved by
moving from Risk to Risk reduction per unit cost. That is, the best index yields
nearly optimal decisions for these problems, leaving very little room for further
improvement by using more sophisticated (non-index) decision rules.
Discussion and Conclusions
In a simple, idealized setting, with statistically independent values for the compo-
nents of risk, multiplicative formulas for combining them into risk indices, additively
independent costs and benefits (i.e., risk reductions) across risks, and known values
for all costs, risks, and risk reductions, each of the three indices examined has some
value. The best of them, the Risk reduction per unit cost ratio, provides nearly opti-
mal resource allocations for almost all budget levels considered in the simple simula-
tion exercise reported here (Table 4.4). The other two indices, Risk and Risk reduction,
are significantly correlated with Risk reduction per unit cost and with each other, so
it is not surprising that they provide some information useful for setting priorities and
allocating resources. Specifically, Risk reduction is proportional to Risk (with a ran-
dom coefficient of proportionality, corresponding to the U[0, 1] random variable
Fraction of Risk eliminated if addressed), and Risk reduction per unit cost is derived
from Risk reduction by multiplying it by a random variable, 1/Cost, where Cost is an
independent U[0, 1] random variable. Conversely, Risk may be viewed as being
derived from the high-performing index Risk reduction per unit cost by multiplying
it by the random variable Cost and dividing the result by the random variable Fraction
of Risk eliminated if addressed. These transformations distort the information in Risk
reduction per unit cost, making Risk less useful than Risk reduction per unit cost; the
result is that Risk may achieve only a fraction (e.g., 60%) of the risk-reduction
benefits of Risk reduction per unit cost, for the same cost.
If similar results hold in practice – an if which depends on the empirical joint
distributions of risk sizes, risk-reduction opportunities, and costs to reduce risks –
then they provide both good news and bad news for providers and customers of
current risk management software systems. The bad news is that risk management
software packages that implement simple indices, such as Risk=Probability×Impact
or Risk=Threat×Vulnerability×Consequence, are probably supporting relatively
inefficient risk management priorities and resource allocations, unless cost
information is added after the risk indices have been computed and displayed.

151Discussion and Conclusions
The heat maps that they typically provide suggest that high-ranked (e.g., red or
high) risks should be prioritized ahead of low-ranked (e.g., green or low) risks for
risk management attention and remediation. Unfortunately, following these recom-
mendations may achieve only a fraction (e.g., 60%, depending on the number and
costs of risk-reduction opportunities and the budget available to address them) of
the risk-reduction benefits that could be achieved by more effective indices.
The good news is that data already being collected in some systems as part of
risk management documentation can be used to substantially improve upon the
above indices, at least in the simple random test bed demonstrated here. The
improvement method is simple: as illustrated in Table 4.4, multiplying each value of
a Risk index by a (Risk reduction fraction per unit Cost) factor to obtain a Risk
reduction per unit cost index can lead to revised priorities that capture almost 100%
of the maximum possible risk reduction. (As already discussed, this gain is possible
for almost any given budget level, as long as it allows for funding a sizable portfolio
of risk-reduction opportunities.) Even if this new factor can only be estimated
imprecisely, the potential gains from using it to refine current Risk indices may be
substantial enough to warrant adding it as a post-processing step to current methods
that stop with Risk indices.
Figure 4.1 makes clear that the simulation test bed conditions are favorable, com-
pared to the case of zero or negative lift, which previous work has established can
arise when index procedures are applied to situations with negatively correlated
input values (e.g., low frequencies of high-consequence events, high frequencies of
low-consequence events) (Cox 2008a). Such situations are common in practice,
including ERM applications domains.
Some other important complexities that might arise in practice include:
• Allow risk-averse or risk-seeking utility functions. Rather than simple expected
value (e.g., Probability×Impact) formulas for risk, exponential or other utility
functions would allow greater flexibility in expressing risk attitudes.
• Consider uncertain ability to reduce risk by taking expensive actions. Rather
than spending a known cost to achieve a known risk reduction, it may be neces-
sary to make some investments that return only uncertain reductions in risk.
• Model interactions among risk-reducing investment opportunities. For example,
some risk-reducing investments (e.g., upgrading an alarm system) may only be
possible when others (e.g., installing an alarm system) have already been suc-
cessfully completed; or some investments may only be valuable if others that
attempt to protect the same assets in different ways fail.
• Generalize to arbitrary joint distributions of costs and risk reductions, rather
than statistically independent uniform distributions, as in this chapter.
• Consider randomly deteriorating or changing situations, where a risk may ran-
domly increase (e.g., as more supports for a bridge fail) during the time that no
risk management interventions (e.g., inspection and replacement of failing sup-
ports) are funded.
Although no general results are yet available for situations involving all these
complexities, some important advances have been made recently on each of these

dimensions by showing that index policies are optimal in broad classes of models
(e.g., random forest models) that allow for precedence relations and other con-
straints among activities, arbitrary costs of activities and probability distributions
for rewards (e.g., risk reductions), and exponential utility functions that allow for
risk aversion (Denardo et al. 2004).
In addition, the theory of Gittins indices in operations research (Denardo et al.
2004; Sethuraman and Tsitsiklis 2007; Glazebrook and Minty 2009) has recently been
shown to provide excellent heuristics for allocating resources in large classes of risky
restless bandit problems that greatly generalize the resource allocation task consid-
ered here, by letting risk-reduction opportunities (or other projects) evolve randomly
while not being worked on and by allowing uncertainty about the true value of each
project. Many such indices are generalizations of the bang for the buck ratio (i.e., the
risk reduction per unit cost) index considered in this chapter. These results suggest that
using relatively easily computed indices to set priorities for resource allocation can
provide nearly optimal risk management decisions in many interesting settings
beyond the idealized setting considered here. However, even in these more general
cases, high-performing indices are usually generalizations of the benefit-per-unit-cost
criterion that has proved to be so effective in our simple context.
Many risk analysts already recognize that including costs in risk ranking efforts
can significantly improve budget allocations, with high-level committees making
this point over 2 decades ago in the context of risk ranking activities performed by
the US Environmental Protection Agency (EPA SAB 1990; Davies 1996). In this
context, the results reported here will seem hardly surprising to some readers.
However, as a practical matter, many computer-aided risk analysis software products,
formulas (e.g., Risk=Threat×Vulnerability×Consequences), and consulting tools
(e.g., risk matrices) do not yet include bang for the buck information or show
estimates of risk reduction achieved per dollar spent as an option. Thus, the many
organizational risk management initiatives and software products that now use sim-
ple risk indices with the aim of ranking (i.e., suggesting priorities and supporting
risk management resource allocation decisions) might be significantly improved
simply by multiplying current risk indices by the estimated ratio of the risk-reduction
fraction to the cost of a risk-reducing intervention. This would make a useful start
toward improving their performance in increasing the risk-reduction benefits
achieved for resources spent.
This chapter has only provided quantitative results for the special case of inde-
pendent, uniformly distributed, random inputs, illustrated in a simple test bed of
randomly generated budget allocation problems. At least in this idealized setting,
the results suggest that a better choice of risk index can lead to significantly more
effective resource allocation decisions for constrained risk management budgets.
Generalizing to more complex, realistic, and interesting settings, such as those for
which Gittins indices provide useful decision rules, represents a potentially valuable
next step for understanding how far simple changes in the indices used to rank and
compare risk-reducing investments can improve the current generation of risk
management software and practices.

153References
References
Atkinson W (2003) Enterprise risk management at Walmart. Risk Manag. http://www.rmmag.
com/Magazine/PrintTemplate.cfm?AID=2209
Bernstein PL (1998) Against the Gods: the remarkable story of risk. Wiley, New York
Cox LA Jr (2008a) What’s wrong with risk matrices? Risk Anal 28(2):497–512
Cox LA Jr (2008b) Some limitations of “Risk=Threat×Vulnerability×Consequence” for risk
analysis of terrorist attacks. Risk Anal 28(6):1749–1762
Davies JC (1996) Comparing environmental risks: tools for setting government priorities.
Resources for the Future, Washington, DC
Denardo EV, Rothblum UG, van der Heyden L (2004) Index policies for stochastic search in a
forest with an application to R&D project management. Math Oper Res 29(1):162–181
Doctor JN, Bleichrodt H, Miyamoto J, Temkin NR, Dikmen S (2004) A new and more robust test
of QALYs. J Health Econ 23(2):353–367
Dyer JS, Jia J (1998) Preference conditions for utility models: a risk-value perspective. Ann Oper
Res 80(1):167–182
Dyer JS, Sarin RK (1979) Measurable multiattribute value functions. Oper Res 27(4):810–822
EPA SAB (U.S. Environmental Protection Agency Science Advisory Board) (1990) Reducing
risk: setting priorities and strategies for environmental protection. SAB-EC-90-021. U.S.
Environmental Protection Agency Science Advisory Board, Washington, DC [online].
Available http://yosemite.epa.gov/sab/sabproduct.nsf/28704D9C420FCBC1852573360053C6
92/$File/REDUCING+RISK++++++++++EC-90-021_90021_5-11-1995_204.pdf. Accessed
14 Sept 12
Gintis H, Bowles S, Boyd R, Fehr E (2003) Explaining altruistic behavior in humans. Evol Hum
Behav 24:153–172
Gintis H (2000) Game Theory Evolving: A problem-centered introduction to modeling strategic
interaction. Princeton University Press, Princeton, NJ
Glazebrook KD, Minty R (2009) A generalized gittins index for a class of multiarmed Bandits with
general resource requirements. Math Oper Res 34(1):26–44
Harford T (2011) Adapt: why success always starts with failure. Farra, Straus and Giroux, New
York
Hazen G, Sounderpandian J (1999) Lottery acquisition versus information acquisition: price and
preference reversals. J Risk Uncertainty 18(2):125–136
Hubbard DW (2009) The failure of risk management: why it’s broken and how to ﬁx it. Wiley,
New York
Infanger G (2006) Dynamic asset allocation strategies using a stochastic dynamic programming
approach. Chapter 5. In: Zenios SA, Ziemba WT (eds) Handbook of assets and liability man-
agement, volume 1. North Holland, New York
ISO31000 http://www.iso.org/iso/catalogue_detail?csnumber=43170. Accessed 8 July 2011
Jones P, Edmonds Y (2008) Risk-based strategies for allocating resources in a constrained environ-
ment. J Homeland Security. www.homelandsecurity.org/newjournal/Articles/displayArticle2.
asp?article=171
Keeney RL, Raiffa H (1976) Decisions with multiple objectives: preferences and value trade-offs.
Wiley, New York
MacIntyre CR, Seccull A, Lane JM (2006) Plant A. Development of a risk-priority score for cat-
egory A bioterrorism agents as an aid for public health policy. Mil Med 171(7):589–594
Martello S, Toth P (1990) Knapsack problems: algorithms and computer interpretations.
Wiley-Interscience, New York, NY
Mitchell C, Decker C (2004) Applying risk-based decision-making methods and tools to U.S.
Navy Antiterrorism Capabilities. J Homeland Security http://www.au.af.mil/au/awc/awcgate/
ndia/mitchell_rbdm_terr_hls_conf_may04.pdf. Accessed 14 Sept 12

Pfanzagl J (1959) A general theory of measurement. Applications to utility. Naval Research
Logistic Quarterly 6:283–294
Rosenthal EC (2011) The Complete idiot’s guide to game theory. The Penguin Group. Alpha
Books, New York, New York
Senju S, Toyoda Y (1968) An approach to linear programming with 0–1 variables. Manag Sci
15(5):B-196–B-207
Sethuraman J, Tsitsiklis J (2007) Stochastic search in a forest revisited. Math Oper Res 589–593.
http://www.columbia.edu/~js1353/pubs/search.pdf
Wilson R (1968) The theory of syndicates. Econometrica 336(1):119–132

Improving risk analysis

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to Improving risk analysis (20)

More from Springer (20)

Recently uploaded (20)

Improving risk analysis