Modelling Pitfalls - extra resources

Pitfalls of ABM, Bruce Edmonds, ESSA Summer School, Aberdeen, June 2019. slide 1
the Pitfalls of ABM
– some resources
Bruce Edmonds
Centre for Policy Modelling
Manchester Metropolitan University

Some Different Model Purposes

Purpose 1: Prediction

Motivation
• If you can reliably predict something about the
world (that you did not already know), this is
undeniably useful…
• ...even if you do not know why your model
predicts (e.g. a black-box model)!
• But it has also become the ‘gold standard’ of
science…
• ...becuase (unlike many of the other purposes) it
is difficult to fudge or fool yourself about – if its
wrong this is obvious.

Predictive modelling
Target system
Initial
Conditions
Outcomes
Predictive Model
Model
set-up
Model
results
(Hesse 1963)

What it is
The ability to anticipate unknown data reliably and
to a useful degree of accuracy
• Some idea of the conditions in which it does this
have to be understood (even if this is vague)
• The data it anticipates has to be unknown to the
modeller when using the model
• What is a useful degree of accuracy depends on
the purpose for predicting
• What is predicted can be: categorical, probability
distributions, ranges, negative predictions, etc.

Examples
• The gas laws (temperature is proportional to
pressure at the same volume etc.) predict future
measurements on a gas without any indication of
why this works
• Nate Silver’s team tries to predict the outcome of
sports events and elections using computational
models. These are usually probabilistic
predictions and the predicted distribution of
predictions is displayed (http://fivethirtyeight.com
and Silver 2013)

Risks and Warnings
• There are two different uses of the word ‘predict’:
one as above and one to indicate any calculation
made using a model (the second confuses others)
• This requires repeated attempts at anticipating
unknown data (and learning from this)
• because it is otherwise impossible to avoid ‘fitting’
known data (due to publication bias etc.)
• If the outcome is unknown and can be
unambiguously checked it could be predictive
• Prediction is VERY hard in the social sciences –
for this reason, it is rarely done

Mitigating Measures
• The following are documented:
– what aspects it predicts
– roughly when it predicts well
– what degree of accuracy it predicts with
• Check that the model predicts on several
independent cases
• Ensure the program is distributed so others can
independently check its predictions

Purpose 2: Explanation

Motivation
• When one wants to understand why or how
something happens
• One makes a simulation with the mechanisms
one wants and then shows that the results fit the
observed data
• The intricate workings of the simulation runs
support an explanation of the outcomes in terms
of those mechanisms
• The explanation is usually an abstraction of the
model workings, so as to be comprehensible to us
(e.g. a hypothesis about model behaviour)

What it is
Establishing a possible causal chain from a set-up
to its consequences in terms of the mechanisms of
a simulation
• The causation can be deterministic, possibilistic or
probabilistic
• The nature of the set-up constrains the terms that
the explanation is expressed in
• Only some aspects of the results will be relevant
to be matched to data
• But how the model maps to data/evidence is
explicitly specified

Explanatory modelling
Mechanisms
Model
processes
Model
results
Outcomes
Model
Target System
Outcomes are explained
by the processes

Examples
• The model of a gas with atoms randomly bumping
around explains what happens in a gas (but does
not directly predict the values)
• Lansing & Kramer’s (1993) model of water
distribution in Bali, explained how the system of
water temples act to help enforce social norms
and facilitate a complicated series of negotiations

Risks and Warnings
• A bug in the code is fatal to this purpose if this
could change the outcomes substantially
• The fit to the target data maybe a very special
case which would limit the likelihood of the
explanation over similar cases
• The process from mechanisms to outcomes might
be complex and poorly understood. The
explanation should be clearly stated and tested.
Assumptions behind this must be tested.
• There might well be more than one possible
explanation (and/or model)!

Mitigating Measures
• Ensure the built-in mechanisms are plausible and
at the right kind to support an explanation
• Be clear which aspects of the output are
considered significant (i.e. those that are
explained) and which artifacts of the simulation
• Probe the simulation to find when the explanation
works (noise, assumptions etc)
• Do classical experiments to show your
explanation works for your code

Purpose 3: Theory Exposition

Motivation
• If one has a system of equations, sometimes one
can analytically solve the equations to get a
general solution (i.e. a ‘closed form’ solution)
• When this is not possible (the case for almost all
complicated systems) we can calculate specific
examples – i.e. we simulate it!
• Using multiple runs, we aim to sufficiently explore
the whole space of behaviour to understand the
effect of this particular set of abstract mechanisms
• We might approximate these with equations (or a
simpler model) to check this understanding

What it is
Discovering then establishing (or refuting)
hypotheses about the general behaviour of a set of
mechanisms
• The hypotheses may need to be discovered
• But, crucially, showing the hypotheses hold (or
are refuted) by the set of experiments
• There needs to be a wide (maybe even complete)
exploration of outcomes
• The hypotheses need to be quite general for the
exercise to be useful to others
• Does not say anything about the observed world!

Modelling to understand Theory
Model
processes
Model
results
Model
Target System
Hypothesis or general characterisation of behaviour

Examples
• Many economic models are explorations of sets of
abstract mechanisms
• Deffuant, G., et al. (2002) How can extremism
prevail?
jasss.soc.surrey.ac.uk/5/4/1.html
• Edmonds & Hales (2003) Replication…
jasss.soc.surrey.ac.uk/6/4/11.html

Risks, Warnings & Mitigation
• A bug in the code is dangerous to this purpose
since the explanation might partly be based on an
understanding of what the code was to do
• A general idea of the outcome behaviour is
needed so the exploration needs to be extensive
• The code needs to be available so that people
can test its assumptions etc.
• Clarity about what is claimed, the model
description etc. is very important

Purpose 4: Illustration

Motivation & What it is
• An idea is new but has complex ramifications and
one wants to simply illustrate it
• This is a way of communicating through a single
(but maybe complex) example
A behaviour or system is illustrated precisely using
a simulation in an understandable way
• It might be a very special case, no generality is
established or claimed
• It might be used as a counter-example
• Simpler models are easier to communicate and
hence often make more vivid illustrations

Examples
• Sakoda/Schelling’s 2D Model of segregation
which showed that a high level of racial
intollerance was not necessary to explain patterns
of segregation
• Riolo et al. (2001) Evolution of cooperation
without reciprocity, Nature 414:441-443.
• Baum, E. (1996) Toward a model of mind as a
laissez-faire economy of idiots.

Purpose 5: Analogy

Motivation & What it is
• Provides a ‘way of thinking about’ stuff
• The model does not (directly) tell us about
anything observed, but is about ideas (which, in
turn, may or may not relate to something
observed)
• It can suggest new insights – e.g. new
hypotheses or future research directions
• We need analogies to help us think about what to
do (e.g. what and how to model)
• They are unavoidable
• They are very useful, but can also be deceptive

Models as Analogies
Intuitive understanding expressed in normal
language
Observations of the system of concern
Models of the processes in the
system
Common-SenseComparison

Examples
• Axelrod’s Evolution of Cooperation models (1984
etc.)
• Hammond & Axelrod (2006) The Evolution of
Ethnocentrism. Journal of Conflict Research
• Many economic models which show an ‘efficient’
market
• Many ecological models showing how systems
reach an equilibrium

Warnings
• When one has played with a model the whole
world looks like that model (especially to the
model builder)
• But this does not make this true!
• Such models can be very influential but (as with
the economic models of risk about lending) can
be very misleading
• At best, they can suggest hypotheses about the
observed world, but they don’t demonstrate
anything

How to decide what a model purpose is
Is there a well-defined
mapping from model
to evidence?Yes No
Does it describe a
particular case?Yes
Does it predict
unknown data reliably?Yes
No
No
Descriptive
Predictive
Explanatory
Does it thoroughly explore
the consequences of
some mechanisms?
NoYes
Theory
Exposition
Is it a general way of
thinking about a set of
phenomena?
Yes
Analogical
No
Illustration
Empirical Theoretical
or
Conceptual

Some Pitfalls in Model Construction

Modelling Assumptions
• All models are built on assumptions, but…
• They have different origins and reliability, e.g.:
– Empirical evidence
– Other well-defined theory
– Expert Opinion
– Common-sense
– Tradition
– Stuff we had to assume to make the model possible
• Choosing assumptions is part of the art of
simulation but which assumptions are used
should be transparent and one should be honest
about their reliability – plausibility is not enough!

Theoretical Spectacles
• Our conceptions and models constrain how we
1. look for evidence (e.g. where and what kinds)
2. what kind of models we develop
3. how we evaluate any results
• This is Kuhn’s “Theoretical Spectacles” (1962)
– e.g. continental drift
• This is MUCH stronger for a complex simulation
we have immersed ourselves in
• Try to remember that just because it is useful to
think of the world through our model, this does
not make them valid or reliable

Over-Simplified Models
• Although simple models have many pragmatic
advantages (easier to check, understand etc.)…
• If we have missed out key elements of what is being
modelled it might be completely wrong!
• Playing with simple models to inform formal and
intuitive understanding is an OK scientific practice
• …but it can be dangerous when informing policy
• Simple does not mean it is roughly correct, or more
general or gives us useful intuitions
• Need to accept that many modelling tasks requested
of us by policy makers are not wise to do with
restricted amounts of time/data/resources

Underestimating model limitations
• All models have limitations
• They are only good for certain things: a model
that explains well might not predict well
• The may well fail when applied in a different
context than the one they were developed in
• Policy actors often do not want to know about
limitations and caveats
• Not only do we have to be 100% honest about
these limitations, but we also have to ensure that
these limitations are communicated with the
model

Not checking & testing a model
thoroughly
• Doh!
• Sometimes there is not a clear demarcation
between an exploratory phase of model
development and its application to serious
questions (whose answers will impact on others)
• Sometimes an answer is demanded before
thorough testing and checking can be done – “Its
OK, I just want an approximate answer” :-/
• Sometimes researchers are not honest
• Depends on the potential harm if the model is
relied on (at all) and turns out to be wrong

Some Pitfalls in Model Application

Insufficiently Validated Models
• One can not rely on a model until it has been
rigorously checked and tested against reality
• Plausibility is nowhere NEAR enough
• This needs to be on more than one case
• Its better if this is done independently
• You can not validate a model using one set of
settings/cases then rely on it in another
• Validation usually takes a long time
• Iterated development and validation over many
cycles is better than one-off models (for policy)

Promising too much
• Modellers are in a position to see the potential of
their work, and so can tantalise others by
suggesting possible/future uses (e.g. in the
conclusions of papers or grant applications)
• They are tempted to suggest they can ‘predict’,
‘evaluate the impact of alternative polices’ etc.
• Especially with complex situations (that ABM is
useful for) this is simply deceptive
• ‘Giving a prediction to a policy maker is like giving
a sharp knife to a child’

The inherent plausibility of ABMs
• Due to the way ABMs map onto reality in a
common-sense manner (e.g. peopleagents)…
• …visualisations of what is happening can be
readily interpretted by non-modellers
• and hence given much greater credence than
they warrant (i.e. the extent of their validation)
• It is thus relatively easy to persuade using a good
ABM and visualisation
• Only we know how fragile they are, and need to
be especially careful about suggesting otherwise

Model Spread
• On of the big advantages of formal models is that
they can be passed around to be checked, played
with, extended, used etc.
• However once a model is out there, it might get
used for different purposes than intended
• e.g. the Black-Scholes model of derivative pricing
• Try to ensure a released model is packaged with
documentation that warns of its uses and
limitations

Narrowing the evidential base
• The case of the Newfoundland cod, indicates how
models can work to constrain the evidence base,
therefore limiting decision making
• If a model is considered authoritative, then the
data it uses and produces can sideline other
sources of evidence
• Using a model rather than measuring lots of stuff
is cheap, but with obvious dangers
• Try to ensure models are used to widen the
possibilities considered, rather than limit them

Other/General Pitfalls

Confusion over model purpose
• A model is not a picture of reality, but a tool
• A tool has a particular purpose
• A tool good for one purpose is probably not good
for another
• These include: prediction, explanation, as an
analogy, an illustration, a description, for theory
exploration, or for mediating between people
• Modellers should be 100% clear under which
purpose their model is to be judged
• Models need to be justified for each purpose
separately

When models are used out of the
context they were designed for
• Context matters!
• In each context there will be many
conditions/assumptions we are not even aware of
• A model designed in one context may fail for
subtle reasons in another (e.g. different ontology)
• Models generally need re-testing, re-validating
and often re-developing in new contexts

What models cannot reasonably do
• Many questions are beyond the realm of models
and modellers but are essentially
– ethical
– political
– social
– semantic
– symbolic
• Applying models to these (outside the walls of
our academic asylum) can confuse and distract

The uncertainty is too great
• Required reliability of outcome values is too low
for purpose
• Can be due to data or model reasons
• Radical uncertainty is when its not a question of
degree but the situation might fundamentally
change or be different from the model
• Error estimation is only valid in absence of radical
uncertainly (which is not the case in almost all
ecological, technical or social simulations)
• Just got to be honest about this and not only
present ‘best case’ results

A false sense of security
• If the outcomes of a model give a false sense of
certainly about outcomes then a model can be
worse than useless; positively damaging to policy
• Better to err on the side of caution and say there
is not good model in this case
• Even if you are optimistic for a particular model
• Distinction here between probabilistic and
possibilistic views

Not more facts, but values!
• Sometimes it is not facts and projections that are
the issue but values
• However good models are, the ‘engineering’
approach to policy (enumerate policies, predict
impact of each, choose best policy) might be
inappropriate
• Modellers caught on the wrong side of history
may be blamed even though they were just doing
the technical parts

The End
Bruce Edmonds: bruce@edmonds.name
Centre for Policy Modelling: http://cfpm.org
A version of these slides will be in the shared dropbox
folder and at:
http://slideshare.com/BruceEdmonds

Modelling Pitfalls - extra resources

More Related Content

Similar to Modelling Pitfalls - extra resources (20)

More from Bruce Edmonds (16)

Recently uploaded (20)

Modelling Pitfalls - extra resources