SlideShare a Scribd company logo
Model Purpose and Complexity
Bruce Edmonds
Centre for Policy Modelling
Manchester Metropolitan University
A model is not a ‘picture’
• Even if we often think of it in this way…
• …a model is not a picture of its target, not a
simulacrum (well almost never).
• Rather it is a tool to help deal with it, e.g.:
– To understand it
– To predict it
– To communicate about it
• Just as machines extend our physical
abilities, models extend our mental abilities.
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 2
Different tools for different jobs
• A good tool is well designed for its purpose
• Each model is just such a tool
• However, there are many alternative
models for every target so that we do not
know what model is good for what purpose
and what target
• So to be worth bothering other people about
our models, to not waste their time…
• …our models needs to be justified with
respect to a stated purpose and target etc.
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 3
There are no generic models…
• ..yet and there seem little prospect for them
in the foreseeable future.
• A ‘Darwin’ for the social sciences has not
arrived with a integrative explanatory theory
that connects social phenomena in a way
that checks out against evidence
• Rather, we are in a ‘pre-integrative’ stage –
doing the equivalent of describing finches
on the Galapagos Islands – specific models
for specific purposes
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 4
Simpler is not more general
• Whilst one can add in specific detail to a
simpler model to fit what is known about a
specific case (making it less general)…
• ...the other way around does not generally
work, making a model simpler usually makes it
less general!
• This is because we do not know which of the
processes are essential to a target, and might
simplify these away
• Imagine removing acceleration from
Newtonian dynamics to just use linear
equations – the resulting approximations would
only work at a few specific points!
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 5
Using models as an analogy
• Sometimes models are not about the observed
world, but related to our ideas.
• In this case a model is used as a kind of
complicated analogy – a way of thinking
• It does not relate to data but to a natural
language understanding
• The trouble is we apply analogies with great
(unconscious) fluidity, inventing a new way of
relating the analogy to a situation ‘on the fly’
• But this is different from empirical models
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 6
Models stage understanding
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 7
Intuitive understanding expressed in normal
language
Observations of the system of concern
Data obtained by measuring the
system
Models of the processes in the
system
Common-SenseComparison
ScientificComparisons
Different model purposes
• There are many different kinds of ways to
use a model.
• Each such purpose has its own benefits
and dangers...
• ...and needs different things checking for
different purposes...
• ...and probably needs to be developed in a
different way.
• The next section looks at these different
model purposes.
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 8
Purpose 1: Prediction
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 9
Motivation
• If you can reliably predict something about
the world, this is undeniably useful…
• ...even if you do not know why your model
predicts (e.g. a black-box model)!
• But it has also become the ‘gold standard’
of science…
• ...becuase (unlike many of the other
purposes) it is difficult to fudge or fool
yourself about – if its wrong this is obvious.
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 10
Predictive modelling
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 11
Target system
Initial
Conditions
Outcomes
Predictive Model
Model
set-up
Model
results
What it is
The ability to anticipate unknown data reliably
and to a useful degree of accuracy
• Some idea of the conditions in which it does
this well enough have to be understood
• The data it anticipates has to be unknown to
the modeller when building the model
• What is a useful degree of accuracy depends
on its application
• What is predicted can be: categorical,
probability distributions, ranges, negative
predictions, etc.
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 12
Examples
• The gas laws (temperature is proportional
to pressure at the same volume etc.) predict
future measurements on a gas without any
indication of why this works
• Nate Silver’s team tries to predict the
outcome of sports events and elections
using computational models. These are
usually probabilistic predictions and the
wider predicted distribution of outcomes is
displayed (http://fivethirtyeight.com)
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 13
Warnings
• There are two different uses of the word
‘predict’: one as above and one to indicate any
calculation made using a model.
• This requires repeated attempts at anticipating
unknown data (and learning from this)
• because it is impossible to avoid ‘fitting’ known
data (e.g. due to publication bias)
• If the outcome is unknown and can be
unambiguously checked it could be predictive
• Prediction is VERY hard in the social sciences,
it is rarely done
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 14
Mitigating Measures
• The following are documented:
– what aspects it predicts
– roughly when it predicts well
– what degree of accuracy it predicts with
• That the model has been tested that it
predicts on several independent cases
• That the model code is distributed so others
can test it and seek to understand how and
when it predicts
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 15
Purpose 2: Explanation
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 16
Motivation
• When one wants to understand why or how
something happens
• One makes a simulation with the
mechanisms one wants and then shows
that the results fit the observed data
• The intricate workings of the simulation runs
support an explanation of the outcomes in
terms of those mechanisms
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 17
What it is
Establishing a possible causal chain from a
set-up to its consequences in terms of the
mechanisms of a simulation
• The causation can be deterministic,
possibilistic or probibilistic
• The nature of the set-up determines the
terms that the explanation is expressed in
• Only some aspects of the results will be
relevant to be matched to data
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 18
Explanatory modelling
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 19
Mechanisms
Model
processes
Model
results
Outcomes
Model
Target System
Outcomes are explained
by the processes
Examples
• The model of a gas with atoms randomly
bumping around explains what happens in
a gas (but does not directly predict the
values)
• Lansing & Kramer’s (1993) model of water
distribution in Bali, explained how the
system of water temples acted to enforce
social norms and a complicated series of
negotiations
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 20
Warnings
• The fit to the target data maybe a very
special case which would limit the likelihood
of the explanation over other cases
• The process from mechanisms to outcomes
might be complex and poorly understood.
The explanation should be clearly stated
and tested. Assumptions behind this must
be tested.
• There might well be more than one possible
explanation (and/or model)!
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 21
Mitigating Measures
• Ensure the built-in mechanisms are
plausible and at the right level
• Be clear which aspects of the output are
considered significant and which artifacts of
the simulation
• Probe the simulation to find when the
explanation works (noise, assumptions etc)
• Do classical experiments to show your
explanation works for your code
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 22
Purpose 3: Theory Exposition
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 23
Motivation
• If one has a system of equations,
sometimes one can analytically solve the
equations to get a general solution
• When this is not possible (almost all
complicated systems) we can calculate
specific examples – to simulate it!
• We aim to sufficiently explore the whole
space of behaviour to understand a
particular set of abstract mechanisms
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 24
What it is
Discovering then establishing (or refuting)
hypotheses about the general behaviour of a
set of mechanisms
• The hypotheses may need to be discovered
• But crucially showing the hypotheses hold
(or are refuted) by the set of experiments
• the hypotheses need to be quite general for
the exercise to be useful to others
• Does not say anything about the observed
world!
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 25
Modelling to understand Theory
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 26
Model
processes
Model
results
Model
Target System
Hypothesis or general characterisation of behaviour
Examples
• Many economic models are explorations of
sets of abstract mechanisms
• Deffuant, G., et al. (2002) How can
extremism prevail?
jasss.soc.surrey.ac.uk/5/4/1.html
• Edmonds & Hales (2003) Replication…
jasss.soc.surrey.ac.uk/6/4/11.html
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 27
Warnings & Mitigation
• A bug in the code is fatal to this purpose
• A general idea of the outcome behaviour so
the exploration needs to be extensive
• Clarity about what is claimed, the model
description etc. is very important
• It it tempting to use the model as a way of
thinking about the world, but this is
dangerous!
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 28
Purpose 4: Illustration
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 29
Motivation & What it is
• An idea is new but complex and one wants
to simply illustrate it
• This is a way of communicating through a
single (but maybe complex) example
A behaviour or system is illustrated precisely
using a simulation
• It might be a very special case, no
generality is established
• It might be used as a counter-example
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 30
Examples
• Sakoda/Schelling’s 2D Model of
segregation which showed that a high level
of racial intollerance was not necessary to
explain patterns of segregation
• Riolo et al. (2001) Evolution of cooperation
without reciprocity, Nature 414:441-443.
• Baum, E. (1996) Toward a model of mind
as a laissez-faire economy of idiots.
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 31
Purpose 5: Analogy
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 32
Motivation & What it is
• Provides a ‘way of thinking about’ stuff
• The model is not (directly) about anything
observed, but about ideas (which, in turn,
may or may not relate to something
observed)
• It can suggest new insights or new future
directions for research
• We need analogies to help us think about
what to do (e.g. what and how to model)
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 33
An illustration of analogical use
of a model
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 34
Target system 1
Model
Informal Ideas
Target system 2
Examples
• Axelrod’s Evolution of Cooperation models
(1984 etc.)
• Hammond & Axelrod (2006) The Evolution
of Ethnocentrism. Journal of Conflict
Research
• Many economic models which show the
efficiency of markets
• Many ecological models showing how
systems reach an equilibrium
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 35
Warnings
• When one has played with a model the
whole world looks like that model
• But this does not make this true!
• Such models can be very influential but (as
with the economic models of risk about
lending) can be very misleading
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 36
Purpose 6: Description
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 37
Motivation & What it is
• Much science involves a lot of description
(e.g. Darwin drawing Finches)
• Simulations can also be used in this way
This is an attempt to partially represent what
is important of a specific observed case
• It does abstract (as all modelling does) but
cautiously, retaining as much relevant detail
as possible
• Later we might go back to the description
and learn something new from it
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 38
Examples
• Scott Moss’s (1998) Model of handling
crises in a water pumping station. JASSS
1(4):1, jasss.soc.surrey.ac.uk/1/4/1.html
• Richard Taylors (2003) thesis.
http://cfpm.org/cpmrep137.html
• Sukaina Bharwani’s (2004) thesis.
Translating interviews with farmers into a
simulation http://goo.gl/MzJJR7
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 39
Warnings & Mitagation
• These models should have a good
evidential base – qualitative or quantitative
• Might use expert or stakeholder opinion
• Assumptions and their basis should be very
carefully documented
• They maybe complex but have NO level of
generality – they are a particular case
• Need later further work for generalisation or
analysis of complicated simulations
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 40
Summary of Modelling Purposes
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 41
Some common confusions
• Firstly in many publications researchers do
not make their model purpose clear
• So the model is hard to judge properly
• Some have simply not thought about it!
Some common confusions:
• Theory  Analogy
• Illustration  Explanation
• Description  Explanation
• Explanation  Prediction
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 42
Pragmatics of Model Development
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 43
Exploratory & Consolidation
• It is common that one does not know clearly
what one wants to program
• In this case you might ‘play’ with models,
exploring the possibilities, getting an idea of
what works, what is possible etc.
• But this does not make a model suitable for
communicating with others
• Rather you then need to re-do the model
properly and justify what it does rigourously
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 44
How complicated should you
make your model?
• Simpler models are easier for us humans to
deal with. Easier to: program, check,
understand, communicate, analyse etc.
• But if you leave out crucial processes your
model may simply not be adequate to
predict, explain etc. your target
• In general, we have no idea what is
necessary and what is not – this is a
major area of contention and confusion!
• So what to do? What strategy to follow?
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 45
KISS – keep it simple stupid!
• An engineering approach
• Start with a simple model
and add features only
when the simpler is shown
not to work
• Trouble is it maybe a
combination of
mechanisms that are
required – trying them one
at a time might not work
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 46
SimpleComplex
KIDS – keep it descriptive stupid!
• Start with the available
evidence of what is important
• Then explore variations from
there, maybe showing some
are not required
• Trouble is this makes for
complicated models so care
is needed in development
and further work needed to
understand your model
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 47
SimpleComplex
Which is better?
• There is no reason to suppose that simple
models will be adequate for socio-
ecological phenomena
• Limitations on: human understanding, time,
resources, data are inevitable and should
simply be honestly declared
• For each feature the decision as to whether
to include it or not include it is important and
needs justified
• This depends on the model purpose!
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 48
Features and Purpose
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 49
Using models together
• Part of the answer seems to be that we have
to use ‘clusters’ of models together
For example:
• A simpler model of a more complex model to
understand it (model chains, meta-modelling)
• Different (but related) models for different
aspects and different purposes
• Simpler models for understanding, but more
complex for actual use but checked against the
simpler
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 50
Many models but distinct
purposes
• Use different kinds of model together, but
keep them distinct!
• Each model for a single purpose
• Think how the models relate to each other
• If a model has two purposes, it is better to
think of them as different models (e.g.
modelA-theory and modelA-analogy)
• In presentations/publications you need to
justify a model for each use seperately
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 51
Conclusion: Reflective Modelling
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 52
Think!
• Think about what you are trying to do, and
why as you develop or use your simulations
• Keep the models and purposes distinct but
think how they relate
• Analogies are needed to think, including
what we are going to model and why…
• But the serious models themselves are the
business, they either demonstrate or not
that they are adequate to their purpose
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 53
Check!
• There are many problems from jumping to
publication when finding interesting results
• But simulations need a lot of work before it
is worth wasting others’ time with them
• Lots of checking of the code, the results etc.
are necessary
(jasss.soc.surrey.ac.uk/12/1/1.html)
• Lots of documentation of your code, your
results, your experiments, what fails etc. are
necessary
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 54
Share!
• You are not alone!
• The power of formal models (like computer
simulations) is they can be unambiguously
shared with others who can then run, check,
inspect, change etc.
• Share your models and ideas at an early stage
(e.g. at OpenABM.org), be honest and open –
this is the way to learn
• But distinguish between developing work and
mature work so others can understand how
seriously to take it
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 55
Beware!
• We, as humans, are very, VERY, VERY good
at deceiving ourselves
• Simulation models make this worse!
• We use models as a way of thinking about
things and then it is hard to think about them in
other ways (including new ways)
• Any way to shake us up and reconsider this is
good – empirical data is a good way of doing
this, critiquing models can also help
• You should be throwing away most of your
simulations, not elaborating bad models
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 56
The key idea of this talk
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 57
some
observed
phenomena
Model
Simplistic
picture of a
universal
modelling
approach
Illustrative
Model
The key idea of this talk
Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 58
some
observed
phenomena
Explanatory
Model
Predictive
Model
Theoretical
Model Analogical
Model
We build this up
ONE model at a
time, justifying
each as we do
The End
Bruce Edmonds http://bruce.edmonds.name
Centre for Policy Modelling http://cfpm.org
A paper that goes into 5 of the modelling purposes:
http://cfpm.org/discussionpapers/192
These slides will be available at:
http://slideshare.com/BruceEdmonds

More Related Content

PPTX
PPT
Water fluoridation
PPTX
Dental factors affecting occlusal development.pptx
PPT
PPTX
Coronary heart disease
PPT
Dental caries
PPTX
Principles of Radiographic Interpretations
Water fluoridation
Dental factors affecting occlusal development.pptx
Coronary heart disease
Dental caries
Principles of Radiographic Interpretations

Similar to Model Purpose and Complexity (20)

PPTX
6 Modelling Purposes
PPTX
Different Modelling Purposes - an 'anit-theoretical' approach
PPTX
Modelling Pitfalls - extra resources
PPTX
Modelling and Knowledge
PDF
Big Data and algorithms
PPTX
Causal modelling - Series of lectures on causal modelling in the social sciences
PDF
Robustness in Machine Learning Explanations Does It Matter-1.pdf
PPTX
ARTIFICIAL INTELLIGENCE AND ETHICS 29.pptx
PDF
Vodafone Mathematical Modelling 2024.pdf
PDF
Forget about AI and do Mathematical Modelling instead!
DOCX
Exploring the World of Models-A Comprehensive Guide
DOCX
Chapter 2Theories & Models Week II – Slides 1
PPTX
Be ea-talk-final
PPTX
Improving AI Development - Dave Litwiller - Jan 11 2022 - Public
PPTX
Deciphering AI - Unlocking the Black Box of AIML with State-of-the-Art Techno...
PPT
Simulation Powerpoint- Lecture Notes
PPT
20121121101127simulation azmi
PDF
Doing data science for strategic business decisions
PDF
Data science and good questions eric kostello
PPTX
Finding out what could go wrong before it does – Modelling Risk and Uncertainty
6 Modelling Purposes
Different Modelling Purposes - an 'anit-theoretical' approach
Modelling Pitfalls - extra resources
Modelling and Knowledge
Big Data and algorithms
Causal modelling - Series of lectures on causal modelling in the social sciences
Robustness in Machine Learning Explanations Does It Matter-1.pdf
ARTIFICIAL INTELLIGENCE AND ETHICS 29.pptx
Vodafone Mathematical Modelling 2024.pdf
Forget about AI and do Mathematical Modelling instead!
Exploring the World of Models-A Comprehensive Guide
Chapter 2Theories & Models Week II – Slides 1
Be ea-talk-final
Improving AI Development - Dave Litwiller - Jan 11 2022 - Public
Deciphering AI - Unlocking the Black Box of AIML with State-of-the-Art Techno...
Simulation Powerpoint- Lecture Notes
20121121101127simulation azmi
Doing data science for strategic business decisions
Data science and good questions eric kostello
Finding out what could go wrong before it does – Modelling Risk and Uncertainty
Ad

More from Bruce Edmonds (20)

PPTX
Staging Model Abstraction – an example about political participation
PPTX
Modelling Pitfalls - introduction and some cases
PPTX
The evolution of empirical ABMs
PPTX
Mixing fat data, simulation and policy - what could possibly go wrong?
PPTX
Social Context
PPTX
Using agent-based simulation for socio-ecological uncertainty analysis
PPTX
How social simulation could help social science deal with context
PPTX
Agent-based modelling, laboratory experiments, and observation in the wild
PPTX
Culture trumps ethnicity! – Intra-generational cultural evolution and ethnoce...
PPT
An Introduction to Agent-Based Modelling
PDF
Mixing ABM and policy...what could possibly go wrong?
PPTX
Socio-Ecological Simulation - a risk-assessment approach
PPTX
A Simple Model of Group Commoning
PPTX
Are Mixed-Methods Just a Fudge? The Dangers and Prospects for Integrating Qu...
PPTX
The Post-Truth Drift in Social Simulation
PPTX
Drilling down below opinions: how co-evolving beliefs and social structure mi...
PDF
Modelling Innovation – some options from probabilistic to radical
PPTX
Co-developing beliefs and social influence networks
PDF
Simulating Superdiversity
PDF
Risk-aware policy evaluation using agent-based simulation
Staging Model Abstraction – an example about political participation
Modelling Pitfalls - introduction and some cases
The evolution of empirical ABMs
Mixing fat data, simulation and policy - what could possibly go wrong?
Social Context
Using agent-based simulation for socio-ecological uncertainty analysis
How social simulation could help social science deal with context
Agent-based modelling, laboratory experiments, and observation in the wild
Culture trumps ethnicity! – Intra-generational cultural evolution and ethnoce...
An Introduction to Agent-Based Modelling
Mixing ABM and policy...what could possibly go wrong?
Socio-Ecological Simulation - a risk-assessment approach
A Simple Model of Group Commoning
Are Mixed-Methods Just a Fudge? The Dangers and Prospects for Integrating Qu...
The Post-Truth Drift in Social Simulation
Drilling down below opinions: how co-evolving beliefs and social structure mi...
Modelling Innovation – some options from probabilistic to radical
Co-developing beliefs and social influence networks
Simulating Superdiversity
Risk-aware policy evaluation using agent-based simulation
Ad

Recently uploaded (20)

PDF
Placing the Near-Earth Object Impact Probability in Context
PPTX
Taita Taveta Laboratory Technician Workshop Presentation.pptx
PDF
ELS_Q1_Module-11_Formation-of-Rock-Layers_v2.pdf
PPTX
EPIDURAL ANESTHESIA ANATOMY AND PHYSIOLOGY.pptx
PDF
Sciences of Europe No 170 (2025)
PDF
lecture 2026 of Sjogren's syndrome l .pdf
PPTX
Pharmacology of Autonomic nervous system
PDF
The scientific heritage No 166 (166) (2025)
PPTX
Microbiology with diagram medical studies .pptx
PPTX
cpcsea ppt.pptxssssssssssssssjjdjdndndddd
PDF
Biophysics 2.pdffffffffffffffffffffffffff
PPT
POSITIONING IN OPERATION THEATRE ROOM.ppt
PPTX
Classification Systems_TAXONOMY_SCIENCE8.pptx
PPTX
INTRODUCTION TO EVS | Concept of sustainability
PDF
HPLC-PPT.docx high performance liquid chromatography
PPTX
Protein & Amino Acid Structures Levels of protein structure (primary, seconda...
PDF
Cosmic Outliers: Low-spin Halos Explain the Abundance, Compactness, and Redsh...
PDF
Phytochemical Investigation of Miliusa longipes.pdf
PPT
6.1 High Risk New Born. Padetric health ppt
PDF
Looking into the jet cone of the neutrino-associated very high-energy blazar ...
Placing the Near-Earth Object Impact Probability in Context
Taita Taveta Laboratory Technician Workshop Presentation.pptx
ELS_Q1_Module-11_Formation-of-Rock-Layers_v2.pdf
EPIDURAL ANESTHESIA ANATOMY AND PHYSIOLOGY.pptx
Sciences of Europe No 170 (2025)
lecture 2026 of Sjogren's syndrome l .pdf
Pharmacology of Autonomic nervous system
The scientific heritage No 166 (166) (2025)
Microbiology with diagram medical studies .pptx
cpcsea ppt.pptxssssssssssssssjjdjdndndddd
Biophysics 2.pdffffffffffffffffffffffffff
POSITIONING IN OPERATION THEATRE ROOM.ppt
Classification Systems_TAXONOMY_SCIENCE8.pptx
INTRODUCTION TO EVS | Concept of sustainability
HPLC-PPT.docx high performance liquid chromatography
Protein & Amino Acid Structures Levels of protein structure (primary, seconda...
Cosmic Outliers: Low-spin Halos Explain the Abundance, Compactness, and Redsh...
Phytochemical Investigation of Miliusa longipes.pdf
6.1 High Risk New Born. Padetric health ppt
Looking into the jet cone of the neutrino-associated very high-energy blazar ...

Model Purpose and Complexity

  • 1. Model Purpose and Complexity Bruce Edmonds Centre for Policy Modelling Manchester Metropolitan University
  • 2. A model is not a ‘picture’ • Even if we often think of it in this way… • …a model is not a picture of its target, not a simulacrum (well almost never). • Rather it is a tool to help deal with it, e.g.: – To understand it – To predict it – To communicate about it • Just as machines extend our physical abilities, models extend our mental abilities. Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 2
  • 3. Different tools for different jobs • A good tool is well designed for its purpose • Each model is just such a tool • However, there are many alternative models for every target so that we do not know what model is good for what purpose and what target • So to be worth bothering other people about our models, to not waste their time… • …our models needs to be justified with respect to a stated purpose and target etc. Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 3
  • 4. There are no generic models… • ..yet and there seem little prospect for them in the foreseeable future. • A ‘Darwin’ for the social sciences has not arrived with a integrative explanatory theory that connects social phenomena in a way that checks out against evidence • Rather, we are in a ‘pre-integrative’ stage – doing the equivalent of describing finches on the Galapagos Islands – specific models for specific purposes Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 4
  • 5. Simpler is not more general • Whilst one can add in specific detail to a simpler model to fit what is known about a specific case (making it less general)… • ...the other way around does not generally work, making a model simpler usually makes it less general! • This is because we do not know which of the processes are essential to a target, and might simplify these away • Imagine removing acceleration from Newtonian dynamics to just use linear equations – the resulting approximations would only work at a few specific points! Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 5
  • 6. Using models as an analogy • Sometimes models are not about the observed world, but related to our ideas. • In this case a model is used as a kind of complicated analogy – a way of thinking • It does not relate to data but to a natural language understanding • The trouble is we apply analogies with great (unconscious) fluidity, inventing a new way of relating the analogy to a situation ‘on the fly’ • But this is different from empirical models Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 6
  • 7. Models stage understanding Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 7 Intuitive understanding expressed in normal language Observations of the system of concern Data obtained by measuring the system Models of the processes in the system Common-SenseComparison ScientificComparisons
  • 8. Different model purposes • There are many different kinds of ways to use a model. • Each such purpose has its own benefits and dangers... • ...and needs different things checking for different purposes... • ...and probably needs to be developed in a different way. • The next section looks at these different model purposes. Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 8
  • 9. Purpose 1: Prediction Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 9
  • 10. Motivation • If you can reliably predict something about the world, this is undeniably useful… • ...even if you do not know why your model predicts (e.g. a black-box model)! • But it has also become the ‘gold standard’ of science… • ...becuase (unlike many of the other purposes) it is difficult to fudge or fool yourself about – if its wrong this is obvious. Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 10
  • 11. Predictive modelling Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 11 Target system Initial Conditions Outcomes Predictive Model Model set-up Model results
  • 12. What it is The ability to anticipate unknown data reliably and to a useful degree of accuracy • Some idea of the conditions in which it does this well enough have to be understood • The data it anticipates has to be unknown to the modeller when building the model • What is a useful degree of accuracy depends on its application • What is predicted can be: categorical, probability distributions, ranges, negative predictions, etc. Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 12
  • 13. Examples • The gas laws (temperature is proportional to pressure at the same volume etc.) predict future measurements on a gas without any indication of why this works • Nate Silver’s team tries to predict the outcome of sports events and elections using computational models. These are usually probabilistic predictions and the wider predicted distribution of outcomes is displayed (http://fivethirtyeight.com) Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 13
  • 14. Warnings • There are two different uses of the word ‘predict’: one as above and one to indicate any calculation made using a model. • This requires repeated attempts at anticipating unknown data (and learning from this) • because it is impossible to avoid ‘fitting’ known data (e.g. due to publication bias) • If the outcome is unknown and can be unambiguously checked it could be predictive • Prediction is VERY hard in the social sciences, it is rarely done Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 14
  • 15. Mitigating Measures • The following are documented: – what aspects it predicts – roughly when it predicts well – what degree of accuracy it predicts with • That the model has been tested that it predicts on several independent cases • That the model code is distributed so others can test it and seek to understand how and when it predicts Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 15
  • 16. Purpose 2: Explanation Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 16
  • 17. Motivation • When one wants to understand why or how something happens • One makes a simulation with the mechanisms one wants and then shows that the results fit the observed data • The intricate workings of the simulation runs support an explanation of the outcomes in terms of those mechanisms Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 17
  • 18. What it is Establishing a possible causal chain from a set-up to its consequences in terms of the mechanisms of a simulation • The causation can be deterministic, possibilistic or probibilistic • The nature of the set-up determines the terms that the explanation is expressed in • Only some aspects of the results will be relevant to be matched to data Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 18
  • 19. Explanatory modelling Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 19 Mechanisms Model processes Model results Outcomes Model Target System Outcomes are explained by the processes
  • 20. Examples • The model of a gas with atoms randomly bumping around explains what happens in a gas (but does not directly predict the values) • Lansing & Kramer’s (1993) model of water distribution in Bali, explained how the system of water temples acted to enforce social norms and a complicated series of negotiations Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 20
  • 21. Warnings • The fit to the target data maybe a very special case which would limit the likelihood of the explanation over other cases • The process from mechanisms to outcomes might be complex and poorly understood. The explanation should be clearly stated and tested. Assumptions behind this must be tested. • There might well be more than one possible explanation (and/or model)! Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 21
  • 22. Mitigating Measures • Ensure the built-in mechanisms are plausible and at the right level • Be clear which aspects of the output are considered significant and which artifacts of the simulation • Probe the simulation to find when the explanation works (noise, assumptions etc) • Do classical experiments to show your explanation works for your code Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 22
  • 23. Purpose 3: Theory Exposition Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 23
  • 24. Motivation • If one has a system of equations, sometimes one can analytically solve the equations to get a general solution • When this is not possible (almost all complicated systems) we can calculate specific examples – to simulate it! • We aim to sufficiently explore the whole space of behaviour to understand a particular set of abstract mechanisms Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 24
  • 25. What it is Discovering then establishing (or refuting) hypotheses about the general behaviour of a set of mechanisms • The hypotheses may need to be discovered • But crucially showing the hypotheses hold (or are refuted) by the set of experiments • the hypotheses need to be quite general for the exercise to be useful to others • Does not say anything about the observed world! Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 25
  • 26. Modelling to understand Theory Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 26 Model processes Model results Model Target System Hypothesis or general characterisation of behaviour
  • 27. Examples • Many economic models are explorations of sets of abstract mechanisms • Deffuant, G., et al. (2002) How can extremism prevail? jasss.soc.surrey.ac.uk/5/4/1.html • Edmonds & Hales (2003) Replication… jasss.soc.surrey.ac.uk/6/4/11.html Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 27
  • 28. Warnings & Mitigation • A bug in the code is fatal to this purpose • A general idea of the outcome behaviour so the exploration needs to be extensive • Clarity about what is claimed, the model description etc. is very important • It it tempting to use the model as a way of thinking about the world, but this is dangerous! Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 28
  • 29. Purpose 4: Illustration Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 29
  • 30. Motivation & What it is • An idea is new but complex and one wants to simply illustrate it • This is a way of communicating through a single (but maybe complex) example A behaviour or system is illustrated precisely using a simulation • It might be a very special case, no generality is established • It might be used as a counter-example Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 30
  • 31. Examples • Sakoda/Schelling’s 2D Model of segregation which showed that a high level of racial intollerance was not necessary to explain patterns of segregation • Riolo et al. (2001) Evolution of cooperation without reciprocity, Nature 414:441-443. • Baum, E. (1996) Toward a model of mind as a laissez-faire economy of idiots. Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 31
  • 32. Purpose 5: Analogy Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 32
  • 33. Motivation & What it is • Provides a ‘way of thinking about’ stuff • The model is not (directly) about anything observed, but about ideas (which, in turn, may or may not relate to something observed) • It can suggest new insights or new future directions for research • We need analogies to help us think about what to do (e.g. what and how to model) Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 33
  • 34. An illustration of analogical use of a model Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 34 Target system 1 Model Informal Ideas Target system 2
  • 35. Examples • Axelrod’s Evolution of Cooperation models (1984 etc.) • Hammond & Axelrod (2006) The Evolution of Ethnocentrism. Journal of Conflict Research • Many economic models which show the efficiency of markets • Many ecological models showing how systems reach an equilibrium Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 35
  • 36. Warnings • When one has played with a model the whole world looks like that model • But this does not make this true! • Such models can be very influential but (as with the economic models of risk about lending) can be very misleading Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 36
  • 37. Purpose 6: Description Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 37
  • 38. Motivation & What it is • Much science involves a lot of description (e.g. Darwin drawing Finches) • Simulations can also be used in this way This is an attempt to partially represent what is important of a specific observed case • It does abstract (as all modelling does) but cautiously, retaining as much relevant detail as possible • Later we might go back to the description and learn something new from it Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 38
  • 39. Examples • Scott Moss’s (1998) Model of handling crises in a water pumping station. JASSS 1(4):1, jasss.soc.surrey.ac.uk/1/4/1.html • Richard Taylors (2003) thesis. http://cfpm.org/cpmrep137.html • Sukaina Bharwani’s (2004) thesis. Translating interviews with farmers into a simulation http://goo.gl/MzJJR7 Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 39
  • 40. Warnings & Mitagation • These models should have a good evidential base – qualitative or quantitative • Might use expert or stakeholder opinion • Assumptions and their basis should be very carefully documented • They maybe complex but have NO level of generality – they are a particular case • Need later further work for generalisation or analysis of complicated simulations Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 40
  • 41. Summary of Modelling Purposes Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 41
  • 42. Some common confusions • Firstly in many publications researchers do not make their model purpose clear • So the model is hard to judge properly • Some have simply not thought about it! Some common confusions: • Theory  Analogy • Illustration  Explanation • Description  Explanation • Explanation  Prediction Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 42
  • 43. Pragmatics of Model Development Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 43
  • 44. Exploratory & Consolidation • It is common that one does not know clearly what one wants to program • In this case you might ‘play’ with models, exploring the possibilities, getting an idea of what works, what is possible etc. • But this does not make a model suitable for communicating with others • Rather you then need to re-do the model properly and justify what it does rigourously Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 44
  • 45. How complicated should you make your model? • Simpler models are easier for us humans to deal with. Easier to: program, check, understand, communicate, analyse etc. • But if you leave out crucial processes your model may simply not be adequate to predict, explain etc. your target • In general, we have no idea what is necessary and what is not – this is a major area of contention and confusion! • So what to do? What strategy to follow? Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 45
  • 46. KISS – keep it simple stupid! • An engineering approach • Start with a simple model and add features only when the simpler is shown not to work • Trouble is it maybe a combination of mechanisms that are required – trying them one at a time might not work Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 46 SimpleComplex
  • 47. KIDS – keep it descriptive stupid! • Start with the available evidence of what is important • Then explore variations from there, maybe showing some are not required • Trouble is this makes for complicated models so care is needed in development and further work needed to understand your model Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 47 SimpleComplex
  • 48. Which is better? • There is no reason to suppose that simple models will be adequate for socio- ecological phenomena • Limitations on: human understanding, time, resources, data are inevitable and should simply be honestly declared • For each feature the decision as to whether to include it or not include it is important and needs justified • This depends on the model purpose! Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 48
  • 49. Features and Purpose Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 49
  • 50. Using models together • Part of the answer seems to be that we have to use ‘clusters’ of models together For example: • A simpler model of a more complex model to understand it (model chains, meta-modelling) • Different (but related) models for different aspects and different purposes • Simpler models for understanding, but more complex for actual use but checked against the simpler Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 50
  • 51. Many models but distinct purposes • Use different kinds of model together, but keep them distinct! • Each model for a single purpose • Think how the models relate to each other • If a model has two purposes, it is better to think of them as different models (e.g. modelA-theory and modelA-analogy) • In presentations/publications you need to justify a model for each use seperately Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 51
  • 52. Conclusion: Reflective Modelling Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 52
  • 53. Think! • Think about what you are trying to do, and why as you develop or use your simulations • Keep the models and purposes distinct but think how they relate • Analogies are needed to think, including what we are going to model and why… • But the serious models themselves are the business, they either demonstrate or not that they are adequate to their purpose Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 53
  • 54. Check! • There are many problems from jumping to publication when finding interesting results • But simulations need a lot of work before it is worth wasting others’ time with them • Lots of checking of the code, the results etc. are necessary (jasss.soc.surrey.ac.uk/12/1/1.html) • Lots of documentation of your code, your results, your experiments, what fails etc. are necessary Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 54
  • 55. Share! • You are not alone! • The power of formal models (like computer simulations) is they can be unambiguously shared with others who can then run, check, inspect, change etc. • Share your models and ideas at an early stage (e.g. at OpenABM.org), be honest and open – this is the way to learn • But distinguish between developing work and mature work so others can understand how seriously to take it Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 55
  • 56. Beware! • We, as humans, are very, VERY, VERY good at deceiving ourselves • Simulation models make this worse! • We use models as a way of thinking about things and then it is hard to think about them in other ways (including new ways) • Any way to shake us up and reconsider this is good – empirical data is a good way of doing this, critiquing models can also help • You should be throwing away most of your simulations, not elaborating bad models Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 56
  • 57. The key idea of this talk Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 57 some observed phenomena Model Simplistic picture of a universal modelling approach
  • 58. Illustrative Model The key idea of this talk Model Purpose and Complexity, Bruce Edmonds, ESSA Sum Sch. 2017, Wageningen, 58 some observed phenomena Explanatory Model Predictive Model Theoretical Model Analogical Model We build this up ONE model at a time, justifying each as we do
  • 59. The End Bruce Edmonds http://bruce.edmonds.name Centre for Policy Modelling http://cfpm.org A paper that goes into 5 of the modelling purposes: http://cfpm.org/discussionpapers/192 These slides will be available at: http://slideshare.com/BruceEdmonds