Different Modelling Purposes - an 'anit-theoretical' approach
Download as PPTX, PDF0 likes106 views
This document summarizes Bruce Edmonds' presentation on different purposes of modeling. It discusses how models can be used for prediction, explanation, theoretical exploration, and analogy, but each model must be justified for its specific purpose. It argues against vague notions of "theory" and advocates for developing more general models through a staged process of building on empirical models in gentler steps of abstraction, rather than through large leaps. The goal is to justify models and their relationships through empirical work before claiming general theories.
Different Modelling Purposes - an 'anit-theoretical' approach
- 1. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 1 Different Modelling Purposes – an ‘anti-theoretical’ account Bruce Edmonds Centre for Policy Modelling Manchester Metropolitan University
- 2. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 2 Acknowledgements 1. My bl***y PhD examiners 2. Colleagues with whom I have had many discussions on these issues, notably: Scott Moss, Alan McKane and David Hales 3. The attendees of the Manchester workshop on validation in 2013 4. My co-authors of the UK government report on Computational Modelling
- 3. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 3 What the h*** is theory anyway? • Although referred to often, (especially in older texts) it is not clear what a theory is (except normative accounts such as from the Logical Positivists etc.) • If you look up the “theory of *” in a text book, what you actually get is a related collection of: ideas, description of processes, explanations, laws, models, results etc. (Giere 1988) • However, calling something a ‘theory’ does tend to indicate that it (a) is more abstract (b) is more generally applicable and/or (c) has status • I will thus look at how we might build towards more generally applicable models and not focus on the word “theory” (except negatively ;-) )
- 4. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 4 Ways of thinking about a ‘model’ ‘Model’ is an overloaded term, so there are many different meanings and uses of it… …but three views are: 1. A model is a (partial, fuzzy) picture of some part of reality – the bits correspond in some way 2. A model mediates between us and reality – but this is too broad to be helpful – anything can ‘mediate’ in some way or other (e.g. block) 3. A model is a tool – machines extend our physical abilities; models extend our cognitive abilities
- 5. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 5 I argue that…. • Thinking of models as tools is more useful… • ...especially if you understand that different models have different kinds of purpose • That we use a cluster of different kinds of model for understanding what we study • So that the assessment and understanding of any particular model is bootstrapped on many previous models (explicitly or implicitly) • And we have many different strategies for developing useful models (KISS, KIDS etc.) • I will argue for a staged, ‘upwards’ strategy for achieving more general models using ABM
- 6. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 6 Models as Tools • A good tool is designed and assessed for a particular purpose – you could use a kitchen knife for carving wood and a chisel for carving meat, but the results will be messy • A ‘general purpose tool’ needs to be justified against each claimed use separately • A lot of confusion comes from the fact that models can have many different uses (Epstein 2008) • Here I will briefly review only four of these: prediction, explanation, theoretical exploration and analogy. (I will not cover illustration, description or mediation/learning between people)
- 7. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 7 Prediction • Prediction is called the ‘gold standard’ of science, partly because it is very useful to be able to do but also because it is a ‘hard test’ – it is hard to fudge • If you can reliably predict unknown data (unknown to the modeller) to a useful degree of accuracy (especially in social science) you have succeeded • In this case it does not matter what is in your model, or if the model mechanism is at all realistic • Sometimes prediction comes before explanation: the gas laws predicted properties of gasses long before the reasons why became apparent
- 8. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 8 Explanation • Where a model fits some data from observed phenomena using some plausible mechanisms • Such a fit supports an explanation of that phenomena in terms of those mechanisms • ABM can be used to support explanations of complex social systems • This is common in the social sciences • It does not mean the model predicts • Sometimes explanation precedes prediction: Darwin’s theory explained but prediction only came after its synthesis with genetics
- 9. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 9 Theoretical Exploration • Often it is infeasible to analytically derive a general closed form solution for the outcomes of a complex system • In this case one has to simulate – the simulation replaces the mathematical deduction • So models (including ABM) can be used to explore and establish the outcomes of complex mechanisms (especially distributed ones) • However, this is a non-empirical use, it does not tell us anything reliable about anything observed
- 10. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 10 Analogy – intuitive understanding • We need ideas to guide us as in science and how to think about unfamiliar situations • Here the model acts as a kind of analogy – a way of thinking about stuff • There is no well defined mapping from model to what is being modelled but rather this mapping is created ‘on the fly’ for each case it is used • Just because we can think of something using a model does not make it ‘true’… • ... but can fool us into thinking it is so, acting as Kuhn’s ‘theoretical spectacles’
- 11. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 11 Models as Analogies Intuitive understanding expressed in normal language Observations of the system of concern Data obtained by measuring the system Models of the processes in the system Common-SenseComparison ScientificComparisons
- 12. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 12 An example confusion • Friedman (1953) convinced others in Economics that their models did not have to be realistic in terms of their construction but should only predict • It became standard to base economic models on mechanisms that were known to be unrealistic (e.g. bounded rationality) – thus they did not support good explanations • But then it became common to ‘predict’ out-of- sample data known to the modeller – this is not prediction in the sense described since it is impractical to avoid tuning the model to that data • Thus these models failed in either sense
- 13. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 13 Private and Public Knowledge • It is often valuable to play with models in terms of enriching one’s own intuitive understanding • This may be very exciting to the researcher concerned who may then rush to publication • And, indeed, see the world in terms of their model • However, the kind of knowledge one gets through this is not public knowledge… • ...because it has not been justified that it is useful for anyone else in any particular way • Just because it helps your intuitive understanding does not mean it is worth communicating to others
- 14. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 14 Back to ‘Theory’ • The danger with the word ‘theory’ is that it is hopelessly vague • It can thus allow researchers to dodge the question of the purpose of their models and … • …claim that a model that is not justified against any purpose is somehow ‘theory’ or ‘proto-theory’ • Any old junk can be called ‘theory’, including ideas one gets whilst playing around with models • But things only deserve the status of ‘theory’ when it has proved itself by producing models that work (as assessed against a specified purpose)
- 15. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 15 The ‘Formalist Fallacy’ • That: building and understanding a formal model + some plausibility = a proto-theory • …because this does not tell us anything empirically and can be lead us astray • Humans are very good at deceiving ourselves – finding rationales for thinking that the world fits our conceptions of it • We are prone to the delusion that simple and/or abstract theory is 'roughly correct' just because it is plausible and our minds are limited • Deceiving Ourselves By Simulation (DOBS)
- 16. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 16 The Cure 1. Accept that any project will involve a cluster of related models: an idea, assumptions, data, a simulation maybe some maths etc. etc. 2. But insist that each model (however made) has a purpose – so others know how to judge it 3. Also the relationship between the models 4. Be skeptical about any ‘theory’ without empirical justification or that is vague in nature until it has proved itself by producing useful models 5. If you want generality you have to work for it – ‘brave’ jumps of abstraction won’t get you there
- 17. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 17 NOT ‘brave’ or floating ‘theory’ Very Abstract Model Micro-Evidence Macro-Data Ideas More applied models
- 18. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 18 Staging Abstraction in Gentler Steps Reduced Simulation ModelsReduced Simulation Models Data-Integration Simulation Model Micro-Evidence Macro-Data Reduced Simulation Models Analytic Model Even Simpler Simulation Model Description Data Integration Explanation Theoretical Exploration Analytic Exploration
- 19. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 19 Getting towards generality Empirical Model 1 Empirical Model 2 Empirical Model 3 Generalised Model
- 20. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 20 We can not expect it to be easy… • Although frustrating, there is no guarantee that there are good-enough accessible general theories... • …especially when faced with complex systems! • And particularly when so much is based on empirically weak micro-theories • And when we have no idea which mechanisms are needed for modelling any particular system • It WILL take a LOT of time, but I think hoping for short-cuts or believing vauge theories are roughly right will not help in the long run
- 21. Different Modelling Purposes – building towards theory, Bruce Edmonds, Hannover, July 2018. slide 21 The End! Bruce Edmonds: http://bruce.edmonds.name Centre for Policy Modelling: http://cfpm.org These slides will be at: http://slideshare.net/bruceedmonds A paper on modelling purposes: http://cfpm.org/discussionpapers/192