Mixing fat data, simulation and policy - what could possibly go wrong?
Download as PPTX, PDF0 likes231 views
The document discusses the integration of 'fat data', simulation, and policy-making, focusing on the limitations of predictive modeling in complex social systems. It highlights lessons from robotics, weather forecasting, and radiation analysis, emphasizing the effectiveness of quick reactions over predictions. The use of agent-based simulation models is advocated as a means to understand unexpected outcomes and inform policy adjustments, rather than relying solely on probabilistic forecasting.
![Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 55
A Cautionary Tale
• On the 2nd July 1992 Canada’s fisheries minister,
placed a moratorium on all cod fishing off
Newfoundland. That day 30,000 people lost their jobs.
• Scientists and fisheries departments throughout much
of the 1980s estimated a 15% annual rate of growth in
the stock – (figures that were consistently disputed by
inshore fishermen).
• The subsequent Harris Report (1992) said (among
many other things) that: “..scientists, lulled by false
data signals and… overconfident of the validity of
their predictions, failed to recognize the statistical
inadequacies in … [their] model[s] and failed to …
recognize the high risk involved with state-of-stock
advice based on … unreliable data series.”](https://image.slidesharecdn.com/cecan-london-v0-190225111311/85/Mixing-fat-data-simulation-and-policy-what-could-possibly-go-wrong-55-320.jpg)
Mixing fat data, simulation and policy - what could possibly go wrong?
- 1. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 1 Mixing fat data, simulation and policy – what could possibly go wrong? Bruce Edmonds Centre for Policy Modelling Manchester Metropolitan University
- 2. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 2 Motivation and Issues Part 1:
- 3. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 3 Lessons from robotics: Part I Robotics in the 70s and 80s tried to (iteratively): 1. build a map of its situation (i.e. a predictive model) 2. use this model to plan its best action 3. then try to do this action 4. check it was doing OK go back to (1) But this did not work in any realistic situation: • It was far too slow to react to its world • to make useable predictions it had to make too many dodgy assumptions about its world
- 4. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 4 Lessons from robotics: Part II Rodney Brooks (1991) Intelligence without representation. Artificial Intelligence, 47:139–160 A different approach: 1. Sense the world in rich fast ways 2. React to it quickly 3. Use a variety of levels of reaction a. low simple reactive strategies b. switched by progressively higher ones Do not try to predict the world, but react to it quickly This worked much better.
- 5. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 5 Lessons from Weather Forecasting • Taking measurements at a few places and trying to predict what will happen based on simple models based on averages does not work well • Understanding the weather improved with very detailed simulations fed by rich and comprehensive sensing of the system • Even then they recognize that there are more than one possibilities concerning the outcomes (using ensembles of specific outcomes) • If these indicate a risk of severe weather they issue a warning so mitigating measures can be taken even if
- 6. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 6 Lessons from Radiation Levels • The human body is a very complex system • It has long been known that too much radiation can cause severe illness or death in humans • In the 30s & 40s it was assumed there was a “safe” level of radiation • However it was later discovered that any level of radiation carried a risk of illness • Including naturally occurring levels • Although an increase in radiation might not seem to affect many people, it did result in more illnesses in some
- 7. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 7 Prediction in Complex Situations • For many kinds of complex situation or issue prediction (in the sense of correctly anticipating what would happen) is impossible for most desired indicators of outcomes • There are many reasons for this, including: chaos, structural change, multiple assumptions, cascading social effects, context-sensitivity etc. • In particular, this is true of many social systems, where individuals influence each other • This is impossible even probibilistically, that is completely different outcomes may happen
- 8. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 8 Finding the best policy beforehand • A lot of policy making (e.g. the classic process in the Magenta book) relies on being able to anticipate the costs and benefits of policy before they are implemented, to inform the choice • However, in cases where prediction of the impact of policy is impossible, then this approach fails • This includes any optimisation of policy • This talk looks at some of the ways we might approach policy evaluation and monitoring in such situations and the role that data can have in this
- 9. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 9 Kinds of Data Part 2:
- 10. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 10 Randomised Control Trials (RCT) • Tells you whether any measurable indicator is affected by particular interventions or policies with relatively few assumptions • Difficult to fudge due to wishful thinking • Strong at showing a proposed intervention or policy does not work as hoped at the macro level • But it is specific to the set of circumstances it was tried in – it does not tell one why the outcomes happen or whether it will work in other situations • It will not tell you about the enablers (what makes the desired outcome possible) or the frustrators
- 11. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 11 Pure Qualitative Evaluation • Gives rich accounts of what happened, why people did things and their perceptions of outcomes • Useful with stakeholders for collecting a variety of views and evidence which otherwise might be missed • Does not provide a representative sample • Is vulnerable to many human & micro-level biases – noticing what is different, wishful thinking etc. • Difficult to know how to deal this in a policy context
- 12. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 12 Big Data • Can give a very detailed picture of a particular aspect of what is happening • Showing a broad spectrum of responses and actors (but biased to those who leave traces) • Depends upon its availability due to happenstance of sources, privacy, terms of service etc. • Needs extensive pre-processing using complex algorithms to make sense of it • (Usually) limited by the biases inherent in the data • Outcomes usually measured by proxy measures
- 13. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 13 ‘Fat’ data • ‘Fat’ data is when one has multiple data about the same set of events, individuals etc. • Often data can be ‘fattened’ at little extra cost • For example, if one interviewed some of the teachers and students involved in a RCT, so that their responses could be linked to the RCT results, that would be a fattening of that data • Key feature: links data from different levels (micro, meso, macro) and kinds via individual/event IDs • However, it has been difficult to know how to deal with such data
- 14. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 14 A role for simulation models Part 3:
- 15. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 15 Agent-Based Simulation • Is a computer program • Much like a multi-character game, where each social actor is represented by a different “agent” • These agents can each have very different behaviours and characteristics • Social phenomena (such as social networks) can emerge out of the decisions and interaction of these individual agents (upwards “emergence”) • But, at the same time, the behaviour of individuals can be constrained by “downwards” acting rules and social norms from society and peers • Can be complicated – avoids strong assumptions
- 16. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 16 What happens in ABS • Entities in simulation are decided on • Behavioural Rules for each agent specified (e.g. sets of rules like: if this has happened then do this) • Repeatedly evaluated in parallel to see what happens • Outcomes are inspected, graphed, pictured, measured and interpreted in different ways Simulation Representations of OutcomesSpecification (incl. rules)
- 17. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 17 E.G: Schelling’s Segregation Model Schelling, Thomas C. 1971. Dynamic Models of Segregation. Journal of Mathematical Sociology 1:143-186. Rule: each iteration, each dot looks at its 8 neighbours and if less than 30% are the same colour as itself, it moves to a random empty square Segregation can result from wanting only a few neighbours of a like colour
- 18. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 18 Using simulation models to integrate evidence • Agent-based simulation models are particularly good at using a variety of evidence, including: – qualitative micro-level evidence to ensure ‘menu’ of (behaviours is inclusive enough – (if available) meso-level surveys or big data might inform when individuals influence each other and how – quantitative macro-level evidence to validate/check that we are not grossly misleading ourselves • In other words, fat data and simulation analysis are made for each other • However, simulation models are not magic – they do not predict. So what use are they?
- 19. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 19 Micro-Macro Relationships Micro/ Individual data Qualitative, behavioural, social psychological data Theory, narrative accounts Social, economic surveys; Census Macro/ Social data Simulation
- 20. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 20 Using Simulation for Uncertainty Analysis • Agent-based simulation models are good at revealing (and then understanding) otherwise unexpected outcomes • That is showing future possibilities – not probabilities (this is not probabilistic prediction) • In other words, understanding the ways in which policies/interventions could go unexpectantly wrong (or indeed right) • This allows for monitoring for these emergent outcomes to be put in place to give the earliest possible warning to policy actors to adjust policy
- 21. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 21 An Example: Domestic Water Demand Part 4:
- 22. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 22 Social Influence and Domestic Water Demand (SI&DWD) • Produced for the Environment Agency/DEFRA • Part of a bigger project to predict future domestic water demand in the UK given some different future politico-economic scenarios and climate change • The rest of the project were detailed statistical models to do the prediction • This model was to examine the assumptions and look at the envelope of possibilities • Joint work with Olivier Barthelemy and Scott Moss
- 23. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 23 Different kinds of data used • Actual climate data from the last 40 years • Detailed statistics of the usage of different kinds of appliances within the home • Time series data concerning aggregate domestic water consumption for clusters of 200-500 houses • Expert opinion as to the various influences upon householders as to water usage decisions (but not the prevalence of these in the population!)
- 24. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 24 Monthly Water Consumption REL_CHNG .88 .75 .63 .50 .38 .25 .13 0.00 -.13 -.25 -.38 -.50 20 10 0 Std.Dev = .17 Mean = .01 N = 81.00
- 25. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 25 Relative Change in Monthly Consumption Date FEB 2001 SEP 2000 APR 2000 N O V 1999 JU N 1999 JAN 1999 AU G 1998 M AR 1998 O C T 1997 M AY 1997 D EC 1996 JU L 1996 FEB 1996 SEP 1995 APR 1995 N O V 1994 JU N 1994 REL_CHNG 1.0 .8 .6 .4 .2 -.0 -.2 -.4 -.6
- 26. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 26 Model Structure - Overall Structure •Activity •Frequency •Volume Households Policy Agent •Temperature •Rainfall •Sunshine Ground Aggregate Demand
- 27. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 27 Model Structure - Microcomponents • Each household has a variable number of micro- components (power showers etc.): bath other_garden_watering shower hand_dishwashing washing_machine sprinkler clothes_hand_washing hand_dishwashing toilets sprinkler power_shower • Actions are expressed by the frequency and volume of use of each microcomponent • AVF distribution in model calibrated by data from the Three Valleys
- 28. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 28 An Example Social Structure - Global Biased - Locally Biased - Self Biased
- 29. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 29 Policy Agent - Behaviour • After the first month of dry conditions, suggests AFV actions to all households • These actions are then included in the list of those considered by the households • If the household’s weights predispose it, it may decide to adopt these actions • Some other neighbours might imitate these actions etc.
- 30. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 30 Number of consecutive dry months in historical scenario 0 1 2 3 4 5 6 7 8 9 J-73 J-74 J-75 J-76 J-77 J-78 J-79 J-80 J-81 J-82 J-83 J-84 J-85 J-86 J-87 J-88 J-89 J-90 J-91 J-92 J-93 J-94 J-95 J-96 J-97 Simulation Date Numberofconsequativedrymonths
- 31. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 31 Simulated Monthly Water Consumption REL_CHNG .075 .063 .050 .037 .025 .012 -.000 -.013 -.025 -.038 -.050 120 100 80 60 40 20 0 Std. Dev = .01 Mean= -.000 N = 325.00
- 32. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 32 Monthly Water Consumption (again) REL_CHNG .88 .75 .63 .50 .38 .25 .13 0.00 -.13 -.25 -.38 -.50 20 10 0 Std.Dev = .17 Mean = .01 N = 81.00
- 33. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 33 Simulated Change in Monthly Consumption Date SEP 1997 APR 1996 N O V 1994 JU N 1993 JAN 1992 AU G 1990 M AR 1989 O C T 1987 M AY 1986 D EC 1984 JU L 1983 FE B 1982 SEP 1980 APR 1979 N O V 1977 JU N 1976 JAN 1975 AU G 1973 M AR 1972 O C T 1970 REL_CHNG .10 .08 .06 .04 .02 0.00 -.02 -.04 -.06
- 34. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 34 Relative Change in Monthly Consumption (again) Date FEB 2001 SEP 2000 APR 2000 N O V 1999 JU N 1999 JAN 1999 AU G 1998 M AR 1998 O C T 1997 M AY 1997 D EC 1996 JU L 1996 FEB 1996 SEP 1995 APR 1995 N O V 1994 JU N 1994 REL_CHNG 1.0 .8 .6 .4 .2 -.0 -.2 -.4 -.6
- 35. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 35 30% Neigh. biased, historical scenario, historical innov. datesAggregate demand series scaled so 1973=100 0 20 40 60 80 100 120 140 160 180 200 J- 73 J- 74 J- 75 J- 76 J- 77 J- 78 J- 79 J- 80 J- 81 J- 82 J- 83 J- 84 J- 85 J- 86 J- 87 J- 88 J- 89 J- 90 J- 91 J- 92 J- 93 J- 94 J- 95 J- 96 J- 97 Simulation Date RelativeDemand
- 36. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 36 80% Neigh. biased, historical scenario, historical innov. datesAggregate demand series scaled so 1973=100 0 20 40 60 80 100 120 140 160 180 200 J- 73 J- 74 J- 75 J- 76 J- 77 J- 78 J- 79 J- 80 J- 81 J- 82 J- 83 J- 84 J- 85 J- 86 J- 87 J- 88 J- 89 J- 90 J- 91 J- 92 J- 93 J- 94 J- 95 J- 96 J- 97 Simulation Date RelativeDemand
- 37. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 37 80% Neigh. biased, medium-high scenario, historical innov. DatesAggregate demand series scaled so 1973=100 0 20 40 60 80 100 120 140 160 180 200 Jan- 73 Jan- 74 Jan- 75 Jan- 76 Jan- 77 Jan- 78 Jan- 79 Jan- 80 Jan- 81 Jan- 82 Jan- 83 Jan- 84 Jan- 85 Jan- 86 Jan- 87 Jan- 88 Jan- 89 Jan- 90 Jan- 91 Jan- 92 Jan- 93 Jan- 94 Jan- 95 Jan- 96 Jan- 97 Simulation Date RelativeDemand
- 38. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 38 What did the model tell us? • That it is possible that social processes: – can cause a high and unpredictable variety in patterns of demand – can ‘lock-in’ behavioural patterns and partially ‘insulate’ them from outside influence (droughts only occasionally had a permenant affect on patterns of consumption) • The availability of new products could dominate effects from changing consumptions habits • Downing, T.E, et al. (2003). Climate Change and the Demand for Water, Research Report, Stockholm Environment Institute Oxford Office, Oxford. (http://www.sei.se/oxford/ccdew/)
- 39. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 39 Conclusions and Implications Part 5:
- 40. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 40 Summary • In many kinds of situation we can not predict – even approximately. This makes traditional policy evaluation impossible • All kinds of data have their disadvantages singly • ‘Fattened’ data can be used using simulation analysis to understand the possible ways in which a policy/intervention might work out • Monitors (e.g. ‘live’ data visualisations) can be designed based on this understanding to allow policy to be better ‘driven’ and for better-informed contingency planning to done
- 41. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 41 Towards Adaptive Policy • Where impacts of policy/interventions/nudges can be predicted continue as before… • But in other (complex) situations: 1. Collect enriched (‘fat’) data (including from stakeholders) 2. Understand some of the ways a policy could go wrong (maybe using simulation analyses) and implement monitors for these possibilities 3. Adapt policy rapidly based on monitors and knowledge of some of the emergent possibilities
- 42. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 42 The End Manchester Metropolitan University http://mmu.ac.uk Centre for Policy Modelling http://cfpm.org Slides are Available at: http://slideshare.net/BruceEdmonds @
- 43. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 43 Understanding Voter Turnout Additional Example 1:
- 44. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 44 The Social Complexity of Immigration and Diversity was a 5-year project with the Institute for Social Change and the Department of Theoretical Physics at University of Manchester. It was funded under the “Complexity Science for the Real World” initiative of the EPSRC from July 2010 to Jan 2016. The idea of SCID was to apply the techniques and tools of complexity science to real world issues, in this case of immigration and diversity. The example here focuses on the issue why people bother to vote and how the mix of ethnicities and immigration might impact upon this. Copy of Project Website: http://cfpm.org/scid
- 45. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 45 Model ‘layers’ and the different kinds of data used to support these Underlying data about population composition Demographics of people in households Social network formation and maintenance (homophily) Influence via social networks • Political discussions Voting Behaviour
- 46. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 46 Discuss-politics-with person-23 blue expert=false neighbour-network year=10 month=3 Lots-family-discussions year=10 month=2 Etc. Memory Level-of-Political-Interest Age Ethnicity Class Activities AHousehold An Agent’s Memory of Events Etc. Changing personal networks over which social influence occurs Composed of households of individuals initialised from detailed survey data Each agent has a rich variety of individual (heterogeneous) characteristics Including a (fallible) memory of events and influences
- 47. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 47 Evidence schema Qualitative Intuitive understanding Observations of Phenomena Quantitative Models Data Text from interviewsTime Series Data etc. A-B Simulation ‘Causal Stories’ Expert OpinionConclusions
- 48. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 48 Example Output: why do people vote (if they do) Intervention: voter mobilisation Effect: on civic duty norms Effect: on habit- based behaviour Time %ofvotersbyreason
- 49. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 49 Simulated Social Network at 1950 Established immigrants: Irish, WWII Polish etc. Majority: longstanding ethnicities Newer immigrants
- 50. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 50 Simulated Social Network at 2010
- 51. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 51 What did we learn from this? • Previous work claimed that persuading people to vote would have a ‘cascade’ effect, resulting in much more participation – the simulation showed that such a claim might require very strong assumptions • The structure of the social network and immigration can matter (e.g. if immigrants come in as individuals or families) • That the network is dynamic might also matter and affect outcomes
- 52. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 52 Fishing Additional Example 2:
- 53. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 53 Socio-Ecological Systems • Are the combination of human society embedded within an ecological system (SES) • Many social and ecological systems are far too complex to predict • Their combination is doubly complex • E.g. fisheries, deforestation, species extinctions • Yet we still basically use the 1970s robotics “predict and plan” approach to these… • …as if we can plan optimum policies by estimating/projecting future impact
- 54. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 54 Why simple ecological models won’t predict levels of fish • Simpler models do not necessarily get things “roughly” right • Simpler models are not more general • They can also be very deceptive – especially with regards to complex ways things can go wrong • In complex systems the detailed interactions can take outcomes ‘far from equilibrium’ and far from average behaviour • Sometimes, with complex systems, a simple model that relies on strong assumptions can be far worse than having no models at all
- 55. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 55 A Cautionary Tale • On the 2nd July 1992 Canada’s fisheries minister, placed a moratorium on all cod fishing off Newfoundland. That day 30,000 people lost their jobs. • Scientists and fisheries departments throughout much of the 1980s estimated a 15% annual rate of growth in the stock – (figures that were consistently disputed by inshore fishermen). • The subsequent Harris Report (1992) said (among many other things) that: “..scientists, lulled by false data signals and… overconfident of the validity of their predictions, failed to recognize the statistical inadequacies in … [their] model[s] and failed to … recognize the high risk involved with state-of-stock advice based on … unreliable data series.”
- 56. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 56 The Model Allows for a complex, multi-species grid of ecologies to evolve This is highly dynamic, showing a continual changing balance between species There is: movement between patches, complex and dynamic predation between (non-plant) species, competition for resources, slow evolution "A test-bed ecological model”. CoMSES Computational Model Library. Documentation and code available via: http://comses.net
- 57. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 57 This version designed to test possible outcomes of fishing policies • Complex aquatic plant ecology of ~30 species evolved from scratch • Herbivore fish injected into ecology and whole system further evolved (to 40-70 species) • Once a complex ecology with higher-order predators then system is fixed as starting point • Then possible impacts of fishing policies can be examined and understood
- 58. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 58 Total Extinction Prob. & Av. Total Harvest (last 100 ticks) for different catch levels Catch level (per tick) ProportionofMaximum
- 59. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 59 Num Fish (all species, 20 runs) – catch level 25 – each line is a different model run 0 1000 2000 3000 4000 5000 6000 0 31 62 93 124 155 186 217 248 279 310 341 372 403 434 465 496 527 558 589 620 651 682 713 744 775 806 837 868 899 930 961 992
- 60. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 60 Num Fish (all species, 20 runs) – catch level 35 – each line is a different model run 0 1000 2000 3000 4000 5000 6000 0 31 62 93 124 155 186 217 248 279 310 341 372 403 434 465 496 527 558 589 620 651 682 713 744 775 806 837 868 899 930 961 992 Catch target=30
- 61. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 61 Average Number of Species vs. Catch Level (from a different starting ecology) 0 2 4 6 8 10 12 14 0 5 10 15 20 25 30 35 40 Num Species Fish
- 62. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 62 Average Number of Species, Catch=30 0 5 10 15 20 25 30 35 0 200 400 600 800 1000 AverageNumberofSpecies Time "by patches" "uniform"
- 63. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 63 Average Number of Species, Catch=40 0 5 10 15 20 25 30 35 0 200 400 600 800 1000 AverageNumberofSpecies Time "by patches" "uniform"
- 64. Mixing fat data, simulation and policy - what could possibly go wrong?, Bruce Edmonds, CECAN, London, Feb. 2019. slide 64 A risk-analysis approach 1. Give up on estimating future impact or “safe” levels of exploitation 2. Make simulation models that include more of the observed complication and complex interactions 3. Run these lots of times with various scenarios to discover some of the ways in which things can go surprisingly wrong (or surprisingly right) 4. Put in place sensors/measures that would give us the earliest possible warning that these might be occurring in real life 5. React quickly if these warning emerge