Four different views of a policy model : an analysis and some suggestions
Download as PPTX, PDF2 likes4,293 views
The document discusses four different views of a policy model, highlighting the divide between researchers and policymakers regarding expectations, complexity, and usability. It emphasizes the importance of participatory modeling, stakeholder involvement, and iterative modeling processes to bridge the gap between empirical truth and instrumental outcomes. The author suggests a multi-level modeling approach, reinforcing that research and policy are fundamentally distinct in their goals and methods.
Four different views of a policy model : an analysis and some suggestions
- 1. Four different views of a policy modelan analysis and some suggestionsBruce EdmondsCentre for Policy ModellingManchester Metropolitan University
- 2. Two WorldsResearchUltimate Goal is Agreement with Observed (Truth)Modeller also has an idea of what the model is and how it worksPolicyUltimate Goal is in Final Outcomes (Usefulness)Decisions justified by a communicable causal storyPolicy AdvisorModeller Policy ModelLabels/Documentation may be different from all of the above!Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 2
- 3. Four Meanings (of the PM)Research WorldThe researcher’s idea/intention for the PMThe fit of the PM with the evidence/dataThe ideavalidation relation extensively discussed within research worldPolicy WorldThe usefulness of the PM for decisionsThe communicable story of the PMThe goalinterpretation relation extensively discussed within policy worldFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 3
- 4. Exemplars of the Divide between Researchers and Policy Makers/AdvisorsPolicy makers try to get researchers to predict the unpredictableResearchers can’t get the input (time, money, data) they need from policy makersPolicy advisors want support for a policy (their best guess that it’s good) and researchers reply with caveats and complicationsPolicy Makers (over) simplify the resultsResearchers make the model so complex nobody but they can understand itFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 4
- 5. Two Models of TruthRealismTruth through relationship of output to observationsFeedback through correspondence (RMSE error, stylized facts, knowledge about processes)ForecastObserve loopOften associated with hard sciences, numbers and analytic modelsInstrumentalismTruth through trying something and assessing how well it worksFeedback through success (cost, votes, pleasure, pain)ActionAssess loopOften associated with the humanities, narratives, politics, perspective and aggregate statisticsFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 5
- 6. The Anti-AnthropocentricAssumptionThat the universe is not arranged for our benefit(as academics trying to understand it)That assumptions as these are likely to be wrong:Our planet is the centre of the universePlanetary orbits are circlesRisky events follow a normal distributionBut here… that the social phenomena we study happen to be such that numerical and other easy ‘surface’ methods of understanding will be sufficientIn other words, we can not side-step the difficulties with some clever proxies/mathematics/statistics/etc.Simplicity (what is easier) is not any guide to truth Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 6
- 7. Consequences of the AAAThere is no reason to suppose that a model that is adequate to modelling policy issues will be simple enough for us to understand(e.g. that it will be analytically solvable)We may well have to abandon hope of genericmodels and settle for context-specific approaches
- 8. We will need to use a combination of different approaches
- 9. We will need to model our models etc. to have a change of understanding themFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 7
- 10. Some modelling tensionsprecision (model not vague)Economic ModelsAgent-basedmodelsgeneralityof scopeScenariosStats/regressionmodelsWanted forpolicy decisionsrealism(reflects knowledge of processes)Reality?Lack of error (accuracy of results)Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 8
- 11. But this is the same as what confronts anyorganism/robot…An adaptive entity faces an environment which is overwhelmingly complex, uncertain, unknown etc.Thus this is a dilemma that AI/Robotics also confronts… whether to make robots:Intelligent, learning knowledge about its world and reasoning (i.e. a realist approach)Or “hard-wired” with some imperfect but effective tricks to enable it to survive and do the essential (i.e. an instrumental approach)Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 9
- 12. AI/ML/Robotics/Adaptive Behaviour etc.They are deeply split about the best approach, with competing traditions that do communicateStarted with reasoning and problem solving, assuming correct knowledge was at hand (a strictly realist limited-context approach)……but in more challenging domains, quick and dirty instrumentalist approaches (following Brooks) seemed to do better but these had limitations and were specific to a certain goalThey have not resolved this issue……but at least the competing approaches are recognised and much discussed.So here are some outcomes from that discussion…Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 10
- 13. 1: the Symbol Grounding ProblemHarnad (1990) Physica DFormal representation can not gain meaning from internal relations with other part of the formal representation In order to attach internal symbols to their meaning one needs a repeated and frequent interaction in contextBy repeatedly changing a model with respect to the available evidence By repeatedly using the model for decision making and seeing the resultsVia frequent participatory methods, involving policy makers/advisors/stakeholders in the modellingFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 11
- 14. 2: Starting “small”Elman (1993) CognitionStart with easy and specific situations and learn about themDoes not matter if these are instrumental and largely wrong, but the incremental building up to more useful domains essentialOnlygeneralising to harder and more general situations slowlyIn other words, a commitment to a continual and lengthy modelling processFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 12
- 15. 3: Multi-Level ModellingBrooks (1991) Artificial IntelligenceHas fast local and specific reactive circuits for control of short-term behaviourWith layers that coordinate these lower level control strategies at progressively higher levelsLower levels continue what they are doing feeding back to higher levelsHigher levels occasionally adjust or interrupt lower level unitsResulted in robots that could navigate rough and unknown domains like insectsFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 13
- 16. A Layered approach to modellingFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 14
- 17. A tighter loop involving stakeholders...Stakeholders are involved in parts of the “modelling” loop: criticising model, providing data, specifying model, determining goal etc.Involvement comes from: relevance to their goals, having some effect/control, quickly seeing the results, feeling involved, not onerous, being situated in their livesThis inevitably means a loss of control by modellers! This is unavoidably political.A radical move: giving this power more directly to people rather than their representativesFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 15
- 18. Two Worlds – EmpiricalUltimate Goal is Agreement with Observed (Truth)Modeller also has an idea of what the model is and how it worksInstrumentalUltimate Goal is in Final Outcomes (Usefulness)Decisions justified by a communicable causal storyTighter loop = participatory modellingModelLabels/Documentation may be different from all of the above!Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 16
- 19. A Multi-Level Vision of PM Feedback on UseEmpirical InputTop-level PMTop-level Policy Advisors, Academics, Official Stats.Top-level Policy MakersContinualDockingPressure groups, Qual. Data, AcademicsRegion Specific PMTopic Specific PMRegional-level Policy Makers, StakeholdersContinualDockingContinualDockingCitizen-level PMCitizen-level PMLocal Stakeholders, activistsCrowd sourcing/input, individuals Citizen-level PMCitizen-level PMFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 17
- 20. Consequences of Multi-Level PMDifferent models being “pulled” in different ways by different groups, inputs and needsContinual re-modelling to keep models ‘docked’ with each other and to incorporate new observed processes (maybe with a distributed ‘wiki’-like structure)A lot of work by stakeholders as well as researchersA lot of data of ALL levels and kinds: textual, anecdotal, network, aggregate statistical, mass data, time-series etc.Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 18
- 21. Main ConclusionsThe Policy/Research divide is deep, not just a matter of inclination, stubbornness or training, but rooted in their different goalsLessons from the AI/Robotics world imply:Need for tight and sustained iteration of modelling processesStart with easier problems, work up laterMulti-level approach, with different levels of model, generalising only when possible and necessaryFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 19
- 22. Auxiliary ConclusionsA need for techniques (human and computer aided) for “translating” qualitative evidence from conversations and texts to the agent rules in simulationsThe need for an ethnographic observation and study of researchers and policy people interactingPossibly writing “manuals” for policy advisors “Researchers: how to deal with them, their advantages and bugs” A similar manual for researchers about policy advisors/makers!Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 20
- 23. The End Bruce Edmondshttp://bruce.edmonds.nameCentre for Policy Modellinghttp://cfpm.orgSlides uploaded to http://slideshare.com
Editor's Notes
- #7: Tale of me and simulations and understanding them