J konpredictingthefuturefornuclearworkshop v3
0 likes369 views
Jonathan Koomey argues that accurate long-term forecasting of economic, social, and technological systems is impossible due to their inherent lack of structural constancy and the potential for unexpected events. While physical systems can be predicted if variables are known, economic factors are more complex and path dependent. No amount of time, money, or resources can overcome the limitations of modeling non-physical systems far into the future. Experimental and adaptive approaches are better suited than predictions for planning emerging technologies.
J konpredictingthefuturefornuclearworkshop v3
- 1. Past performance is no guide to future returns: What can we really say about the future of economic, social, and technological systems? Jonathan Koomey, Ph.D. Consulting Professor, Stanford University http://www.koomey.com Presented at the UC Berkeley Forum on Nuclear Futures December 10, 2010 Copyright Jonathan G. Koomey 2010 1
- 2. My background • Founded LBNL’s End-Use Forecasting group and led the group for more than 11 years. • Peer reviewed articles and books on – Forecasting methodology – Economics of greenhouse gas mitigation – Critical thinking skills – Information technology and resource use Copyright Jonathan G. Koomey 2010 2
- 3. True or False?: If only we had enough… • Time • Money • Graduate Students • Coffee we could accurately predict the cost of energy technologies in 2030. Copyright Jonathan G. Koomey 2010 3
- 4. Widespread modeling practice implies that the answer is “True” Copyright Jonathan G. Koomey 2010 4
- 5. Based on my experience and reviews of historical retrospectives on forecasting, I say “No way” Copyright Jonathan G. Koomey 2010 5
- 6. Aside: Many of the best modelers acknowledge the difficulties in the pursuit of accurate forecasts, but in their heart of hearts they still believe they can predict accurately with greater effort. Copyright Jonathan G. Koomey 2010 6
- 7. Uncertainty affects even physical systems Es<mates of Planck’s constant "h" over <me. In this physical system researchers repeatedly underes<mated the error in their determina<ons. At each stage uncertain<es existed of which the researchers were unaware. The problem of error es<ma<on is far greater in long‐range energy forecas<ng. Taken from Koomey et al. 2003. Copyright Jonathan G. Koomey 2010 7
- 8. Forecasting Accuracy: The Models Have Done Badly • Energy forecasting models have little or no ability to accurately predict future energy prices and demand (Craig et al. 2002) • Even the sign of the impacts of proposed policies is a function of key assumptions (Repetto and Austin 1997) • The dismal accuracy and inherent limitations of these models should make modelers modest in the conclusions they draw (Decanio 2003) Craig, P., A. Gadgil, and J. Koomey (2002). “What Can History Teach Us? A Retrospective Analysis of Long-term Energy Forecasts for the U.S.” Annual Review of Energy and the Environment 2002. R. H. Socolow, D. Anderson and J. Harte. Palo Alto, CA, Annual Reviews, Inc. (also LBNL-50498). 27: 83-118. Repetto, R. and D. Austin (1997). The Costs of Climate Protection: A Guide for the Perplexed. Washington, DC, World Resources Institute. DeCanio, S. J. (2003). Economic Models of Climate Change: A Critique. Basingstoke, UK, Palgrave-Macmillan. Copyright Jonathan G. Koomey 2010 8
- 9. One example: 1970s projections of year 2000 U.S. primary energy Source: Craig, Paul, Ashok Gadgil, and Jonathan Koomey. 2002. "What Can History Teach Us?: A Retrospective Analysis of Long-term Energy Forecasts for the U.S." In Annual Review of Energy and the Environment 2002. Edited by R. H. Socolow, D. Anderson and J. Harte. Palo Alto, CA: Annual Reviews, Inc. (also LBNL-50498). pp. 83-118. Copyright Jonathan G. Koomey 2010 9
- 10. Another example: Oil price projec3ons by U.S. DOE, AEO 1982 through AEO 2000 Copyright Jonathan G. Koomey 2010 10
- 11. Not any beEer aFer 2000: Oil price projec3ons by U.S. DOE, AEO 2000 through AEO 2007 Copyright Jonathan G. Koomey 2010 11
- 12. Why Are Long-term Energy Forecasts Almost Always Wrong? • Core data and assumptions, which drive results, are based on historical experience, which can be misleading if structural conditions change • The exact timing and character of pivotal events and technology changes cannot be predicted Laitner, J.A., S.J. DeCanio, J.G. Koomey, A.H. Sanstad. (2003) “Room for Improvement: Increasing the Value of Energy Modeling for Policy Analysis.” Utilities Policy, 11, pp. 87-94. Copyright Jonathan G. Koomey 2010 12
- 13. Conditions for Model Accuracy • Hodges and Dewar: models can be accurate when they describe systems that – are observable and permit collection of ample and accurate data – exhibit constancy of structure over time – exhibit constancy across variations in conditions not specified in the model Source: Hodges, James S., and James A. Dewar. 1992. Is it you or your model talking? A framework for model validation. Santa Monica, CA: RAND. ISBN 0-8330-1223-1. Copyright Jonathan G. Koomey 2010 13
- 14. ∑: Accurate forecasts require structural constancy and no surprises Copyright Jonathan G. Koomey 2010 14
- 15. Market structure can change fast Source: Scher and Koomey 2010. Copyright Jonathan G. Koomey 2010 15
- 16. Surprises can be big: U.S. nuclear busbar costs Projected cost range from Tybout 1957 Source: Koomey and Hultman 2007. Assumes 7% real discount rate. Copyright Jonathan G. Koomey 2010 16
- 17. Implications for long-term energy forecasting • Forecasting models describing well-defined physical systems using correct parameters can be accurate because physical laws are geographically and temporally invariant (as long as there are no surprises) • Economic, social, and technological systems do not exhibit the required structural constancy, so models forecasting the future of these systems are doomed to be inaccurate. Four big sources of inconstancy – Pivotal events (like Sept. 11th or the 1970s oil shocks) – Technological innovation – Institutional change – Policy choices Copyright Jonathan G. Koomey 2010 17
- 18. ∑: Economics ≠ Physics Copyright Jonathan G. Koomey 2010 18
- 19. So no matter how many $, coffee cups, months, or graduate students you have, accurate long- run forecasting of technology costs is impossible. Copyright Jonathan G. Koomey 2010 19
- 20. Two senses of the word “impossible”: Practically and Theoretically Either way, the net result is the same: inaccurate forecasts. Copyright Jonathan G. Koomey 2010 20
- 21. So what does this result imply for predictions of the costs of energy technologies? Copyright Jonathan G. Koomey 2010 21
- 22. Some lessons • The world is evolutionary and path dependent – Increasing returns, transaction costs, information asymmetries, bounded rationality, prospect theory – Our actions now affect our options later (so do surprises!) • Experimentation is the order of the day • Use real data to prove results – For nuclear power, we’re in the “show me” stage. Cost projections are no longer enough • Prefer technologies that – are mass produced vs. site-built – have short lead times vs. longer lead times Copyright Jonathan G. Koomey 2010 22
- 23. Nuke costs: here we go again… Source: Koomey and Hultman 2007. Copyright Jonathan G. Koomey 2010 23
- 24. More lessons • Use physical and technological constraints to define bounding cases. Examples: – 2 degrees Celsius warming limit implies a carbon budget, which implies a certain rate of implementation of non-fossil energy sources to avoid worst effects of climate change. – Certain technologies use materials in limited supply (e.g. rare earths). Working backwards from a goal can help identify resource constraints. – Lifetime of power generation technologies and buildings limits penetration of new technologies unless we scrap existing capital Copyright Jonathan G. Koomey 2010 24
- 25. Conclusions • It is impossible to accurately forecast energy technology characteristics because of – structural inconstancy and – pivotal events • Forecasting community has yet to absorb the implications of this insight • To cope we need new ways to think about the future – Experimental (Bayesian) approach to implementation (try many things, fail fast, learn quickly, try again) – Rely on physical and technological constraints to create bounding cases – Embrace path dependence (there is no optimal solution, just lots of possible pathways of roughly similar costs) Copyright Jonathan G. Koomey 2010 25
- 26. Some Key References • Craig, Paul, Ashok Gadgil, and Jonathan Koomey. 2002. "What Can History Teach Us?: A Retrospective Analysis of Long-term Energy Forecasts for the U.S." In Annual Review of Energy and the Environment 2002. Edited by R. H. Socolow, D. Anderson and J. Harte. Palo Alto, CA: Annual Reviews, Inc. pp. 83-118. • Ghanadan, Rebecca, and Jonathan Koomey. 2005. "Using Energy Scenarios to Explore Alternative Energy Pathways in California." Energy Policy. vol. 33, no. 9. June. pp. 1117-1142. • Koomey, Jonathan. 2001. Turning Numbers into Knowledge: Mastering the Art of Problem Solving. Oakland, CA: Analytics Press. (2d Printing, 2004). <http:// www.analyticspress.com> • Koomey, Jonathan. 2002. "From My Perspective: Avoiding "The Big Mistake" in Forecasting Technology Adoption." Technological Forecasting and Social Change. vol. 69, no. 5. June. pp. 511-518. • Koomey, Jonathan G., Paul Craig, Ashok Gadgil, and David Lorenzetti. 2003. "Improving long-range energy modeling: A plea for historical retrospectives." The Energy Journal (also LBNL-52448). vol. 24, no. 4. October. pp. 75-92. • Laitner, J.A., S.J. DeCanio, J.G. Koomey, A.H. Sanstad. (2003) “Room for Improvement: Increasing the Value of Energy Modeling for Policy Analysis.” Utilities Policy, vol. 11, no. 2. June. pp. 87-94. • Scher, Irene, and Jonathan G. Koomey. 2010. "Is Accurate Forecasting of Economic Systems Possible?" Climatic Change (forthcoming). Copyright Jonathan G. Koomey 2010 26
- 27. More Key References • Armstrong, J. Scott, ed. 2001. Principles of Forecasting: A Handbook for Researchers and Practitioners. Norwell, MA: Kluwer Academic Publishers. • Ascher, William. 1978. Forecasting: An Appraisal for Policy Makers and Planners. Baltimore, MD: Johns Hopkins University Press. • Cohn, Steve. 1991. "Paradigm Debates in Nuclear Cost Forecasting." Technological Forecasting and Social Change. vol. 40, no. 2. September. pp. 103-130. • Grubler, Arnulf, Nebojsa Nakicenovic, and David G. Victor. 1999. "Dynamics of energy technologies and global change." Energy Policy. vol. 27, no. 5. May. pp. 247-280. • Hodges, James S., and James A. Dewar. 1992. Is it you or your model talking? A framework for model validation. Santa Monica, CA: RAND. ISBN 0-8330-1223-1. • Huntington, Hillard G. 1994. "Oil Price Forecasting in the 1980s: What Went Wrong?" The Energy Journal. vol. 15, no. 2. pp. 1-22. • Huss, William R. 1985. "Can Electric Utilities Improve Their Forecast Accuracy? The Historical Perspective." In Public Utilities Fortnightly. December 26, 1985. pp. 3-8. • Landsberg, Hans H. 1985. "Energy in Transition: A View from 1960." The Energy Journal. vol. 6, pp. 1-18. • O'Neill, Brian C., and Mausami Desai. 2005. "Accuracy of past projections of U.S. energy consumption." Energy Policy. vol. 33, no. 8. May. pp. 979-993. • Tetlock, Philip E. 2005. Expert Political Judgment: How Good Is It? How Can We Know? Princeton, NJ: Princeton University Press. • Tybout, Richard A. 1957. "The Economics of Nuclear Power." American Economic Review. vol. 47, no. 2. May. pp. 351-360. Copyright Jonathan G. Koomey 2010 27