Towards the Validation of National Risk Assessments against Historical Observations Using a Bayesian..., Matteo SPADA
1 like1,138 views
The document discusses the validation of a national risk assessment (NRA) using a Bayesian approach, focusing on the Swiss context and specific hazards such as floods and blackouts. It emphasizes the importance of accurate modeling of historical data to estimate risks and consequences effectively, advocating for a probabilistic method that accounts for uncertainties. The findings indicate discrepancies between expert judgments and historical observations, suggesting the proposed validation method can enhance the reliability of NRAs.
Towards the Validation of National Risk Assessments against Historical Observations Using a Bayesian..., Matteo SPADA
- 1. WIR SCHAFFEN WISSEN – HEUTE FÜR MORGEN Towards the validation of a National Risk Assessment against historical observations using a Bayesian approach: Application to the Swiss case Matteo Spada :: Risk Analyst :: Paul Scherrer Institute Peter Burgherr :: Head of the TA Group :: Paul Scherrer Institute Markus Hohl :: Wissenschaftlicher Mitarbeiter Risikogrundlagen :: Federal Office of Civil Protection (FOCP) IDRC 2016, Davos, Switzerland, 28 August – 01 September 2016
- 2. • Motivation • Method • The Swiss National Risk Assessment • Application to selected hazards: Floods Blackouts • Conclusions Outline Page 2
- 3. • National Risk Assessment (NRA) has received increased interest from governments, authorities and other involved stakeholders • NRA approach results in risk matrixes • Likelihood and consequence in risk matrixes are mainly based upon subjective, qualitative judgments or semi-qualitative approaches • The validation of an NRA is of great importance to ensure that the analysis of national hazard scenarios results in the implementation of adequate prevention and mitigation strategies. Motivation Page 3 FOCP (2013)
- 4. Method Page 4 Collect Historical Observations Apply a Threshold Analysis (In order to get a complete dataset for the consequence under interest) Model the dataset with a Bayesian Approach for both frequency and consequences (Bayesian analysis is a fully probabilistic approach intrinsically accounts for both epistemic and aleatory uncertainty. This serves to assess the parameters to be used in the next steps) For a given Hazard, for which we need to estimate the frequency and the related consequences extent:
- 5. A short Overview on Bayesian Analysis Page 5 𝑝 𝜃 𝑦 = 𝐿 𝑦; 𝜃 𝑝(𝜃) 𝐿 𝑦; 𝜃 𝑝 𝜃 𝑑𝜃 𝑝 𝜃 𝑦 = 𝐿 𝑦; 𝜃 𝑝(𝜃) 𝐿 𝑦; 𝜃 𝑝 𝜃 𝑑𝜃 ∝ 𝐿 𝑦; 𝜃 𝑝(𝜃) Prior Bayes Theorem Posterior Data Bayes Theorem: Posterior: Conditional probability that is assigned after the relevant evidence is taken into account. Thus, it expresses our updated knowledge about parameters after observing data Prior: expresses what is known about the parameters before observing the data. Prior describes epistemic uncertainty. Likelihood Function: Likelihood describes the process giving rise to data y in terms of unknown parameters θ. Likelihood describes aleatory uncertainty. Marginal Likelihood: Normalization constant in order to let the posterior distribution to be “proper”. That is, the posterior should converge to 1. By applying Markov Chain Monte-Carlo (MCMC) algorithms
- 6. Method Page 6 Collect Historical Observations Apply a Threshold Analysis Model the dataset with a Bayesian Approach for both frequency and consequences Build the product between the frequency and the Complementary Cumulative Distribution Function (CCDF) of the consequence in order to get the frequency (1/year) of a given extent of the consequence (For the frequency the posterior distribution is considered. The CCDF is built as 1 – CDF, where the CDF is drawn considering the posterior distributions of the parameters describing the consequence under interest) Calculate and compare the Probability in the next 10 years of the consequence extent defined by the experts (The following is used: 𝑃10 𝑦𝑒𝑎𝑟𝑠 = 1 − (1 − 𝑃1 𝑦𝑒𝑎𝑟)10 , where P1 year is estimated from the frequency (1/year) extracted from the product frequency*CCDF) For a given Hazard, for which we need to estimate the frequency and the related consequences extent:
- 7. The Swiss NRA Page 7 • In FOCP (2013) for each hazard 3 possible scenarios are defined based on on events already happened in Switzerland or worldwide: • Significant • Major • Extreme • Major: a scenario of great intensity. Nevertheless, considerably more severe occurrences and courses of events are imaginable in Switzerland. FOCP (2013) FOCP (2013)
- 8. The Swiss NRA Page 8 • For each scenario both consequences, which are 12 subdivided into 4 damage areas (individual, environment, economy, society), and likelihoods are assessed through a DELPHI survey among experts • First the likelihood, which is defined as the probability in the next 10 years that a given event will indeed materialize, has been selected among different classes. • Second, each consequence under interest is selected among different classes • Finally, marginal costs are estimated for each selected consequences in order to aggregate them. Likelihood Class Description Probability in the next 10 years (%) LC8 On average, few events over a human lifespan in Switzerland > 30% LC7 On average, one event over a human lifespan in Switzerland 10-30% LC6 Has occurred in Switzerland before, but possibly already several generations in the past 3-10% LC5 May not have occurred in Switzerland yet, but is known to have happened in other countries 1-3% LC4 Several known events worldwide 0.3-1% LC3 Only few known events worldwide 0.1-0.3% LC2 Only single known events worldwide, but also conceivable in Switzerland 0.03-0.1% LC1 Only single, if any, known events worldwide. Such an occurrence is regarded as very rare even on a global scale, but cannot be fully excluded for Switzerland either. < 0.03% FOCP (2013) Example for Fatalities, FOCP (2013)
- 9. Page 9 Application to Floods FOCP (2013)
- 10. Data Page 10 • LC5 (p = 1-3%): May not have occurred in Switzerland, but is known to have happened in other countries • The Swiss dataset (Switzerland) is build up from data for Switzerland and Neighboring Countries (Italy, France, Germany and Austria) based on the followings: • Swiss Federal Institute for Forest, Snow and Landscape Research (WSL) data for Switzerland from 1972-2013 (1205 events); • Dartmouth Flood Observatory and MunichRe data worldwide from 1982-2010 (3728 events) • Expected consequences: 25 fatalities and 10 Billion CHF economic losses • Red line is the threshold estimated with a MAXC (Maximum Curvature) method • Model: Log-Normal for the consequences Poisson for the frequency
- 11. Results: Model vs. Historical Observations Page 11
- 12. • For fatalities, experts estimate a probability in the next 10 years slightly lower than the model based on historical observations • For economic losses, experts estimate a probability in the next 10 years significantly larger than the model based on historical observations Results: Experts vs. Model Likelihood Page 12
- 13. Page 13 Application to Blackouts FOCP (2013)
- 14. Data Page 14 • LC8 (p = > 30%): On average few events in a Lifetime in Switzerland • ELCOM data for Switzerland from 2010-2013 (26438 events) • No consequence data available Considering duration and number of affected customers defining the Major scenario: • Affected customers: 800000-1.5 Mio people • Duration: 2-4 days with blackout + 2-3 days of getting back to normality • Red line is the threshold estimated with a MAXC (Maximum Curvature) method • Model: Generalized Pareto for Duration Log-Normal for Affected Costumers Poisson for the frequency
- 15. Results: Model vs. Historical Observations Page 15
- 16. Results: Experts vs. Model Likelihood Page 16 • For affected costumers, experts estimate a probability in the next 10 years significantly larger than the model based on historical observations in all cases • For blackout duration, experts estimate a probability in the next 10 years significantly larger than the model based on historical observations in all cases
- 17. • Fully Probabilistic approach, which accounts for both aleatory and epistemic uncertainty, is defined in order to prove the expectation of an event based on expert judgement, commonly used in NRA • For the floods (P= 1-3%), with respect to the historical observation (1972-2013): • Fatality (25) is expected at an average of 7% (4-12%) • Economic losses (10 Billion CHF) is expected at low probability levels (~10-2%) • For blackouts in Switzerland (P= > 30%) data for 2010-2013 were analyzed, and affected customers and duration of blackout used as consequence indicators: • Affected customers (800000 – 1.5 Mio people) are both expected for very low probability levels 10-8% and 10-9%, respectively • Duration (2-7 days) are all expected for low probability levels (10-2%) • The NRA validation method proposed here has been apply to Dangerous Goods Transportation and Windstorms as well (Spada et al., to be resubmitted) • Results show that the consequences not necessarily match the likelihood defined by the experts • Among the validation purpose, the proposed approach could be useful as a complementary approach to support and build more reliable NRAs. Conclusions Page 17
- 18. Page 18 Wir schaffen Wissen – heute für morgen Thank you! Questions? matteo.spada@psi.ch www.psi.ch/ta/mspada
Editor's Notes
- #18: Drawback: it can only be applied to hazard scenarios for which sufficient historical observations are available. Fatalities assessed by the expert are in good agreement with historical data. Economic losses are not. The latterd could be related to the fact that: it is difficult to quantify the possible cost of the damages caused by a hazardous event, due to the large heterogeneity in the infrastructures economic value. As also stated by Ettlin and Bründl from WSL (Report for FOCP), experts could include a more comprehensive estimation of the loss than the one reported in historical data, which might explain the differences in the results between experts’ assessments and models in all cases. In fact, experts could consider loss such as missed revenue of companies due to production downtime, health cost caused by injuries, cost of the intervention of the public authorities (police, fire fighters etc.) and so on, whereas in databases manly direct costs of damages to infrastructures are reported. It would then be interesting to know which cost components dominate the total estimate by the experts (e.g. downtime or production losses)