The Power of the Cloud, and Global Risk Modelling in the Open
Download as PPTX, PDF0 likes341 views
The document discusses using cloud computing for global risk modeling and earthquake modeling. It describes challenges like working with large, sensitive datasets from different sources and ensuring calculations and results are verifiable. The solution proposed is using an open source cloud platform that can elastically scale to thousands of nodes to run complex modeling as a repeatable service. Lessons discussed include the importance of full data sharing and open science principles to advance global modeling efforts.
The Power of the Cloud, and Global Risk Modelling in the Open
- 1. There is a lot that happens around the world we cannot control. We cannot stop earthquakes, we cannot prevent droughts, and we cannot prevent all conflict, but when we know where the hungry, the homeless and the sick exist, then we can help. - Jan Schakowsky
- 4. THE POWER OF THE CLOUD, AND GLOBAL RISK MODELING IN THE OPEN
- 5. Me: Joshua McKenty Twitter: @jmckenty Email: joshua@pistoncloud.com IT Architect, Global Earthquake Model Chief Architect, NASA Nebula Founding Member, OpenStack OpenStack Project Policy Board CEO, Piston Cloud Computing
- 7. WHAT IS THE GLOBAL EARTHQUAKE MODEL?
- 9. “The GEM Foundation has set itself out to engage a global community in the design, development and deployment of state-of- the-art models and tools for earthquake risk assessment worldwide.”
- 10. An introduction to Risk Modeling
- 11. Reports…
- 12. …and Tools
- 13. CLOUD FOR RISK MODELING?
- 15. Challenges » Complex calculations on large, federated data sets » Data is sensitive, sometimes secret, often proprietary » Process and calculations used need to be certified and verifiable » Results need to be public
- 16. A scientifically valid “mash-up”
- 17. Solution: Cloud » Built to scale elastically to the extents of the infrastructure (1-10,000 nodes) » Run as a service (OpenGEM) » Open source » … and repeatable.
- 19. All i have is a hammer…
- 20. LESSONS LEARNED
- 21. Data Peers Experiments
- 22. "International programs for global change research and environmental monitoring crucially depend on the principle of full and open data exchange" - On the Full and Open Exchange of Scientific Data (A publication of the Committee on Geophysical and Environmental Data - National Research Council) 1995
- 23. …the Internet was conceived as a communication mechanism for the dissemination of ideas and as a means to support distributed collaboration. - Open Source Software Development and Distributed Innovation, Bruce Kogut and Anca Metiu, April 2001
- 24. "...without access to the source for the programs we use... (i.e. when simulation codes or parameter files are proprietary or are hidden by their owners), numerical experimentation isn’t even science. Science has to be 'verifiable in practice' as well as 'verifiable in principle’”
- 26. “...Michael Faraday’s advice to his junior colleague to: “Work. Finish. Publish.” needs to be revised. It shouldn’t be enough to publish a paper anymore. If we want open science to flourish, we should raise our expectations to: “Work. Finish. Publish. Release.” That is, your research shouldn’t be considered complete until the data and meta-data is put up on the web for other people to use, until the code is documented and released… - Dan Gezelter
- 27. Science made real, again Open Data Open Open Science Stack
Editor's Notes
- #3: How many of you have been in an earthquake?How many of you died?The difference between an earthquake killing 40% of the population, and an earthquake killing 0.01% of the population, is in our understanding of risk.
- #5: We’re all clouds
- #6: Who am I - I'm a one-trick pony (but it's a hell of a trick) - Open Source, distributed systems at scale - Data analytics (Netscape, AOL) - NASA and OpenStack
- #7: Where we’re headed:What is GEMHow is this Cloud?What did we learn?
- #9: What is GEM - Global Earthquake Modelling - A social problem, a technical problem, and a user experience problem
- #11: Risk modeling is the same in all fields – financial risk, political risk, disaster risk.But when it really matters, it sharpens the mind.
- #12: We wanted to move from the old model (static products)…
- #13: to the new model (dynamic systems).
- #14: In order to answer the “Why Cloud” – we have to look at what makes GEM hard…
- #15: Monte Carlo – 10,000 realizations for a decent result
- #17: All of the challenges of a web application at scale, plus - repeatability
- #23: http://en.wikipedia.org/wiki/Open_science_data
- #24: http://knowledge-stage.wharton.upenn.edu/papers/1252.pdfOpen Source Software Development and Distributed Innovation
- #26: Everyone has a big enough computerEveryone can reproduce the resultsEveryone can use the same methods with additional datasets
- #27: http://www.openscience.org/blog/?p=269 July 28, 2009 by Dan Gezelter, and until the comments start coming in to your blog post announcing the paper. If our general expectations of what it means to complete a project are raised to this level, the scientific community will start doing these activities as a matter of course."
- #28: There was a point when the internet was no longer a tool for doing things BETTER – it was a tool for doing entirely NEW THINGS.CLOUD is reaching that point. And I believe that 50 years from now, we’ll be able to look back and say that OpenQuake was one of those things.