Python the lingua franca of FEWS

Editor's Notes

• #2: What is this presentation about? In essence it is the presentation that I had seen several years ago. Many presentations are about the what, but not many are about the ‘how’. I have had a go at solving a few problems that have cropped up. I am no expert configuration guru nor am I formally trained in programming. I have just hammered away problems and found solutions that I would like to share with the rest of you. Everything I am showing is available on GitHub and I am more than happy to talk it through with anyone wanting more detail. If I can inspire one person in this audience to have a go at coding either in Python or Delft FEWS then it was success. A few years ago I began to realise during FEWS training that the days of sloppy spreadsheets and manual translation of model input/outputs was not the right way to attack all of this ensemble /monte carlo modelling. Last year I presented I on Data Fabriek.
• #3: The inspiration from this presentation came from reflecting on why did I start doing so much coding. I sat in Audience in 2016 and listened to Antonio discuss coupling of FEWS to various models and then chaining them together in a workflow. About the same time I was involved in the Brisbane River Flood Study where URBS monte carlo models had been done. I had no idea how all of that was happening, it was an itch I had to scratch. First it started with a simple python script that linked URBS csv straight into Tuflow boundaries. Then it became For loop that scrapped all of the ensemble results, then it went to plotting. It was a rabbit hole that I disappeared down. I found it very satisfying and rewarding, with a little persistence I started figuring out how to do more and more.
• #5: My colleagues, Steve and Dave presented on the work they have done using FEWS with Design Hydrology. This work is also inspired by cracking the same problem – Australian Rainfall and Runoff. I am confident that Steve convinced many of you that my using FEWS to aggregate data is possible. I’d like to think that his presentation bookends my presentation from last year of a possible vision for FEWS outside of just Forecasting.
• #6: So the Presentation is the HOW you can do this. I want you to leave here today knowing that you have all the tools to get started to stitch a model into FEWS. Have greater flexibility, stay on the bleeding edge, learn something. You might have simple models and spreadsheets and wonder why change, alternatively you may be interested in expanding the scope of your system – this will allow you to scale it up. The future is probabilistic models. Handling all of that data is going to need new tools Try a new feature, calibrate/compare models
• #7: Known output probabilistic determination of possible inputs Hamiltonian PY mc3 or Stan sumsar.net Rasmuss Baath youtube PyMC3 strives to make Bayesian modeling as simple and painless as possible, allowing users to focus on their scientific problem, rather than on the methods used to solve it. Here is a partial list of its features: Modern methods for fitting Bayesian models, including MCMC and VI. Includes a large suite of well-documented statistical distributions. Uses Theano as the computational backend, allowing for fast expression evaluation, automatic gradient calculation, and GPU computing. Built-in support for Gaussian process modeling. Model summarization and plotting. Model checking and convergence detection. Extensible: easily incorporates custom step methods and unusual probability distributions. Bayesian models can be embedded in larger programs, and results can be analyzed with the full power of Python.
• #8: Highlights Top Five Weblinks Things to Take Cuisine Do’s and Don’ts Resources Essential Information Exploring/Orientation Price Guide Phrase Book Itineraries Don’t Miss Best Places to Visit Guide Map Walking Tour Trivia
• #11: I am going to gloss over the detail of configuration course, but essentially the one file you need to know about is the General Adapter. It lives in the Module Config Files In this file, you can layout your executables – this case batch file that calls Python pre/post adapter or your Model. Other housekeeping can be accomplished in this XML file. The model and its binary are zipped up and live in the Module Data Set files. The zip contains all of the batch and your running model files.
• #14: Simple tasks can be accomplished using batch command syntax. More complicated steps you can use Powershell, R, Python Anaconda takes care of a lot of the necessary steps Python from
• #16: One of the biggest frustrations with Python is that you have some code, but it won’t run because you have an error message about a missing library. Arrgh - what to do. It is quite simple, ICT willing, it should just be matter of pulling the latest library from the repository