Kokemuksia tiedon avaamisesta, Tarja Riihisaari
1 like1,489 views
The Finnish Meteorological Institute (FMI) opened its data in 2013, making everything it owns freely available in machine-readable formats. The FMI Open Data portal follows INSPIRE requirements and works as both an open data and INSPIRE portal. It provides meteorological and environmental observations, forecasts, and other data through catalog, view, and download services. While the open data project has increased access and reuse of FMI's data, some challenges remain regarding supporting complex data types and meeting varied user needs.
Kokemuksia tiedon avaamisesta, Tarja Riihisaari
- 1. Open Meteorological Data with OGC and INSPIRE Avoin tieto ja tilastotoimi 02/2015 Finnish Meteorological Institute Roope Tervo, Mikko Visa, Tarja Riihisaari
- 2. Finnish Meteorological Institute opened its data in 2013 Basically everything FMI owns was opened Data is provided in freely in machine readable format 3.2.2015 Avoin tieto ja tilastotoimi 2 FMI Open Data https://en.ilmatieteenlaitos.fi/open-data
- 3. FMI Open Data Portal follows INSPIRE requirements. 3.2.2015 3 FMI Open Data Portal Meta data Data Models Services The very same data portal works as Open Data and INSPIRE portal. Avoin tieto ja tilastotoimi
- 4. Data set Description Time Interval Estimated publish date Weather Observations Temperature, Wind, Humidity, Ground Temperature… 10 min Open, older data to be added Sun Radiation UV, Short and Long Term Radiation… 1 min Open Marine Observations Waves, Sea Temperature, Sea Level… 1 h Open Weather Radars Precipitation Rate, Precipitation Amount… 5 min Open, older data to be added Lightning Thunder Strikes in Finland 5 min Open 3.2.2015 4 Example of Data Sets Avoin tieto ja tilastotoimi
- 5. 3.2.2015 5 Example of Data Sets Data set Description Time Interval Estimated publish date Real Time Observations Real Time Observations from specific location(s) AWS 2010 – Soundings 1959 – Flashes 1998 – Sea Level 1971 – Waves 2005 – Open older data will be added Climatological Observations Dayly and monthly temperature mean and extreme values from weather stations 1959 - Open Climatological Observations Monthly temperature and precipitation rate mean values interpolated to grid 1961 - Open Climatological Reference Climatological Reference. Temperature, humidity, pressure, precipitation amount and snow depth. Reference seasons: 1971-2000 1981- 2010 Open Avoin tieto ja tilastotoimi
- 6. 3.2.2015 6 Example of Data Sets Data set Description Time Interval Estimated publish date Weather forecast model HIRLAM RCR Point forecasts and grid data Latest model run (4 times a day) 0…54 h Open Sea forecast models Sea level point forecasts, Wave (WAM) and current (HBM) as grid data Latest model run (4 times a day) 0...54 h Open Environmental Monitoring Facilities Weather observation stations, radars… 2014 Aviation Observations METAR 30 min 2014 Ground & mast observations Special observations from ground and masts 2014 Avoin tieto ja tilastotoimi
- 7. 3.2.2015 7 Example of Data Sets Data set Description Time Interval Estimated publish date Air Quality Observations Air Quality Observations 1h 2015-2016 Silam Model Dispersion Model for Air Quality, Forest Fire and Pollen Latest model run (once a day) 0…96h 2014-2015 HELMI Ice Model Ice forecast model Latest model run (4 times a day) 0...54 h 2014 Soundings Temperature, Humidity, Pressure, Wind from ground to 25 km height 2 times a day 2014 Avoin tieto ja tilastotoimi
- 8. Catalog Service (CSW) o Based on GeoNetwork 3.2.2015 8 View Service (WMS) o Based on GeoServer o Only the most common layers published Avoin tieto ja tilastotoimi
- 9. Download Service (WFS 2.0) o Web Feature Service (WFS) 2.0 Simple Profile o Based on stored queries o Predefined data sets with possibility for additional parameters (i.e. time and area) o In-house production 3.2.2015 9Avoin tieto ja tilastotoimi
- 10. 3.2.2015 10 MetoLib o Open source JavaScript library produced by Finnish Meteorological Institute o Helps users to load and use the data o Supports multi point coverage data format o Python version is on the list Easy requests Cache Parse the data to as JSON Avoin tieto ja tilastotoimi
- 11. 3.2.2015 11 Registration o Registration is required to use View and Download Services o Working email address is the only mandatory information o After registration the user gets an API key which have to be added into all requests o POST field fmi-apikey=… o GET parameter fmi-apikey=…& o Header fmi-apikey; … o Part of url http://wms.fmi.fi/fmi-apikey/…/wms? o One can create several API keys with one email Avoin tieto ja tilastotoimi
- 12. 3.2.2015 12 Usage Limits With one API key it’s allowed to o do at most 20 000 requests per day to Download Service o do at most 10 000 requests per day to View Service o do at most 600 requests per 5 minutes to both services o If all observations from one time step is calculated to as one, little over 17 000 new data sets are published daily o So, with one API key it’s allowed load everything once o View service can be used for testing but can not be used as a back end for popular clients Avoin tieto ja tilastotoimi
- 13. Help Desk: Questions about open data can be sent on the feedback form available on our website, e-mail avoin-data@fmi.fi or through Facebook www.facebook.com/fmibeta Help desk usage 2014/2015 approx. 10-15 requests/week 3.2.2015 13 FMI Open Data logo & Help Desk Avoin tieto ja tilastotoimi
- 14. Few examples of Applications: •Aaltopoiju.fi www.aaltopoiju.fi (Apps4Finland 2014) •FMI-asemien havainnot ja historiatietoa vuodesta 1959 - http://suja.kapsi.fi/fmi-suomi.php http://suja.kapsi.fi/fmi-tilastot.php R-liberary FMI data https://github.com/rOpenGov/fmi ….. Workshops: FMI has organized couple of Workshops for Developers approx. 60 participants FMI participant in Apps4Finland 3.2.2015 14 Applications and Workshops Avoin tieto ja tilastotoimi
- 15. And a little over 300 000 (2014 summer 100000) data downloads per day (3,7 req/s, 2014 summer 1,2 req/s) At the moment about 7200 (2014 summer 5000) registered users 3.2.2015 15 Some Experiences Avoin tieto ja tilastotoimi
- 16. Practically no client supports complex features There’s a gap between provided data model and clients’ capabilities 3.2.2015 16 Some Experiences Avoin tieto ja tilastotoimi
- 17. Industry is happy to use standardized services Amateur and freelancer coders would prefer simple JSON API 3.2.2015 17 Some Experiences Avoin tieto ja tilastotoimi
- 18. So far maybe even more professional interest than private Quite many expected a user interface to load data to i.e. to Excel instead of machine readable interface 3.2.2015 18 Some Experiences Avoin tieto ja tilastotoimi
- 19. For now, very few have been interested in forecast models as a grid data Point forecasts, observations and radar images are the most interesting data types 3.2.2015 19 Some Experiences Avoin tieto ja tilastotoimi
- 20. Finally we have all the data behind one access point behind standard interfaces and in harmonized formats. The Open Data project has been a great boost for ourselves both technically and for working culture. 3.2.2015 20 Some Experiences Avoin tieto ja tilastotoimi
- 21. www.fmi.fi