Meteorological and Aviation Weather Open Data implementation utilising OGC standards
Download as PPTX, PDF1 like1,306 views
The Finnish Meteorological Institute (FMI) opened its meteorological and aviation weather data in 2013, following OGC standards, and provides it in a machine-readable format. The data includes various weather observations, forecasts, and environmental monitoring information with differing publication intervals and accessibility requirements through an API. The FMI data portal complies with INSPIRE requirements and offers structured data models for users, although some users prefer simpler APIs.
![17.7.2015 23
Data Models File size Comparison
81.7
52.9
1.81.3 1.2 0.2
0
10
20
30
40
50
60
70
80
90
Document Size
[MB]
Compressed
DocumentSize
[MB]
Meteorological and Aviation Weather Open Data implementation OGC standards
Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari](https://image.slidesharecdn.com/foss4gfmiopendataroopetervo-150717104122-lva1-app6892/85/Meteorological-and-Aviation-Weather-Open-Data-implementation-utilising-OGC-standards-23-320.jpg)
![17.7.2015 24
Data Models Popularity
Comparison
80
19.8
0.2
0
10
20
30
40
50
60
70
80
90
Downloads[%]
Meteorological and Aviation Weather Open Data implementation OGC standards
Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari](https://image.slidesharecdn.com/foss4gfmiopendataroopetervo-150717104122-lva1-app6892/85/Meteorological-and-Aviation-Weather-Open-Data-implementation-utilising-OGC-standards-24-320.jpg)
Meteorological and Aviation Weather Open Data implementation utilising OGC standards
- 1. Meteorological and Aviation Weather Open Data implementation utilising OGC standards Finnish Meteorological Institute Finnish Meteorological Institute Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 2. Finnish Meteorological Institute opened its data in 2013. Basically everything that FMI has property rights was opened. Data is provided in freely in machine readable format. 17.7.2015 Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari 2 FMI Open Data https://en.ilmatieteenlaitos.fi/open-data
- 3. Data set Description Time Interval Estimated publish date Weather Observations Temperature, Wind, Humidity, Ground Temperature… 10 min Open, older data to be added Sun Radiation UV, Short and Long Term Radiation… 1 min Open Marine Observations Waves, Sea Temperature, Sea Level… 1 h Open Weather Radars Precipitation Rate, Precipitation Amount… 5 min Open, older data to be added Lightning Thunder Strikes in Finland 5 min Open Example of Data Sets 17.7.2015 3Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 4. Example of Data Sets Data set Description Time Interval Estimated publish date Real Time Observations Real Time Observations from specific location(s) AWS 2010 – Soundings 1959 – Flashes 1998 – Sea Level 1971 – Waves 2005 – Open older data will be added Climatological Observations Dayly and monthly temperature mean and extreme values from weather stations 1959 - Open Climatological Observations Monthly temperature and precipitation rate mean values interpolated to grid 1961 - Open Climatological Reference Climatological Reference. Temperature, humidity, pressure, precipitation amount and snow depth. Reference seasons: 1971-2000 1981- 2010 Open 17.7.2015 4Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 5. Example of Data Sets Data set Description Time Interval Estimated publish date Weather forecast model HIRLAM RCR Point forecasts and grid data Latest model run (4 times a day) 0…54 h Open Sea forecast models Sea level point forecasts, Wave (WAM) and current (HBM) as grid data Latest model run (4 times a day) 0...54 h Open Environmental Monitoring Facilities Weather observation stations, radars… 2015 Aviation Observations METAR 30 min open Ground & mast observations Special observations from ground and masts 2016 /Open 17.7.2015 5Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 6. Example of Data Sets Data set Description Time Interval Estimated publish date Air Quality Observations Air Quality Observations 1h 2015-2016 Silam Model Dispersion Model for Air Quality, Forest Fire and Pollen Latest model run (once a day) 0…96h 2015 HELMI Ice Model Ice forecast model Latest model run (4 times a day) 0...54 h open Soundings Temperature, Humidity, Pressure, Wind from ground to 25 km height 2 times a day 2015 Road Weather Observations (LIVI) Road Weather Observations 10 min open 17.7.2015 6Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 7. FMI Open Data Portal follows INSPIRE requirements. FMI Open Data Portal Meta data Data Models Services The very same data portal works as Open Data and INSPIRE portal. 17.7.2015 7Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 8. Catalog Service (CSW) o Based on GeoNetwork View Service (WMS) o Based on GeoServer o Only the most common layers published 17.7.2015 8Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 9. Download Service (WFS 2.0) o Web Feature Service (WFS) 2.0 Simple Profile o Based on stored queries o Predefined data sets with possibility for additional parameters (i.e. time and area) o In-house production 17.7.2015 9Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 10. Registration o Registration is required to use View and Download Services o Working email address is the only mandatory information o After registration the user gets an API key which have to be added into all requests o GET parameter fmi-apikey=…& o Header fmi-apikey; … o Part of url http://wms.fmi.fi/fmi-apikey/…/wms? o One can create several API keys with one email 17.7.2015 10Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 11. Usage Limits With one API key it’s allowed to o do at most 20 000 requests per day to Download Service o do at most 10 000 requests per day to View Service o do at most 600 requests per 5 minutes to both services o If all observations from one time step is calculated to as one, little over 17 000 new data sets are published daily o So, with one API key it’s allowed load everything once o View service can be used for testing but can not be used as a back end for popular clients 17.7.2015 11Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 12. Open Data Service Cluster S1 S2 S3 Client Data Service Cluster S1 S2 S3 Load Balancer Configuration Data (NFS) Configuration (NFS) Database
- 13. MetoLib o Open source JavaScript library produced by Finnish Meteorological Institute o Helps users to load and use the data o Supports multi point coverage data format o Python version is on the list Easy requests Cache Parse the data to as JSON 17.7.2015 13Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 14. Data Models o Observations and point forecasts as GML o The same data is published in: o MultiPointCoverage o MeasurementTimeSeries o SimpleFeature o Gridded data is provided in appropriate binary format (Grib, NetCDF, GeoTiff…) o WFS members contains the metadata ‘envelope’ with a link to a actual data 17.7.2015 14Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 15. Data Models gmlcov:MultiPointCoverage 17.7.2015 15 gml:rangeSet gml:doubleOrNilReasonTupleList The data is listed for every point defined in domain set. gml:domainSet gmlcov:simpleMultiPoint The coverage is defined as a list of points in 4 dimensional grid (lat, lon, height, time). gmlcov:rangeType The parameters listed in range set are defined in separate element. Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 16. Cons - Not intuitive - No natural structure of XML XSLT and Xpath don’t work Pros + Compact + Efficient + Small file size + Works for many data types 17.7.2015 16 Data Models gmlcov:MultiPointCoverage Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 17. Data Models wml2:MeasurementTimeseries 17.7.2015 17 wml2:MeasurementTimeseries One member contains time series for one parameter and one location Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 18. Cons - Lots of repetition - Large file size - Heavy for DOM- based parsers - Don’t work i.e. for thunder strikes Pros + Intuitive + Easy to use + XSLT & XPath works 17.7.2015 18 Data Models wml2:MeasurementTime series Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 19. Data Models SimpleFeature 17.7.2015 19 SimpleFeature One member contains one time, one parameter and one location Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 20. Cons - Lots of repetition - Very large file size - Heavy for DOM- based parsers Pros + Intuitive + Easy to use + XSLT & XPath works + Ready client support 17.7.2015 20 Data Models SimpleFeature Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 21. • Aviation weather reposts are delivered as IWXXM • New data model coming into use in aviation • Consists of the same elements than other messages • om:phenomenonTime, om:procedure, om:featureOfInterest, om:result • Content of the METAR is in om:result part as • extracted into XML elements • original, “old fashion”, METAR string Data Models aviation observations IWXXM / METARS 17.7.2015 21Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 22. Data Type Data Format Observations wml2:MeasurementTimeseries gmlcov:MultiPointCoverage SimpleFeature Point Forecasts wml2:MeasurementTimeseries gmlcov:MultiPointCoverage SimpleFeature Lighting Observations gmlcov:MultiPointCoverage SimpleFeature Grid Forecasts XML Envelope + Grib2/NetCDF Radar Images GeoTiff / PNG images METAR IWXXM 17.7.2015 22Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 23. 17.7.2015 23 Data Models File size Comparison 81.7 52.9 1.81.3 1.2 0.2 0 10 20 30 40 50 60 70 80 90 Document Size [MB] Compressed DocumentSize [MB] Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 24. 17.7.2015 24 Data Models Popularity Comparison 80 19.8 0.2 0 10 20 30 40 50 60 70 80 90 Downloads[%] Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 25. And a little over 430 000 data downloads per day (5 req/s) At the moment about 7200 registered users Some Experiences 17.7.2015 25Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 26. Practically no client supports complex features Although standards are followed, there’s a gap between provided data model and clients’ capabilities Some Experiences 17.7.2015 26Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 27. GeoServer is modified to support stored queries in WFS 2.0 (released in version 2.7) Also simple features had to be supported Some Experiences 17.7.2015 27Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
- 28. Industry is happy to use standardized services Amateur and freelancer coders would prefer simple JSON API Some Experiences 17.7.2015 28Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari