The Role of Semantics in Harmonizing YOPP Observation and Model Data
0 likes34 views
The document discusses the goals and activities of the Year of Polar Prediction (YOPP) in improving polar prediction models through enhanced observational data from field sites. It describes YOPP's efforts to standardize data collection and model output across sites to facilitate direct comparisons between observations and multiple models. This includes developing common file formats, defining essential climate variables to be collected, and making both observation and model output available through a central data portal. The goals are to evaluate model performance against observations to identify areas for model improvement and advance polar prediction capabilities.
![CSDMS 2.0 Standard Names template
§ Template for creating unambiguous and easily-understood
standard variable names according to a set of rules:
object name + [operation name] + quantity name
§ Examples:
atmosphere_water__liquid_equivalent_precipitation_rate
bedrock_surface__2nd_time_derivative_of__elevation
earth_ellipsoid__equatorial_radius
soil__time_derivative_of__saturated_hydraulic_conductivity
Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 12](https://image.slidesharecdn.com/khalsapdf3yoppsitemipupdate-191125152217/85/The-Role-of-Semantics-in-Harmonizing-YOPP-Observation-and-Model-Data-12-320.jpg)
![NASA EOSDIS Unified Metadata
Model
The proposed syntactic rules are:
(1) in the first position, put the medium/context within which the measurement occurred;
(2) in the second position, put the object that was measured;
(3) in the third position, put the quantity
(4) use _ to separate the words in multi-word terms
(5) use __ to separate the three terms
MeasurementIdentifiers [0..1]
MeasurementIdentifiers/MeasurementContextMedium [1]: atmosphere
MeasurementIdentifiers/MeasurementObject [1]: air
MeasurementIdentifiers/MeasurementQuantity [0..N]: pressure, at_cloud_top
Which can be expressed in a single string as: atmosphere__air__pressure_at_cloud_top](https://image.slidesharecdn.com/khalsapdf3yoppsitemipupdate-191125152217/85/The-Role-of-Semantics-in-Harmonizing-YOPP-Observation-and-Model-Data-13-320.jpg)
![CF Standard Names
§ In widespread use within the climate modeling and broader Earth science
communities (N=4318, v.61 Nov 2018)
§ Construction guidelines
§ [surface] [component] standard_name [at surface] [in medium] [due to process]
[assuming condition]
§ Optional phrases in [], with terms to be substituted in italics
§ Can derive additional names via transformations on other standard names
§ integral_of_Y_wrt_X
§ horizontal_convergence_of_X
§ Where X and Y are base standard names
Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 15](https://image.slidesharecdn.com/khalsapdf3yoppsitemipupdate-191125152217/85/The-Role-of-Semantics-in-Harmonizing-YOPP-Observation-and-Model-Data-15-320.jpg)
![§ InteroperAble Descriptions of Observable Property Terminology WG
§ Seeks to create “a framework for representing observable properties [via]
terminologies [that] accurately encode what was measured, observed,
derived, or computed.”
§ So that terminology developers can formulate local, but globally aligned,
design patterns
§ Incorporating the concept of versioned observable properties (VOP)
and assigning them PIDs in order to foster better citablity and
traceability in an interoperable network of datasets and terminologies.
RDA I-ADOPT Working Group](https://image.slidesharecdn.com/khalsapdf3yoppsitemipupdate-191125152217/85/The-Role-of-Semantics-in-Harmonizing-YOPP-Observation-and-Model-Data-17-320.jpg)
![YOPP Consolidation Phase - MOSAiC
22
• YOPP Targeted Observing Periods
• Will be supersite for YOPPSiteMIP
• MOSAiC Data Policy states:
All variables and parameters must be documented with an attribute name
and attribute definition that provides a human-readable context for the
measurement. ... Any deviations from [the NERC Vocabulary Standard] must
be individually discussed with the MOSAiC data manager. In case a specific
vocabulary is agreed on, a mapping between the NERC vocabulary term and
the term used in the metadata must be provided by the requesting party.](https://image.slidesharecdn.com/khalsapdf3yoppsitemipupdate-191125152217/85/The-Role-of-Semantics-in-Harmonizing-YOPP-Observation-and-Model-Data-22-320.jpg)
The Role of Semantics in Harmonizing YOPP Observation and Model Data
- 1. The Role of Semantics in Harmonizing YOPP Observation and Model Data Siri Jodha S. Khalsa, NSIDC, CIRES, U. Colorado Taneil Uttal, NOAA Leslie Hartten, NOAA National Snow and Ice Data Center Advancing knowledge of Earth’s frozen regions
- 2. The Polar Prediction Project • It aims to promote cooperative international research enabling development of improved weather and environmental prediction services for the polar regions, on time scales from hours to seasonal. A 10-year (2013–2022) endeavour of the WMO’s World Weather Research Programme (WWRP) • Coordinates intensive observation periods, as well as modelling, verification, user-engagement and education activities • Entering consolidation phase: synthesize research, OSEs, verification, operational implementation, data legacy, … The “Year” of Polar Prediction (YOPP) - Flagship activity of PPP
- 3. IASOA (International Arctic Systems for Observing the Atmosphere) Supersites: Suites of instruments measuring variables that lead to process understanding Models: High frequency column output on model levels at supersites MIP: Developed Format and Semantics used for both models and observations promoting multi-model and multi-site process evaluation Data: Available through the YOPP Data Portal (yopp.met.no) Targeted processes: low-level clouds (including phase), stable boundary layers, atmosphere/snow interactions over land and sea ice, coupling procedures (variables and frequencies), ocean mixing, … White Horse Cape Baranova Iqaluit ECCC supersites – soon members of IASOA Cape Baranova – soon member of IASOA MOSAiC drifting station Also: Antarctic, Arctic Ocean and Third Pole sites Models: DWD, ECCC, ECMWF, FMI, MetNorway, MetOffice, NOAA/NCEP, MeteoFrance, Russian Met, CORDEX, CESM, … Arctic sites initial focus
- 4. Goals of the YOPP Site MIP Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 4 Improve numerical models used for polar prediction • Show how an enhanced observational network can lead to better prediction • Use observations from process studies to improve model physics Limitations of model – observation comparisons • Models are imperfect representations of reality • Observations cannot reveal everything that happens in the system • Both have limited temporal and spatial resolution Matching a measurement to output from a numerical model must account for: • Observational error (instrumentation, digitization) • Model error (incorrect assumptions, missing physics, etc.) • Representativeness (sampling,averaging, extrapolations,, etc.) • Point-to-grid comparisons
- 5. Essential coordination between data managers and end users What Data Managers Care About What Scientists Care About What is Needed Objective: Data Stewardship Objective: Research Observations Team Modeling Team YOPPsiteMIP Data Scientists Standards & Protocols F.A.I.R Repositories Services DOI’s Portals MetadataInternal formats Visualization Click Counts Lineage Embargos Time Variables Uncertainties Workflows QC Acknowledgement Access Units Definitions Promote an advanced level of data usability and interoperability to expedite research and predictive services outcomes and foundational legacy data sets • YOPP: develop tool to write Merged Observatory and Merged MOSAiC data files (MODF, MMDF) • MOSAiC: create data stewardship manual to assist scientists in meeting requirements of the MOSAiC Data Agreement
- 6. The IASOA Merged Observatory Data Files (MODF) A unified file format (netcdf with CF conventions, aligned with NWP model output), having standardized quality controls and data processing, which includes all measurements from all sensors, for each observatory.
- 7. YOPP supersite NWP models Major NWP centres are producing time series of high frequency (~ minutes) column output (on model levels) for (a beam of) grid-points at the site, for the physical variables supported by measurements at the observatories. The YOPPsiteMIP NWP time series are available on the YOPP Data Portal at http://thredds.met.no/thredds/catalog/alertness/YOPP_supersite/catalog.html • ECCC-CAPS (Arctic, coupled), available also at http://dd.alpha.meteo.gc.ca/yopp/model_caps • MeteoFrance ARPEGE-MF (Arctic) • MeteoFrance ARPEGE-SH (Antarctic) available also at ftp.umr-cnrm.fr user: yopp, pwd: Arpage • ECMWF-IFS (Global, coupled) available also at https://www.ecmwf.int/en/forecasts/datasets/archive-datasets Coming soon: ECCC-Global, Russian Met, UK MetOffice, DWD, FMI, MetNorway, NOAA/NCEP, CORDEX, CESM, …
- 8. YOPP supersite common model output polarprediction.net ›YOPP Task Teams ›YOPP Modelling Task Team Table 2 Model site-specific output. Tier 2 variables are shaded. Variable name as in CMIP Longer name Unit Notes Single Level fixed variables Sftlf Land area fraction % If applicable, provide information on tiles, and how they are populated for the main model output and the surrounding locations. For each tile provide information on what type of soil and vegetation Orog Surface altitude M Provide information for the main grid output as well as for the surrounding locations, using the WGS84 CRS. Lat Latitude degrees East Lon Longitude degrees North Atmospheric variables on model levels Zg Geopotential height m Provide for both full and half levels if applicable pfull Pressure on full levels Pa phalf Pressure on half levels Pa ua Eastward wind component m s-1 va Northward wind component m s-1 wap Vertical large-scale wind in pressure coordinates Pa s-1 Omega, positive downwards ta Temperature K tdps Dew-point temperature K hus Specific humidity kg kg-1 tnt Tendency of air temperature K s-1 tnta Tendency of air temperature due to advection K s-1 tus Tendency of specific humidity s-1 tusa Tendency of specific humidity due to advection s-1 tnmmutot Tendency of eastward wind m s-2 tnmmvtot Tendency of northward wind m s-2 rlu Upward short-wave radiation W m-2 upperair tendency rld Downward short-wave radiation W m-2 rsu Upward long-wave radiation W m-2 rsd Downward long-wave radiation W m-2 evu Vertical eddy diffusivity coefficient for momentum due to parameterized turbulence m2 s-1 edt Vertical eddy diffusion coefficient for temperature due to parameterized turbulence m2 s-1 wthv Turbulent sensible heat flux based on virtual potential temperature W m-2 GCSS variable name wqv Turbulent moisture flux based on vapor content W m-2 GCSS variable name uw Eastward turbulent momentum flux kg m-1 s-2 GCSS variable name vw Northward turbulent momentum flux kg m-1 s-2 GCSS variable name tke Turbulent kinetic energy m2 s-2 GCSS variable name cl Percentage cloud cover, including both large-scale and convective cloud % clw Mass fraction of cloud liquid water kg kg-1 cli Mass fraction of cloud ice kg kg-1 Single Level atmospheric variables z0m Surface roughness for momentum m No CMIP name z0h Surface roughness for heat m No CMIP name If a different value is used for moisture, please provide that as z0q psl Mean sea level pressure Pa ps Surface pressure Pa uas 10 m eastward wind m s-1 vas 10 m northward wind m s-1 zmla Height of Boundary Layer m Provide description on how it is calculated tas 2m temperature K radiation turbulence near-surface clouds tdps 2m dew point temperature K huss 2m specific humidity kg kg-1 pr Total precipitation kg m-2 s-1 At surface, both liquid and rain prsn Snowfall flux kg m-2 s-1 At surface, all precipitation in solid phase clt Total cloud cover % cod Cloud optical thickness prw Total column water vapour kg m-2 clwvi Total column liquid water kg m-2 clivi Total column icewater kg m-2 vias Horizontal visibility m No CMIP name, CF long name visibility_in_air Surface and TOA variables snd Surface snow thickness m snc Surface snow area fraction % snw Snow water equivalent kg m-2 ts Skin temperature K tsns Snow surface skin temperature K tsnl Snow temperature K Provide vertical grid if more than one layer (as snowlevel) rhos Snow density kg m-3 No CMIP name. Provide vertical grid if more than one layer cnc canopy area fraction 0-1 tgs Surface ground skin temperature K tsl Soil temperature profile K Provide vertical grid if more than one layer (as soillevel) mrlsl Soil moisture profile kg m-2 Provide vertical grid if more than one layer rlut Top-of-atmosphere outgoing long wave radiation W m-2 Follow the CF/CMIP convention that outgoing fluxes are positive upward rsdt Top-of-atmosphere incoming short-wave radiation W m-2 rsut Top-of-atmosphere outgoing short-wave radiation W m-2 rsus Upward surface short- wave radiation W m-2 Follow the CF/CMIP convention that outgoing fluxes are positive upward precip clouds surface rsds Downward surface short- wave radiation W m-2 rlus Upward surface long- wave radiation W m-2 Follow the CF/CMIP convention that outgoing fluxes are positive upward rlds Downward surface long- wave radiation W m-2 hfls Surface turbulence latent heat flux W m-2 hfss Surface turbulence sensible heat flux Wm-2 hfds Surface downward heat flux Wm-2 Ground heat flux hfdsn Surface downward heat flux in snow Wm-2 hfdsnb Downward heat flux at snow botton Wm-2 albs Surface albedo 0-1 albsn snow and ice albedo 0-1 Albedo over snowcovered portion of gridcell tauv Time-average northward turbulence surface stress N m-2 tauu Time-average eastward turbulence surface stress N m-2 For ocean locations only, reported on atmospheric grid Fixed ocean variables thkcello Ocean model cell thickness m Ocean variables on model levels to Ocean temperature K so Sea water salinity The units of salinity are dimensionless and the units attribute should normally be given as 1e-3 or 0.001 i.e. parts per thousand. uo Ocean u-velocity m s-1 vo Ocean v-velocity m s-1 wo Ocean w-velocity m s-1 Ocean single level variables tos Sea surface temperature K mlotst Ocean mixed-layer depth m Defined by sigma T hfsso Atmosphere-ocean sensible heat flux W m-2 hflso Atmosphere-ocean latent heat flux W m-2 radiation ocean rsntds Net downward shortwave radiation at sea water surface W m-2 rlntds Net downward longwave radiation at sea water surface W m-2 wfo Fresh water flux into sea water kg m-2 s-1 fsitherm Water flux into sea water due to sea ice thermodynamic kg m-2 s-1 tauuo Ocean surface x-stress N m-2 Surface downward x Stress tauvo Ocean surface y-stress N m-2 Surface downward y Stress sigwave Significant wave height m No CMIP name Sea ice variables, report on atmospheric grid Quantities refer to the ice-covered fraction portion of the grid-cell only siconc Sea ice concentration (area fraction) % Only report variables if siconc > 0 siitdconc Sea-ice concentration (area fraction) in categories 0-1 sithick Sea ice thickness m siitdthick Sea-ice thickness in thickness categories sisnthick Snow thickness on sea- ice m siage Sea-ice age s siu Sea ice u-velocity m s-1 siv Sea ice v-velocity m s-1 sisali Sea ice salinity g kg-1 sistressave Sea ice normal stress (pressure) Pa sicompstren Compressive sea ice strength Pa m sitemptop Surface temperature (temperature at atmosphere-cryosphere interface) K sitempsnic Temperature at snow-ice interface K sitempbot Temperature at ice- ocean interface K sialb Sea-ice / snow albedo % siflsensupbot Ocean-ice net sensible heat flux W m-2 siflsenstop Net upward sensible heat flux over sea ice W m-2 sifllatstop Net upward latent heat flux over sea ice W m-2 siflswdtop Downwelling shortwave flux over sea ice W m-2 siflswutop Upwelling shortwave flux over sea ice W m-2 sifllwdtop Downwelling longwave flux over sea ice W m-2 sifllwutop Upwelling longwave flux over sea ice W m-2 siflcondtop Net conductive heat flux in ice at the surface W m-2 sipr Rainfall rate over sea ice kg m- ficonc Fast ice concentration (area fraction) 0-1 fithick Fast ice thickness m riconc Ridged ice concentration (area fraction) 0-1 sea-ice
- 9. Aligning NWP and MODF (IASOA + YOPP verif, model+process, data Task Teams) models observatoriesVariable names
- 10. ADC-IARPC- SCADM Vocabularies and Semantics Working Group § Promote awareness of existing vocabularies and semantics initiatives to increase effectiveness and reduce or eliminate redundancy § Coordinate vocabularies and semantics development activities across the polar information community § Enable and organize regular communication within the community § Help members of the community connect to useful and interoperable vocabularies § Inform the polar community about broader activities (e.g. ESIP, RDA), and act as ambassadors from the polar community to other initiatives Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 10 Why Pay Attention to Variable Names? Sharing or comparing data from different sources requires an agreed-to terminology to ensure meanings are clear and unambiguous Model-to-model comparison and model-to-observation validation both rely on common understanding of the variables that were measured/modeled Paying attention to semantics is part of the increasing emphasis on F.A.I.R. data Ideally, vocabularies are developed via a community-consensus process and codified in web-referenceable resources
- 11. Global Change Master Directory § Earth Science Keywords – hierarchical structure where the Variable levels define the measured variables/parameters Keyword Level Example Category Earth Science Topic Atmosphere Term Weather Events Variable Level 1 Subtropical Cyclones Variable Level 2 Subtropical Depression Variable Level 3 Subtropical Depression Track Detailed Variable (Uncontrolled Keyword)
- 12. CSDMS 2.0 Standard Names template § Template for creating unambiguous and easily-understood standard variable names according to a set of rules: object name + [operation name] + quantity name § Examples: atmosphere_water__liquid_equivalent_precipitation_rate bedrock_surface__2nd_time_derivative_of__elevation earth_ellipsoid__equatorial_radius soil__time_derivative_of__saturated_hydraulic_conductivity Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 12
- 13. NASA EOSDIS Unified Metadata Model The proposed syntactic rules are: (1) in the first position, put the medium/context within which the measurement occurred; (2) in the second position, put the object that was measured; (3) in the third position, put the quantity (4) use _ to separate the words in multi-word terms (5) use __ to separate the three terms MeasurementIdentifiers [0..1] MeasurementIdentifiers/MeasurementContextMedium [1]: atmosphere MeasurementIdentifiers/MeasurementObject [1]: air MeasurementIdentifiers/MeasurementQuantity [0..N]: pressure, at_cloud_top Which can be expressed in a single string as: atmosphere__air__pressure_at_cloud_top
- 15. CF Standard Names § In widespread use within the climate modeling and broader Earth science communities (N=4318, v.61 Nov 2018) § Construction guidelines § [surface] [component] standard_name [at surface] [in medium] [due to process] [assuming condition] § Optional phrases in [], with terms to be substituted in italics § Can derive additional names via transformations on other standard names § integral_of_Y_wrt_X § horizontal_convergence_of_X § Where X and Y are base standard names Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 15
- 16. Issues with CF names § CF construction guidelines are open to interpretation and have not been applied consistently § They use domain-specific terminology § “tendency” (858 names) instead of “time derivative” § “anomaly” to mean difference from climatology § Abbreviations (“toa”, “stp”) § They are oriented to model output and not so much to observations § Slow turnaround of new name approval; large number needed by YOPPSiteMIP § Only a few disciplines in the CF governance body Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 16
- 17. § InteroperAble Descriptions of Observable Property Terminology WG § Seeks to create “a framework for representing observable properties [via] terminologies [that] accurately encode what was measured, observed, derived, or computed.” § So that terminology developers can formulate local, but globally aligned, design patterns § Incorporating the concept of versioned observable properties (VOP) and assigning them PIDs in order to foster better citablity and traceability in an interoperable network of datasets and terminologies. RDA I-ADOPT Working Group
- 18. Preliminary Model-Obs Comparisons Jonathan Day, ECMWF Multi-model forecasts of 2m air temperature at Utqiagvik. Observations in black, each colour represents a different forecast (initialised at 00UTC)
- 20. Examples from YOPPSiteMIP Master Table Variable name as in CMIP long_name standard_name (cf) Unit Notes sifllwdtop Downwelling longwave flux over sea ice surface_downwelling_longwave_flux_in_air1 (specify ice fraction in other variable) W m-2 ficonc Fast ice concentration (area fraction) sea_ice_area_fraction (specify ice category in other variable) 0-1 Fast Ice not in CMIP 1From CMIP6 tables Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 20
- 21. ECCC CAPS model § 140 variables, only 85 have standard names, many of those repeated atmosphere_mass_content_of_cloud_liquid_water ?atmosphere mass content of cloud ice water incloud atmosphere_mass_content_of_cloud_liquid_water ?atmosphere mass content of cloud liquid water incloud atmosphere_mass_content_of_cloud_liquid_water ?atmosphere mass content of cloud supercooled liquid water atmosphere_optical_thickness_due_to_cloud ?atmosphere optical thickness due to ice cloud atmosphere_optical_thickness_due_to_cloud ?atmosphere optical thickness due to liquid cloud visibility_in_air ?visibility through liquid fog visibility_in_air ?visibility through rain visibility_in_air ?visibility through snow tendency_of_air_temperature_due_to_convection ?tendency of air temperature due to convection and condensation tendency_of_air_temperature_due_to_diffusion ?tendency of temperature due to vertical diffusion
- 22. YOPP Consolidation Phase - MOSAiC 22 • YOPP Targeted Observing Periods • Will be supersite for YOPPSiteMIP • MOSAiC Data Policy states: All variables and parameters must be documented with an attribute name and attribute definition that provides a human-readable context for the measurement. ... Any deviations from [the NERC Vocabulary Standard] must be individually discussed with the MOSAiC data manager. In case a specific vocabulary is agreed on, a mapping between the NERC vocabulary term and the term used in the metadata must be provided by the requesting party.
- 23. Conclusions § Variable name table is still in progress § Matching CF standard names mostly complete § Still working long names for those variables lacking standard names § CF/ACDD conventions enabled through MODF tool § Modeling and observing groups lack sufficient resources to § Map internal variable names to CF standard names § Create netCDF files with all required metadata § Success with this task will benefit YOPP, as well as the broader science community Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 23
- 24. The YOPP Supersite-Model Intercomparison Project Other Key Participants (data side) • Gunilla Svensson (Stockholm University) • Barbara Casati (ECCC) • Jonny Day (ECMWF) • Sara Morris (NOAA) • Michael Gallagher (NOAA) • Øystein Godøy (Met Norway) • Lara Ferrighi (Met Norway)
- 25. Resources § Polar Prediction WebSite: https://www.polarprediction.net/ § YOPP Data Portal: https://yopp.met.no/ § ASOA: https://www.esrl.noaa.gov/psd/iasoa/ 25
- 26. Thanks for Your Attention! Questions?
- 27. Backup Slides
- 28. Motivation to Improve Forecasting climatological benchmark forecast better worse Zampieri, et al., (2018) GRL Vol. 45, Issue 18, DOI: 10.1029/2018GL079394 ArcticSeaIceEdge
- 29. YOPP Common Model Output Document Table of Variables has 3 names for each Variable Variable name in code (CMIP) long_name (netCDF) standard_name (cf) Unit Notes • Attribute Convention for Data Discovery (ACDD) specifies attributes for describing NetCDF files • The ACDD global attribute standard_name_vocabulary can be used to identify the name and version of the controlled vocabulary from which variable standard names are taken • Values for any standard_name attribute must come from the CF Standard Names to comply with CF long_name A long descriptive name for the variable (not necessarily from a controlled vocabulary). This attribute is recommended by the NetCDF Users Guide, the COARDS convention, and the CF convention. standard_name A long descriptive name for the variable taken from a controlled vocabulary of variable names. This attribute is recommended by the CF convention. units The units of the variable's data values. This attribute value should be a valid udunits string. The "units" attribute is recommended by the NetCDF Users Guide, the COARDS convention, and the CF convention. Polar Data Forum #3, Helsinki, Finland, 18-22 November 2019 29