Adding value to data isaac
Download as PPTX, PDF1 like412 views
This document discusses the OzFlux facility, which operates a network of flux towers across Australia to measure ecosystem exchange of carbon, water, and energy. It describes the data collection process, including instrumentation used and data processing steps. Key points are that OzFlux provides standardized, high quality data through a public online portal using common file formats. It also notes lessons learned about gaining community buy-in for collaborative data sharing practices.
Adding value to data isaac
- 1. Adding Value to Data A Rake’s Progress
- 2. Contents • Anecdotes from A Rake’s Progress • The TERN/OzFlux facility • The OzFlux data path • Non-generic things about OzFlux • Generic things about OzFlux • Conclusions
- 3. TOGA-COARE
- 4. OASIS ‘94 and ‘95
- 7. Surface fluxes Radiation Meteorology Soil properties AusPlots, Supersites Intensive field campaigns Site characteristics Knowledge of ecosystem exchange of carbon, water & energy. Ecosystem dynamics Spatial and temporal dynamics Continental & global budgets Vegetation type Leaf area index Gross primary product Soil moisture Hyperspectral OzFlux Flux tower network AusCover Remote sensing eMAST Land surface models Biomass Soil carbon & nutrients Leaf-level photosynthesis
- 8. The OzFlux Site Data Path u’, v’, w’, T’ q’,c’ WS, WD, T, RH Fsd, Fsu, Fld, Flu Fg, Sws, Tsoil Rain 10 Hz 0.1 Hz Fast (10 Hz raw) Slow (30 minute average) CR3000 data logger Slow (30 minute average) Fast (10 Hz) (optional with ethernet modem) Modmax modem CF card
- 9. OzFlux Data Path Tower data OzFluxQC L3 data netCDF files L4 data netCDF files L5 data L6 data OAI-PMH server OzFlux portal THREDDS server Community developed Python scripts WWW netCDF files PI institution NeCTAR/RDSI
- 10. OzFlux Data Portal • Publicly accessible archive of data • Ability to audit data provided by TERN funded sites • Publication of data via RDA and TDDP – Make the data easy to find • Repository of self-documenting data files – netCDF files conforming to CF Metadata Conventions
- 11. Non-generic things about OzFlux • Data is very homogeneous – Standard suite of instruments – Standard data collection and processing – High level of automation • Existed prior to TERN with a long history of collaboration • Lucky breaks – Standard data processing and portal developed before TERN – Choice of data format compatible with public access to data
- 12. Generic things about OzFlux • Prior to TERN – Individual solutions to data curation – No license applied to data – Data difficult to discover • Constituency – Community of rugged individualists
- 13. Messages for Others • Clear vision of what data curation is achievable by your community – Accept outside help • Solutions need to fit the community – Voluntary compliance is better than enforcement – Guiding is better than directing • Staged solutions to help community accept change – Herding cats
- 14. Communication • Email lists • ASN/OzFlux newsletter • Use of collaborative resources – Trello for scheduling workflow – GitHub for software updates – CloudStor+ for providing data to site Pis – ResearchGate project
- 15. What happens when communication fails? • Establishment of 2 data paths: – Arose through misunderstanding and a lack of formality in process – Compromise allowed co-existence as “research” and “operational” systems – OzFlux will need to make sure boundaries between the 2 systems remain clear inside and outside OzFlux – OzFlux will need to make sure maintaining 2 systems does not adversely affect resources
- 16. Conclusions • Scientists can be lead to water and they will drink if; – They can see there is something in it for them. – Change is managed, staged and suits the community.
Editor's Notes
- #4: Picture of the S/V Malaita Map of TOGA-COARE research area Describe project and data collection and curation Hold up box of 3.5” diskettes Make the point that the data was used once and would be difficult to use again (media may not be readable)
- #5: Photo of HNK Map of OASIS experiment area Description of experiment and use of data 2 papers from aircraft data Data collection and curation Hold up CD-ROM of HNK data
- #6: Not sure about this … Picture of towers Map of NATT tower area Description of data collection and curation Screen shot of data available on OzFlux Data Portal At least part of the data, the data from the network of towers, from this experiment is now available as self-documenting netCDF files available for public download from the OzFlux Data Portal
- #7: Network of 28 sites across Australia, 13 funded by TERN Measurements of the exchange of water and carbon between the Australian biosphere and the atmosphere 80 site-years data available from the portal 64 site-years contributed to the global FluxNet initiative Characterise the climate and biotic drivers of carbon and water exchange Parameterise and validate land-surface models
- #8: OzFlux is one of many facilities within TERN. Complexity of the ecosystem science problems being addressed now requires contributions from multiple facilities across TERN. OzFlux needs to work with other facilities. Establishing a framework for collaboration is difficult and often ad-hoc. Allocating resources to implement the frameworks is also problematic. Clear view of facility and TERN strategic goals are required.
- #9: High level diagram of OzFlux data path from towers
- #10: High level diagram of the OzFlux data quality control and post-processing system QC and post-processing occurs at the site PI institution. Pis upload processed data to the OzFlux data portal Public access to data via the portal Rif-cs files available via the OAI-PMH server for RDA, TDDP and anyone else netCDF files available for download from the OzFlux data portal netCDF files available for remote processing via the THREDDS server
- #11: Need to increase return on tax-payers investment by making data available for multiple uses. Many Pis see the data as their own possession and are reluctant to make it easily available to anonymous users. Many Pis are not convinced of the need for public availibility of their data. Many Pis are unaware of the implications of data licensing.
- #14: TERN’s role in facilitating this process in OzFlux has been extremely important. TERN have provided a knowledge and experience bank that facilities can draw on. In the main, TERN have sought to help and guide facilities rather than force them to adopt solutions. Data curation solutions need to be driven by research needs, this will result in the research community seeing the benefit of the overhead required and increase the chances of voluntary compliance. This means the organisation will not have to dedicate resources to enforcing compliance. Allow people within the community to change at their own pace. Introduction of common license across OzFlux was done only after consultation with the community and was staged. The community asked for a non-commercial license, a fair use data policy and a restricted access provision. All of these were implemented about 18 months ago. It is likely the community will accept less restrictive license model in the near-future.