GSIF utilities
0 likes630 views
This document discusses the development of software tools called Global Soil Information Facilities (GSIF) for global soil mapping. It describes existing GSIF components like global soil databases and proposed new modules for tasks like data entry, harmonization, spatial analysis, and visualization. Key proposed software include the Global Soil Mapping package, plotKML for visualization, and the Soil Reference Library package. The document outlines the status of current work and provides next steps like releasing initial packages and continuing development through user feedback. It encourages participation in the GSIF workshop to help develop the software functionality.
GSIF utilities
- 1. Global Soil Information Facilities Software developments Tomislav Hengl ISRIC World Soil Information, Wageningen University GSM2011.org, June 2024th 2011
- 2. Acknowledgement 1. Hannes Reuter GIS and WPS functionality. 2. Pierre Roudier Dylan Beaudette plotKML package. 3. Brendon Malone David Jacquier spline tting function. 4. Keith Shepherd Bob MacMillan Soil Reference Library. GSM2011.org, June 2024th 2011
- 3. GSIF components 1. Cyber infrastructure for input, analysis and visualization of data. 2. Global databases (legacy data, gridded covariates) that are main inputs to global soil mapping. 3. Software tools (modules and packages) and manuals for creation of geoinformation, for instance, according to the GlobalSoilMap.net specications. 4. Standards and protocols for data entry, map generation and data sharing. GSM2011.org, June 2024th 2011
- 4. Book chapter GSM2011.org, June 2024th 2011
- 5. Overview Open Soil Profiles Soil covariates (worldgrids) Global Continental scale Country/state-level Soil variables Soil site info Soil analytical data Descriptive properties 5.6 km repository 1 km repository 250 m repository R packages Map import module Data entry module Harmonization module Spline fitting Spatial analysis module (Servers) cyber infrastructure Data visualization Data export Soil property maps Webmapping API Six+four key soil parameters Global coverage (organic carbon, pH, clay, silt, sand, coarse fragments) Real-time spatial prediction at six standard depths (0-5, 5- (Google Maps) 15, 15-30, 30-60, 60-100, 100- GlobalSoilMap.net functionality 200 cm) for web-applications and with included upper and Geo-serving and geoprocessing lower 95% probability ranges functionality 100 m (250 m, 1 km and 5.6 km) GSM2011.org, June 2024th 2011
- 6. GSIF modules New soil profile New covariates data MAP IMPORT MODULE DATA ENTRY MODULE Upload of gridded maps to the soilprofiles.org: live data soilgrids.org repository; entry forms for point data; HARMONISATION MODULE Meta-data generation tool; Geo-registry module; Translation of laboratory Automated map matching and Automated data screening and methods (correlation) validation (mask maps); detection of gross errors and Upscaling / downscaling Automated conversion and artifacts; functionality harmonization of soil polygon Data translation (re- maps; formatting) functionality SPATIAL ANALYSIS MODULE Overlay (covariates) and regression analysis; Multiscale prediction - trend models; Variogram analysis (automap); Prediction and simulations; Cross-validation; Predict secondary soil Open Soil Profiles Soil Gridded Covariates (soilprofiles.org) parameters (PTF) (soilgrids.org) DATA EXPORT MODULE SUPPORT MODULE Subsetting and export to GIS Help and F.A.Q. data formats (geotiff and ESRI Variable descriptions Shape file), KML and table (meta-data) formats (DBF); Search functionality API services to serve the data (manuals and user without accessing URL (e.g. forums, demos and via mobile-phone); multimedia) Data serving Soil property maps (globalsoilmap.net) GSM2011.org, June 2024th 2011
- 7. Proposed implementation 1. Produce a suite of utilities to import, re-format, analyze and visualize spatial soil data GSM2011.org, June 2024th 2011
- 8. Proposed implementation 1. Produce a suite of utilities to import, re-format, analyze and visualize spatial soil data 2. Design them so they t the needs of operational global soil mapping GSM2011.org, June 2024th 2011
- 9. Proposed implementation 1. Produce a suite of utilities to import, re-format, analyze and visualize spatial soil data 2. Design them so they t the needs of operational global soil mapping 3. Focus on using R+OSGeo GSM2011.org, June 2024th 2011
- 10. Proposed implementation 1. Produce a suite of utilities to import, re-format, analyze and visualize spatial soil data 2. Design them so they t the needs of operational global soil mapping 3. Focus on using R+OSGeo 4. Get the whole DSM community involved (in design, in development, in use) GSM2011.org, June 2024th 2011
- 11. Proposed implementation 1. Produce a suite of utilities to import, re-format, analyze and visualize spatial soil data 2. Design them so they t the needs of operational global soil mapping 3. Focus on using R+OSGeo 4. Get the whole DSM community involved (in design, in development, in use) 5. Provide training in development and use to countries and nodes GSM2011.org, June 2024th 2011
- 12. List of utilities 1. Global soil mapping (core) package GSIF GSM2011.org, June 2024th 2011
- 13. List of utilities 1. Global soil mapping (core) package GSIF 2. Soil visualization package plotKML GSM2011.org, June 2024th 2011
- 14. List of utilities 1. Global soil mapping (core) package GSIF 2. Soil visualization package plotKML 3. Soil Reference Library SRL GSM2011.org, June 2024th 2011
- 15. List of utilities 1. Global soil mapping (core) package GSIF 2. Soil visualization package plotKML 3. Soil Reference Library SRL 4. Geo-services (PythonWPS, Geoserver, RServe, GDAL utilities) GSM2011.org, June 2024th 2011
- 16. Functionality (GSIF) Estimate the spatial domain and the tiling system; Fit splines to soil horizon records and convert from block to point support in vertical dimension; Query and download point and gridded data from the data portals; Convert harmonized soil prole data from relational structure to single table records; Automatically lter suspicious records and detect outliers in the soil prole records; Generate one set of globally consistent predictions using point observations (OSP) and gridded predictors (worldgrids); Convert gridded predictions to formats required for submission to GSIF; Generate metadata and data analysis reports using XML formats; GSM2011.org, June 2024th 2011
- 17. Status (GSIF) Progress so far: Derive cell ID for any location in the world and estimate number of 1degree blocks required to map an area (based on a land mask); Fit equal-area splines to soil prole data (the method of Bishop et al. (1999)); Get values at point locations from worldgrids.org (covariates); Convert site-horizon DB to single-table structure; GSM2011.org, June 2024th 2011
- 18. Functionality (plotKML) Visualize soil proles measurements (using the original soil colors); Visualize soil prole photographs; Plot results of prediction (soil property maps) using standard color schemes; Default distribution model for the GlobalSoilMap.net property maps (?); Visualize uncertainty of the maps; GSM2011.org, June 2024th 2011
- 19. Soil prole GSM2011.org, June 2024th 2011
- 20. Soil prole attribute plot GSM2011.org, June 2024th 2011
- 21. Organic carbon mapped using RK GSM2011.org, June 2024th 2011
- 22. Soil grids as transparent polygons GSM2011.org, June 2024th 2011
- 23. Soil type maps GSM2011.org, June 2024th 2011
- 24. Multiple layers (above each other) GSM2011.org, June 2024th 2011
- 25. Animations GSM2011.org, June 2024th 2011
- 26. Why KML? (1) Google Earth is #1: 350 millions of downloads! GSM2011.org, June 2024th 2011
- 27. Why KML? (2) People that made Google Earth understand statistics GSM2011.org, June 2024th 2011
- 28. plotKML GSM2011.org, June 2024th 2011
- 29. SRL package Harmonization of soil prole data; Estimation of secondary soil properties using pedo-transfer functions; Estimation of soil properties using soil spectroscopy; GSM2011.org, June 2024th 2011
- 30. Overview SOIL REFERENCE Soil Spectral LIBRARY Library Dependent R libraries Soil Reference Data (soil referent profiles with complete Obtain additional laboratory methods, soil description field data soil.spec aqp and scanned soil spectra) soiltexture NO HydroMe Conversion functions Fits (various R packages for generalized Fit conversion requir- linear modeling, fuzzy matching, model parameters ed accur- regression trees etc.) acy? YES Design the Conversion coefficients conversion model (most accurate models to estimate NO standard parameters; extendible) Convers- Estimate values of Unharmonized Standardized value + ion model YES the standardized record (new data) Associated uncertainty available? variable GSM2011.org, June 2024th 2011
- 31. Status It is not dicult to build a package, but to get soil reference data We would need (at least) 300500 points: Points have to be representative (hypercube sampling, the whole world) Each point should be sampled using standard protocol (soil eld description, soil lab analysis, soil spectroscopy) Project designers need to decide if existing samples can be used as well as new ones We probably need new point samples GSM2011.org, June 2024th 2011
- 32. ISRIC monoliths Figure: ISRIC referent samples (monoliths) and occurrence probability. Derived using the MaxEnt package (climatic images, HWSD and vegetation maps). GSM2011.org, June 2024th 2011
- 33. Main principles of programming 1. Hide complexity from the users (scale, eective precision, 3D geostat) 2. Deliver data and results so that no software training is required to open it (KML) 3. Link to R+OSGeo community (do not invent functionality that already exists and is operational) GSM2011.org, June 2024th 2011
- 34. Why R? 1. It is a trustworthy software because it is open source GSM2011.org, June 2024th 2011
- 35. Why R? 1. It is a trustworthy software because it is open source 2. It is accessible to anyone via most of Operating Systems GSM2011.org, June 2024th 2011
- 36. Why R? 1. It is a trustworthy software because it is open source 2. It is accessible to anyone via most of Operating Systems 3. People in developing countries can start picking things up today! GSM2011.org, June 2024th 2011
- 37. Why R? 1. It is a trustworthy software because it is open source 2. It is accessible to anyone via most of Operating Systems 3. People in developing countries can start picking things up today! 4. It is the fastest growing open source environments for statistical computing GSM2011.org, June 2024th 2011
- 38. Why R? 1. It is a trustworthy software because it is open source 2. It is accessible to anyone via most of Operating Systems 3. People in developing countries can start picking things up today! 4. It is the fastest growing open source environments for statistical computing 5. It can handle space-time data GSM2011.org, June 2024th 2011
- 39. Why R? 1. It is a trustworthy software because it is open source 2. It is accessible to anyone via most of Operating Systems 3. People in developing countries can start picking things up today! 4. It is the fastest growing open source environments for statistical computing 5. It can handle space-time data 6. It is professional (made by top-minds in the business) GSM2011.org, June 2024th 2011
- 40. Next steps Release plotKML and GSIF packages v0.1 Continue developing the functionality via R-forge Use users feedback to improve Optimize the processing speed and improve usability on various platforms Incorporate this functionality within WPS GSM2011.org, June 2024th 2011
- 41. Would you like to join GSIF? Join the GSIF workshop on Friday GSM2011.org, June 2024th 2011
- 42. Would you like to join GSIF? Join the GSIF workshop on Friday There are some expectations: GSM2011.org, June 2024th 2011
- 43. Would you like to join GSIF? Join the GSIF workshop on Friday There are some expectations: 1. You take a responsibility to deliver the functionality on time GSM2011.org, June 2024th 2011
- 44. Would you like to join GSIF? Join the GSIF workshop on Friday There are some expectations: 1. You take a responsibility to deliver the functionality on time 2. You share most of the Edzer Pebesma's Open data principles GSM2011.org, June 2024th 2011
- 45. Would you like to join GSIF? Join the GSIF workshop on Friday There are some expectations: 1. You take a responsibility to deliver the functionality on time 2. You share most of the Edzer Pebesma's Open data principles A 3. You should be familiar with R / LTEX GSM2011.org, June 2024th 2011