Data preparation, depth function

FAO- Global Soil
Partnership
Training on
Digital Soil Organic Carbon
Mapping
20-24 January 2018
Tehran/Iran
Yusuf YIGINI, PhD - FAO, Land and Water Division (CBL)
Guillermo Federico Olmedo, PhD - FAO, Land and Water Division (CBL)

SOIL Databases
Soil Profile Database and Topsoil Database

Scenario 1: SOIL Profile DB
Most commonly, soil are described as vertical
profiles using soil pits . Soil profiles are described
using macro-morphological properties. These
properties can be assessed in the field without
analysis by making a field inventory or land
evaluation. For additional quantitative analysis,
soils are then sampled by genetic horizon or by
depth class.

SOIL Profile DB
In order to process and evaluate soil information
from field assessments, soil profile and analytical
information needs to be stored in a database. This
can be a set of simple Excel Spreadsheets, or a
relational or object-oriented database
management system. When working in R,
SoilProfileCollections from the R ‘aqp’ package
could be a useful tool.

Data Organisation
Table-1: SITE LEVEL DATA
stores site-level data, which describe the location of
the soil description and/or sampling site: spatial
coordinates, landscape attributes such as slope
gradient and slope form, soil class, land cover type,
rock type etc. In this table every row should hold a
single soil profile. One column, usually the first one,
should be the soil profile’s unique identifier. Using
the latter, soil information can be easily linked from
one table to another.

Data Organisation
TABLE-2: Horizon Level Data
stores information from the soil description, such as
horizon name, horizon thickness, organic matter content,
carbonate content, soil color, laboratory soil analysis, etc.
The first column contains the soil profile’s unique
identifier. It is important to include the upper and lower
limits for each soil layer; in case the sampling strategy
deviates from soil layers/soil horizons, the upper and
lower depth of the sampling locations should be specified
if possible. This information is needed for modelling soil
properties over the soil profile.

Soil Profile DB
dat <- read.csv(file = "data/horizons.csv")
# Explore the data
str(dat)
## 'data.frame': 10292 obs. of 10 variables:
## $ ProfID: Factor w/ 4118 levels "P0000","P0001",..: 1 1 1 2 2 ..
## $ HorID : Factor w/ 9914 levels "P0000H01","P0000H02",..: 1 2 ..
## $ top : int 4 23 46 2 11 0 22 63 0 3 ...
## $ bottom: int 23 46 59 11 31 22 63 90 19 10 ...
## $ SOC : num NA NA NA NA NA ...
## $ BLD : num NA NA NA NA NA NA NA NA NA NA ...
## $ CRF : num 54 62 47 66 70 57 77 87 8 4 ...
## $ SAND : int 52 59 67 45 40 52 48 43 50 48 ...
## $ SILT : num 34 31 24 39 31 33 36 42 16 35 ...
## $ CLAY : num 14 11 8 16 28 15 16 16 34 17 ...

Soil Profile DB
dat <- read.csv(file = "data/horizons.csv")
# Explore the data
summary(dat)
## ProfID HorID top
## P2881 : 64 P2881H01: 64 Min. : 0.00
## P1481 : 32 P0434H02: 8 1st Qu.: 0.00
## P2096 : 32 P1286H01: 8 Median : 20.00
## P3623 : 32 P2056H01: 8 Mean : 27.48
## P2056 : 24 P2056H02: 8 3rd Qu.: 47.00
## P2142 : 24 P2056H03: 8 Max. :285.00
## (Other):10084 (Other) :10188
## bottom SOC BLD
## Min. : 1.00 Min. : 0.000 Min. :0.00
## 1st Qu.: 25.00 1st Qu.: 1.090 1st Qu.:1.40
## Median : 45.00 Median : 1.800 Median :1.54
## Mean : 55.82 Mean : 2.635 Mean :1.55
## 3rd Qu.: 80.00 3rd Qu.: 2.940 3rd Qu.:1.66
## Max. :295.00 Max. :306.000 Max. :2.93
## NA's :831 NA's :7845

Soil Profile DB
dat_sites <- read.csv(file = "data/site-level.csv")
# Explore the data
str(dat_sites)
## 'data.frame': 4118 obs. of 6 variables:
## $ X.1 : int 1 2 3 4 5 6 7 8 9 10 ...
## $ ProfID : Factor w/ 4118 levels "P0000","P0001",..: 1 2 3 ..
## $ soiltype : Factor w/ 58 levels "Albic Luvisol",..: 3 3 3 57..
## $ Land.Cover: int 25 24 25 26 26 27 27 27 22 23 ...
## $ X : num 20.8 20.8 20.8 20.8 20.8 ...
## $ Y : num 42 42 42 42 42 ...

Soil Profile DB
dat_sites <- read.csv(file = "data/site-level.csv")
# Explore the data
summary(dat_sites)
X.1 ProfID
Min. : 1 P0000 : 1
1st Qu.:1030 P0001 : 1
Median :2060 P0002 : 1
Mean :2060 P0003 : 1
3rd Qu.:3089 P0004 : 1
Max. :4118 P0005 : 1
(Other):4112
soiltype Land.Cover
Fluvisol :1206 Min. : 2.00
Mollic Fluvisol : 330 1st Qu.:12.00
Cambisol : 301 Median :19.00
Complex of Solonchak and Solonetz: 151 Mean :18.15
Mollic Vertic Gleysol : 150 3rd Qu.:23.00
(Other) :1917 Max. :41.00
NA's : 63
...

Completeness of data
to calculate soil organic carbon stocks,
# Estimate Organic Carbon Stock
# SOC (g.kg-1)
# BD ( kg/m3_
# Coarse fragments (%)

Bulk Density (BD)
BD=1.62−0.06∗OM - Saini (1966)
BD=1/(0.6268+0.0361∗OM) - Drew (1973)
BD=1.482−0.6786∗(logOM) - Jeffrey (1979)
BD=0.669+0.941∗e(−0,06∗OM) - Grigal et. al (1989)
BD=100/(OM/0.244+(100−OM))/MBD - Adams (1973)
BD=1/(0.564+0.0556∗OM)- Honeysett & Ratkowsky (1989)
MDB is the Mineral particle density,

R Function - BD Estimation
# Creating a function in R to estimate BLD using the SOC
# SOC is the soil organic carbon content in %
estimateBD <- function(SOC, method="Saini1996"){
OM <- SOC * 1.724
if(method=="Saini1996"){BD <- 1.62 - 0.06 * OM}
if(method=="Drew1973"){BD <- 1 / (0.6268 + 0.0361 * OM)}
if(method=="Jeffrey1979"){BD <- 1.482 - 0.6786 * (log(OM))}
if(method=="Grigal1989"){BD <- 0.669 + 0.941 * exp(1)^(-0.06 * OM)}
if(method=="Adams1973"){BD <- 100 / (OM /0.244 + (100 - OM)/2.65)}
if(method=="Honeyset_Ratkowsky1989"){BD <- 1/(0.564 + 0.0556 * OM)}
return(BD)
}

BD Estimation
# See the summary of values produced using the pedo-transfer
# function with one of the proposed methods.
summary(estimateBD(dat$SOC[is.na(dat$BLD)],
method="Honeyset_Ratkowsky1989"))
## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 0.0334 1.1599 1.3498 1.2972 1.4981 1.7730 729
# Fill NA's using the pedotransfer function:
dat$BLD[is.na(dat$BLD)] <- estimateBD(dat$SOC[is.na(dat$BLD)],
method="Grigal1989")

BD Estimation
# explore the results
hist(dat$BLD, col = 'light gray', breaks = 32)

Coarse Fragments
# summary of column CRF (Coarse Fragments) in the example data base
summary(dat$CRF)
## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 0.0 2.0 8.0 13.5 21.0 104.0 3360
# Convert NA's to 0
dat$CRF[is.na(dat$CRF)] <- 0
Measuring of coarse fragments is difficult and time consuming,
and therefore not part of the national soil inventories.

Coarse Fragments
hist(dat$CRF, col = "light gray")

DEPTH?
● Option 1: Soil sampling has already considered this depth: data
can be directly used for upscaling see “Upscaling Methods”
section)
● Option 2: Horizons or layers/depth classes are sampled; but
aggregation is needed over the 0-30 cm.
● Option 3: The target depth (0-30 cm) was not completely
covered by sampling e.g. only the A horizon or a topsoil layer
(e.g. 0-20 cm) has been sampled.

DEPTH?
For both options 2 and 3, additional processing is needed
(e.g. equal-area splines).

DEPTH?
For both options 2 and 3, transformation is needed using e.g. equal-area
splines. In the case of option 2, the use of equal-area splines was first
proposed by Ponce-Hernandez et al. (1986), and later tested against real
data (Bishop et al. 1999).
This technique is based on fitting continuous depth functions for
modelling the variability of soil properties with depth. Thus, it is
possible to convert soil profiles to standard depths, but also to fill gaps.
The equal-area spline function consists of a series of local quadratic
polynomials that join at ’knots’ located at the horizon boundaries
thereby the mean value of each horizon is maintained by the spline fit.
They are called equal-area splines because the area to the left of the
fitted spline curve is equal to the area to the right of the curve.

DEPTH?
In case of option 3 additional information on the
vertical distribution of carbon in the soils is
required for accurate recalculation from the
sampling depth to target depth, e.g. as was
shown by Bernoux et al. (1998).

In R environment, the easiest way to apply
equal-area splines is using the function
GSIF::mpspline from the R package GSIF (Hengl
2016, see section 4.3.2). For illustration, a
sample dataset has been used (see Chapter 5.).
This function requires data stored as
SoilProfileCollection (SPC) using package aqp.
Technical Steps - Equal area splines

The function GSIF::mpspline has several arguments.
One of the arguments is the lambda value mentioned
before. The proposed default value is 0.1. Another
argument for this function is the target standard
depths. The function produces spline-estimated values
at these depths. However, this function also produces
spline-estimated values at 1 cm increments.

# Load aqp package
library(aqp)
## This is aqp 1.15
##
## Attaching package: 'aqp'
# Promote to SoilProfileCollection
# The SoilProfileCollection is a object class in R
designed to
# handle soil profiles
depths(dat) <- ProfID ~ top + bottom
## Warning: converting IDs from factor to character

# Merge the soil horizons information with the site-
level
# information from dat_sites
site(dat) <- dat_sites
# Set spatial coordinates
coordinates(dat) <- ~ X + Y
# A summary of our SoilProfileCollection
dat

# A summary of our SoilProfileCollection
dat
## Object of class SoilProfileCollection
## Number of profiles: 4118
## Depth range: 5-295 cm
##
## Horizon attributes:
## ProfID HorID top bottom SOC BLD CRF SAND SILT CLAY
## 1 P0000 P0000H01 4 23 NA NA 54 52 34 14
## 2 P0000 P0000H02 23 46 NA NA 62 59 31 11
...

library(GSIF)
## Estimate 0-30 standard horizon using mass preserving splines
try(SOC <- mpspline(dat, 'SOC', d = t(c(0,30))))
try(BLD <- mpspline(dat, 'BLD', d = t(c(0,30))))
try(CRFVOL <- mpspline(dat, 'CRF', d = t(c(0,30))))

## Prepare final data frame
dat <- data.frame(id = dat@site$ProfID,
Y = dat@sp@coords[,2],
X = dat@sp@coords[,1],
SOC = SOC$var.std[,1],
BLD = BLD$var.std[,1],
CRFVOL = CRFVOL$var.std[,1])
dat <- dat[complete.cases(dat),]
## Take a look to the results
head(dat)

Stock Calculation
the estimation of organic carbon stock could be done using
GSIF package
# SOC must be in g/kg
# BLD in kg/m3
# CRF in percentage
OCSKGM <- OCSKGM(ORCDRC = dat$SOC, BLD = dat$BLD*1000,
CRFVOL = dat$CRFVOL, HSIZE = 30)
dat$OCSKGM <- OCSKGM
dat$meaERROR <- attr(OCSKGM,"measurementError")
dat <- dat[dat$OCSKGM>0,]
summary(dat)
## We can save our processed data as a table
write.csv(dat, "data/dataproc.csv")

Scenario 2 : Topsoil DB
dat <- read.csv(file = "data/auger.csv")
# Explore the data
str(dat)
summary(dat)
# Creating a function in R to estimate BLD using the SOC
# SOC is the soil organic carbon content in %
estimateBD <- function(SOC, method="Saini1996"){
OM <- SOC * 1.724
if(method=="Saini1996"){BD <- 1.62 - 0.06 * OM}
if(method=="Drew1973"){BD <- 1 / (0.6268 + 0.0361 * OM)}
if(method=="Jeffrey1979"){BD <- 1.482 - 0.6786 * (log(OM))}
if(method=="Grigal1989"){BD <- 0.669 + 0.941 * exp(1)^(-0.06 * OM)}
if(method=="Adams1973"){BD <- 100 / (OM /0.244 + (100 - OM)/2.65)}
if(method=="Honeyset_Ratkowsky1989"){BD <- 1/(0.564 + 0.0556 * OM)}
return(BD)
}
Data preparation for top soil or auger samples

# See the summary of values produced using the pedo-transfer
# function with one of the proposed methods.
summary(estimateBD(dat$SOC, method="Honeyset_Ratkowsky1989"))
# Estimate BLD using the pedotransfer function:
dat$BLD <- estimateBD(dat$SOC, method="Grigal1989")
# explore the results
boxplot(dat$BLD)
Bulk Density Estimation

# Remove points with NA's values
d
at <- dat[complete.cases(dat),]
## Take a look to the results
head(dat)
Remove NAs

# SOC must be in g/kg
# BLD in kg/m3
# CRF in percentage
OCSKGM <- OCSKGM(ORCDRC = dat$SOC, BLD = dat$BLD*1000, CRFVOL = 0,
HSIZE = 30)
SOC Stock Estimation

dat$OCSKGM <- OCSKGM
dat$meaERROR <- attr(OCSKGM,"measurementError")
dat <- dat[dat$OCSKGM>0,]
summary(dat)
## We can save our processed data as a table
write.csv(dat, "data/dataproc.csv")
Final bits..!

Next ...
Once we have standardised the SOC concentrations
to the standard depths we need to start thinking
about implementing a Digital Soil Mapping
framework in which we can generate predictions of
soil properties at the fixed depths at unvisited/non-
sampled sites.

Data Preparation HANDS-ON
SAMPLE TOPSOIL DATASET
https://goo.gl/32xLBJ
1. Make your dataset ready for modelling
a. Estimate Bulk Density Values
b. Calculate SOC Stocks for each location
c. Export your data for the further steps...

Data Preparation HANDS-ON
SAMPLE PROFILE DATASET
https://goo.gl/v1mWdU
1. Make your dataset ready for modelling
a. Estimate Bulk Density Values
b. Calculate SOC Stocks for each location
c. Use spline and recalculate SOC Stocks to 0-30 cm
d. Export your data for the further steps

Spline Tool V2.0
This tool can be alternatively used for modelling soil
attribute depth functions. The version v2.0
developed by CSIRO Land and Water. This
standalone tool has an easy-to-use graphical user
interface and allows user to estimate soil
properties for standard depth intervals using mass
preserving splines from input profiles with irregular
or non-contiguous depth intervals.
It is a prerequisite that the Microsoft .NET framework 4
must be installed prior to using the tool (download
standalone .NET framework 4).

Spline Tool V2.0
The Spline Tool uses comma delimited input text files (see
input.txt for formatting – depth must be in centimetres)
and generates cmsout.txt, stdout.txt and single.txt. Check
boxes on the settings tab allow the user to select which
output files to export. The spline tool currently inputs and
outputs one soil property (i.e. one of pH, clay content, etc)
only. The cmsout.txt file contains soil attribute values for
1cm intervals to a depth of 200cm.
The maximum depth of the output is limited by the
maximum input depth.

Spline Tool V2.0
The stdout.txt file contains soil attribute values
for the standard depth specified by
GlobalSoilMap.net as the default setting. The
default depths are: 0-5, 5-15, 15-30, 30-60, 60-
100, 100-200 cm. Custom depths ( GSOCMap
mandatory depth- 30 cm) can assigned from the
settings tab.

Spline Tool V2.0
Installation
To install the Spline Tool simply extract the .zip
content into a new folder.
http://www.asris.csiro.au/downloads/GSM/SplineTool_v2.zip
https://goo.gl/7XRQ7O
or

Spline Tool V2.0
Installation
To install the Spline Tool simply extract the .zip
content into a new folder.
http://www.asris.csiro.au/downloads/GSM/SplineTool_v2.zip

Data Preparation
The Spline Tool uses comma delimited input text
files (see input.txt for formatting – depth must be
in centimetres)

Data preparation, depth function

More Related Content

What's hot (20)

Similar to Data preparation, depth function (20)

More from FAO (20)

Recently uploaded (20)

Data preparation, depth function