Abstract image of a woman sitting on books and studying on a laptop

10. Getting Spatial

FAO, profile picture
FAO

This document summarizes steps for working with spatial point data in R, including: 1. Importing point data from a CSV file and defining the coordinate columns; 2. Specifying the coordinate reference system of the data; 3. Plotting the data spatially and exporting to common GIS formats like shapefiles; 4. Transforming the data to a different CRS (WGS84) in order to visualize in Google Earth.

Education
1 of 64
Downloaded 16 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
GSP - Asian Soil Partnership Training Workshop on Soil Organic Carbon Mapping Bangkok, Thailand, 24-29 April 2017 Yusuf YIGINI, PhD - FAO, Land and Water Division (CBL)
DAY 4 – 28 April 2017 TIME TOPIC INSTRUCTORS 8:30 - 10:30 GSOC Map Depth Classes, Spline Function Hands-on: Data Manipulation Dr. Yusuf Yigini, FAO Dr. Ate Poortinga Dr. Lucrezia Caon, FAO 10:30 - 11:00 COFFEE BREAK 11:00 - 13:00 Cont. GSOC Map Depth Classes, Spline Function 13:00 - 14:00 LUNCH 14:00 - 16:00 R and Spatial Data Spatial Operations: Points, Rasters Hands-on: Basic Spatial Operations 16:00- 16:30 COFFEE BREAK 16:30 - 17:30 Cont. R and Spatial Data Spatial Operations: Points, Rasters Hands-on: Basic Spatial Operations
R - Getting Spatial
Sample Data - Points We will be working with a data set of soil information that was collected from Macedonia (FYROM). https://goo.gl/EKKMAF
Vectors
> setwd("C:/mc") > pointdata <- read.csv("mc_profile_data.csv") > View(pointdata) > str(pointdata) 'data.frame': 3302 obs. of 9 variables: $ ID : int 4 7 8 9 10 11 12 13 14 15 ... $ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... $ X : int 7485085 7486492 7485564 7495075 7494798 7492500 7493700 7490922 7489842 7490414 ... $ Y : int 4653725 4653203 4656242 4652933 4651945 4651760 4652388 4651714 4653025 4650948 ... $ UpperDepth: int 0 0 0 0 0 0 0 0 0 0 ... $ LowerDepth: int 30 30 30 30 30 30 30 30 30 30 ... $ Value : num 11.88 3.49 2.32 1.94 1.34 ... $ Lambda : num 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ... $ tsme : num 0.1601 0.00257 0.0026 0.00284 0.00268 ...
Required Packages Now we need load the necessary R packages (you may have to install them onto your computer first): > library(sp) > library(raster) > library(rgdal)
Coordinates We can use the coordinates() function from the sp package to define which columns in the data frame refer to actual spatial coordinates—here the coordinates are listed in columns X and Y. > coordinates(pointdata) <- ~X + Y
Coordinates > coordinates(pointdata) <- ~X + Y > str(pointdata) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: .. ..$ ID : int [1:3302] 4 7 8 9 10 11 12 13 14 15 ... .. ..$ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... .. ..$ UpperDepth: int [1:3302] 0 0 0 0 0 0 0 0 0 0 ... .. ..$ LowerDepth: int [1:3302] 30 30 30 30 30 30 30 30 30 30 ... .. ..$ Value : num [1:3302] 11.88 3.49 2.32 1.94 1.34 ... .. ..$ Lambda : num [1:3302] 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ... .. ..$ tsme : num [1:3302] 0.1601 0.00257 0.0026 0.00284 0.00268 ... ..@ coords.nrs : int [1:2] 3 4 ..@ coords : num [1:3302, 1:2] 7485085 7486492 7485564 7495075 7494798 ... .. ..- attr(*, "dimnames")=List of 2 .. .. ..$ : chr [1:3302] "1" "2" "3" "4" ... .. .. ..$ : chr [1:2] "X" "Y" ..@ bbox : num [1:2, 1:2] 7455723 4526565 7667660 4691342 .. ..- attr(*, "dimnames")=List of 2 .. .. ..$ : chr [1:2] "X" "Y" .. .. ..$ : chr [1:2] "min" "max"
Coordinates > coordinates(pointdata) <- ~X + Y > str(pointdata) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: .. ..$ ID : int [1:3302] 4 7 8 9 10 11 12 13 14 15 ... .. ..$ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... .. ..$ UpperDepth: int [1:3302] 0 0 0 0 0 0 0 0 0 0 ... .. ..$ LowerDepth: int [1:3302] 30 30 30 30 30 30 30 30 30 30 ... .. ..$ Value : num [1:3302] 11.88 3.49 2.32 1.94 1.34 ... .. ..$ Lambda : num [1:3302] 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ... .. ..$ tsme : num [1:3302] 0.1601 0.00257 0.0026 0.00284 0.00268 ... ..@ coords.nrs : int [1:2] 3 4 ..@ coords : num [1:3302, 1:2] 7485085 7486492 7485564 7495075 7494798 ... .. ..- attr(*, "dimnames")=List of 2 .. .. ..$ : chr [1:3302] "1" "2" "3" "4" ... .. .. ..$ : chr [1:2] "X" "Y" ..@ bbox : num [1:2, 1:2] 7455723 4526565 7667660 4691342 .. ..- attr(*, "dimnames")=List of 2 .. .. ..$ : chr [1:2] "X" "Y" .. .. ..$ : chr [1:2] "min" "max" Note that by using the str function, the class of pointdata has now changed from a dataframe to a SpatialPointsDataFrame. We can do a spatial plot of these points using the spplot plotting function in the sp package.
> setwd("C:/mc") > pointdata <- read.csv("mc_profile_data.csv") > View(pointdata) > str(pointdata) 'data.frame': 3302 obs. of 9 variables: $ ID : int 4 7 8 9 10 11 12 13 14 15 ... $ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... $ X : int 7485085 7486492 7485564 7495075 7494798 7492500 7493700 7490922 7489842 7490414 ... $ Y : int 4653725 4653203 4656242 4652933 4651945 4651760 4652388 4651714 4653025 4650948 ... $ UpperDepth: int 0 0 0 0 0 0 0 0 0 0 ... $ LowerDepth: int 30 30 30 30 30 30 30 30 30 30 ... $ Value : num 11.88 3.49 2.32 1.94 1.34 ... $ Lambda : num 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ... $ tsme : num 0.1601 0.00257 0.0026 0.00284 0.00268 ...
Spatial Data Frame spplot(pointdata, "Value", scales = list(draw = T), cuts = 5, col.regions = bpy.colors(cutoff.tails = 0.1,alpha = 1), cex = 1) There are other plotting options available, so it will be helpful to consult the help file (?). Here, we are plotting the SOC concentration measured at each location
Spatial Data Frame spplot(pointdata, "Value", scales = list(draw = T), cuts = 5, col.regions = bpy.colors(cutoff.tails = 0.1,alpha = 1), cex = 1) There are other plotting options available, so it will be helpful to consult the help file (?). Here, we are plotting the SOC concentration measured at each location
Spatial Data Frame SpatialPointsDataFrame structure is essentially the same data frame, except that additional “spatial” elements have been added or partitioned into slots. Some important ones being the bounding box (sort of like the spatial extent of the data), and the coordinate reference system proj4string(), which we need to define for the sample dataset. To define the CRS, we must know where our data are from, and what was the corresponding CRS used when recording the spatial information in the field. For this data set the CRS used was: Macedonia_State_Coordinate_System_zone_7
Coordinate Reference System To clearly tell R this information we define the CRS which describes a reference system in a way understood by the PROJ.4 projection library http://trac.osgeo.org/proj/. An interface to the PROJ.4 library is available in the rgdal package. Alternative to using Proj4 character strings, we can use the corresponding yet simpler EPSG code (European Petroleum Survey Group). rgdal also recognizes these codes. If you are unsure of the Proj4 or EPSG code for the spatial data that you have, but know the CRS, you should consult http://spatialreference.org/ for assistance.
Spatial Data Frame > proj4string(pointdata) <- CRS("+init=epsg:6316") > > pointdata@proj4string CRS arguments: +init=epsg:6316 +proj=tmerc +lat_0=0 +lon_0=21 +k=0.9999 +x_0=7500000 +y_0=0 +ellps=bessel +towgs84=682,-203,480,0,0,0,0 +units=m +no_defs First we need to define the CRS and then we can perform any sort of spatial analysis.
Spatial Data Frame > writeOGR(pointdata, ".", "pointdata-shape", "ESRI Shapefile") # Check your working directory for presence of this file For example, we may want to use these data in other GIS environments such as ArcGIS, QGIS, SAGA GIS etc. This means we need to export the SpatialPointsDataFrame of pointdata to an appropriate spatial data format such as a shapefile. rgdal is again used for this via the writeOGR() function. To export the data set as a shapefile:
Spatial Data Frame > writeOGR(pointdata, ".", "pointdata-shape", "ESRI Shapefile") # Check your working directory for presence of this file For example, we may want to use these data in other GIS environments such as ArcGIS, QGIS, SAGA GIS etc. This means we need to export the SpatialPointsDataFrame of pointdata to an appropriate spatial data format such as a shapefile. rgdal is again used for this via the writeOGR() function. To export the data set as a shapefile: Note that the object we need to export needs to be a spatial points data frame. You should try opening this exported shapefile in your GIS software (ArcGIS, SAGA, QGIS...=).
Coordinate Transformation > pointdata.kml <- spTransform(pointdata, CRS("+init=epsg:4326")) > writeOGR(pointdata.kml, "pointdata.kml", "ID", "KML") To look at the locations of the data in Google Earth, we first need to make sure the data is in the WGS84 geographic CRS. If the data is not in this CRS (which is the case for our data), then we need to perform a transformation. This is done by using the spTransform function in sp. The EPSG code for WGS84 geographic is: 4326. We can then export out our transformed pointdata data set to a KML file and visualize it in Google Earth.
> pointdata.kml <- spTransform(pointdata, CRS("+init=epsg:4326")) > writeOGR(pointdata.kml, "pointdata.kml", "ID", "KML") To look at the locations of the data in Google Earth, we first need to make sure the data is in the WGS84 geographic CRS. If the data is not in this CRS (which is the case for our data), then we need to perform a transformation. This is done by using the spTransform function in sp. The EPSG code for WGS84 geographic is: 4326. We can then export out our transformed pointdata data set to a KML file and visualize it in Google Earth.
KML’s > pointdata.kml <- spTransform(pointdata, CRS("+init=epsg:4326")) To look at the locations of the data in Google Earth, we first need to make sure the data is in the WGS84 geographic CRS. If the data is not in this CRS (which is the case for our data), then we need to perform a transformation. This is done by using the spTransform function in sp. The EPSG code for WGS84 geographic is: 4326. We can then export out our transformed pointdata data set to a KML file and visualize it in Google Earth.
Coordinate Transformation > pointdata.kml <- spTransform(pointdata, CRS("+init=epsg:4326")) To look at the locations of the data in Google Earth, we first need to make sure the data is in the WGS84 geographic CRS. If the data is not in this CRS (which is the case for our data), then we need to perform a transformation. This is done by using the spTransform function in sp. The EPSG code for WGS84 geographic is: 4326. We can then export out our transformed pointdata data set to a KML file and visualize it in Google Earth.
Sometimes to conduct further analysis of spatial data, we may just want to import it into R directly. For example, read in a shapefile (this includes both points and polygons). Now read in that shapefile that was created just before and saved to the working directory “pointdata-shape.shp”: Read Shape Files in R > pointshape <- readOGR("pointdata-shape.shp") OGR data source with driver: ESRI Shapefile Source: "pointdata-shape.shp", layer: "pointdata-shape" with 3302 features It has 7 fields
The imported shapefile is now a SpatialPointsDataFrame, just like the pointdata data that was worked on before, and is ready for further analysis. Read Shape Files in R > pointshape@proj4string CRS arguments: +proj=tmerc +lat_0=0 +lon_0=21 +k=0.9999 +x_0=7500000 +y_0=0 +ellps=bessel +units=m +no_defs
The imported shapefile is now a SpatialPointsDataFrame, just like the pointdata data that was worked on before, and is ready for further analysis. Read Shape Files in R > str(pointshape) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: ...
The imported shapefile is now a SpatialPointsDataFrame, just like the pointdata data that was worked on before, and is ready for further analysis. Read Shape Files in R > str(pointshape) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: ...
Rasters
Rasters Most of the functions for handling raster data are available in the raster package. There are functions for reading and writing raster files from and to different formats. In digital soil mapping we mostly work with data in table format and then rasterise this data so that we can make a continuous map. For doing this in R environment, we will load raster data in a data frame. This data is a digital elevation model provided by ISRIC for FYROM.
Rasters Most of the functions for handling raster data are available in the raster package. There are functions for reading and writing raster files from and to different formats. In digital soil mapping we mostly work with data in table format and then rasterise this data so that we can make a continuous map.
For doing this in R environment, we will load raster data in a data frame. This data is a digital elevation model provided by ISRIC for FYROM. Read Rasters in R > mac.dem <- raster("covs/dem1.tif") > points <- readOGR("covs/pointshape.shp")
For doing this in R environment, we will load raster data in a data frame. This data is a digital elevation model provided by ISRIC for FYROM. Read Rasters in R > str(mac.dem) Formal class 'RasterLayer' [package "raster"] with 12 slots ..@ file :Formal class '.RasterFile' [package "raster"] with 13 slots .. .. ..@ name : chr "C:mccovsdem1.tif" .. .. ..@ datanotation: chr "INT2S" .. .. ..@ byteorder : chr "little" .. .. ..@ nodatavalue : num -Inf .. .. ..@ NAchanged : logi FALSE .. .. ..@ nbands : int 1
So lets do a quick plot of this raster and overlay the point locations Read Rasters in R plot(mac.dem) points(points, pch = 20)
So lets do a quick plot of this raster and overlay the point locations Read Rasters in R plot(mac.dem) points(points, pch = 20)
So you may want to export this raster to a suitable format to work in a standard GIS environment. See the help file for writeRaster to get information regarding the supported grid types that data can be exported. Here, we will export our raster to ESRI Ascii, as it is a common and universal raster format. Write Raster in R writeRaster(mac.dem, filename = "mac-dem.asc",format = "ascii", overwrite = TRUE)
We may also want to export our mac.dem to KML file using the KML function. Note that we need to reproject our data to WGS84 geographic. The raster re-projection is performed using the projectRaster function. Look at the help file for this! KML is a handy function from raster for exporting grids to kml format. Write Raster in R writeRaster(mac.dem, filename = "mac-dem.asc",format = "ascii", overwrite = TRUE)
We may also want to export our mac.dem to KML file using the KML function. Note that we need to reproject our data to WGS84 geographic. The raster re-projection is performed using the projectRaster function. Look at the help file for this! KML is a handy function from raster for exporting grids to kml format. Export Raster in KML > KML(mac.dem, "macdem.kml", col = rev(terrain.colors(255)), overwrite = TRUE)
We may also want to export our mac.dem to KML file using the KML function. Note that we need to reproject our data to WGS84 geographic. The raster re-projection is performed using the projectRaster function. Look at the help file for this! KML is a handy function from raster for exporting grids to kml format. Export Raster in KML > KML(mac.dem, "macdem.kml", col = rev(terrain.colors(255)), overwrite = TRUE) Check your working space for presence of the kml file!
Now visualize this in Google Earth and overlay this map with the points that we created created before Export Raster in KML
The other useful procedure we can perform is to import rasters directly into R so we can perform further analyses. rgdal interfaces with the GDAL library, which means that there are many supported grid formats that can be read into R. Import Rasters http://www.gdal.org/formats_list.html
Here we will load in the our .asc raster that was made just before. Import Rasters > read.grid <- readGDAL("mac-dem.asc") mac-dem.asc has GDAL driver AAIGrid and has 304 rows and 344 columns
The imported raster read.grid is a SpatialGridDataFrame, which is a class of the sp package. To be able to use the raster functions from raster we need to convert it to the RasterLayer class. Import Rasters > str(grid.dem) Formal class 'RasterLayer' [package "raster"] with 12 slots ..@ file :Formal class '.RasterFile' [package "raster"] with 13 slots .. .. ..@ name : chr "" .. .. ..@ datanotation: chr "FLT4S" .. .. ..@ byteorder : chr "little" .. .. ..@ nodatavalue : num -Inf .. .. ..@ NAchanged : logi FALSE .. .. ..@ nbands : int 1 .. .. ..@ bandorder : chr "BIL" .. .. ..@ offset : int 0 .. .. ..@ toptobottom : logi TRUE .. .. ..@ blockrows : int 0
It should be noted that R generated data source is loaded into memory. This is fine for small size data but can become a problem when working with very large rasters. A really useful feature of the raster package is the ability to point to the location of a raster file without loading it into the memory. Import Rasters grid.dem <- raster(paste(paste(getwd(), "/", sep = ""),"mac-dem.asc", sep = "")) > grid.dem class : RasterLayer dimensions : 304, 344, 104576 (nrow, ncol, ncell) resolution : 0.008327968, 0.008327968 (x, y) extent : 20.27042, 23.13524, 40.24997, 42.78167 (xmin, xmax, ymin, ymax) coord. ref. : NA data source : C:mcmac-dem.asc names : mac.dem
Import Rasters > plot(mac.dem)
Overlaying Soil Point Observations with Environmental Covariates
Data Preparation for DSM In order to carry out digital soil mapping techniques for evaluating the significance of environmental variables in explaining the spatial variation of the target soil variable (for example SOC) , we need to link both sets of data together and extract raster values from covariates at the locations of the soil point data.
Data Preparation for DSM > points class : SpatialPointsDataFrame features : 3302 extent : 20.46948, 23.01584, 40.88197, 42.3589 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 variables : 7 names : ID, ProfID, UpperDepth, LowerDepth, Value, Lambda, tsme min values : 10, P0004, 0, 30, 0.00000000, 0.1, 0.002250115 max values : 999, P6539, 0, 30, 50.33234687, 0.1, 0.160096433 > mac.dem class : RasterLayer dimensions : 304, 344, 104576 (nrow, ncol, ncell) resolution : 0.008327968, 0.008327968 (x, y) extent : 20.27042, 23.13524, 40.24997, 42.78167 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 data source : C:mccovsdem1.tif names : dem1 values : 16, 2684 (min, max)
Data Preparation for DSM > points class : SpatialPointsDataFrame features : 3302 extent : 20.46948, 23.01584, 40.88197, 42.3589 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 variables : 7 names : ID, ProfID, UpperDepth, LowerDepth, Value, Lambda, tsme min values : 10, P0004, 0, 30, 0.00000000, 0.1, 0.002250115 max values : 999, P6539, 0, 30, 50.33234687, 0.1, 0.160096433 > mac.dem class : RasterLayer dimensions : 304, 344, 104576 (nrow, ncol, ncell) resolution : 0.008327968, 0.008327968 (x, y) extent : 20.27042, 23.13524, 40.24997, 42.78167 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 data source : C:mccovsdem1.tif names : dem1 values : 16, 2684 (min, max)
Data Preparation for DSM > DSM_table <- extract(mac.dem, points, sp = 1,method = "simple") > DSM_table class : SpatialPointsDataFrame features : 3302 extent : 20.46948, 23.01584, 40.88197, 42.3589 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 variables : 8 names : ID, ProfID, UpperDepth, LowerDepth, Value, Lambda, tsme, dem1 min values : 10, P0004, 0, 30, 0.00000000, 0.1, 0.002250115, 45 max values : 999, P6539, 0, 30, 50.33234687, 0.1, 0.160096433, 2442 The sp parameter set to 1 means that the extracted covariate data gets appended to the existing SpatialPointsDataFrame object. While the method object specifies the extraction method which in our case is “simple” which likened to get the covariate value nearest to the points
Data Preparation for DSM > DSM_table <- as.data.frame(DSM_table) > write.table(DSM_table, "DSM_table.TXT", col.names = T, row.names = FALSE, sep = ",") The sp parameter set to 1 means that the extracted covariate data gets appended to the existing SpatialPointsDataFrame object. While the method object specifies the extraction method which in our case is “simple” which likened to get the covariate value nearest to the points
Data Preparation for DSM > DSM_table <- as.data.frame(DSM_table) > write.table(DSM_table, "DSM_table.TXT", col.names = T, row.names = FALSE, sep = ",") The sp parameter set to 1 means that the extracted covariate data gets appended to the existing SpatialPointsDataFrame object. While the method object specifies the extraction method which in our case is “simple” which likened to get the covariate value nearest to the points
Using Covariates from Disc > list.files(path = "C:/mc/covs", pattern = ".tif$", + full.names = TRUE) [1] "C:/mc/covs/dem.tif" "C:/mc/covs/dem1.tif" "C:/mc/covs/prec.tif" "C:/mc/covs/slp.tif" > list.files(path = "C:/mc/covs") [1] "dem.tif" "dem1.tfw" "dem1.tif" "dem1.tif.aux.xml" "dem1.tif.ovr" [6] "desktop.ini" "pointshape.cpg" "pointshape.dbf" "pointshape.prj" "pointshape.sbn" [11] "pointshape.sbx" "pointshape.shp" "pointshape.shx" "prec.tif" "slp.tif" This utility is obviously a very handy feature when we are working with large or large number of rasters. The work function we need is list.files. For example:
Using Covariates from Disc > list.files(path = "C:/mc/covs", pattern = ".tif$", + full.names = TRUE) [1] "C:/mc/covs/dem.tif" "C:/mc/covs/dem1.tif" "C:/mc/covs/prec.tif" "C:/mc/covs/slp.tif" > list.files(path = "C:/mc/covs") [1] "dem.tif" "dem1.tfw" "dem1.tif" "dem1.tif.aux.xml" "dem1.tif.ovr" [6] "desktop.ini" "pointshape.cpg" "pointshape.dbf" "pointshape.prj" "pointshape.sbn" [11] "pointshape.sbx" "pointshape.shp" "pointshape.shx" "prec.tif" "slp.tif" This utility is obviously a very handy feature when we are working with large or large number of rasters. The work function we need is list.files. For example:
Using Covariates from Disc Covs <- list.files(path = "C:/mc/covs", pattern = ".tif$",full.names = TRUE) > Covs [1] "C:/mc/covs/dem.tif" "C:/mc/covs/dem1.tif" "C:/mc/covs/prec.tif" "C:/mc/covs/slp.tif" > covStack <- stack(Covs) > covStack Error in compareRaster(rasters) : different extent When the covariates in common resolution and extent, rather than working with each raster independently it is more efficient to stack them all into a single object. The stack function from raster is ready-made for this, and is simple as follow,
Using Covariates from Disc Covs <- list.files(path = "C:/mc/covs", pattern = ".tif$",full.names = TRUE) > Covs [1] "C:/mc/covs/dem.tif" "C:/mc/covs/dem1.tif" "C:/mc/covs/prec.tif" "C:/mc/covs/slp.tif" > covStack <- stack(Covs) > covStack Error in compareRaster(rasters) : different extent If the rasters are not in same resolution and extent you will find the other raster package functions resample and projectRaster as invaluable methods for harmonizing all your different raster layers.
Exploratory Data Analysis
Exploratory Data Analysis We will continue using the DSM_table object that we created in the previous section. As the data set was saved to file you will also find it in your working directory. > str(points) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: .. ..$ ID : Factor w/ 3228 levels "10","100","1000",..: 1896 3083 3136 3172 1 66 117 141 144 179 ... .. ..$ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... .. ..$ UpperDepth: Factor w/ 1 level "0": 1 1 1 1 1 1 1 1 1 1 ... .. ..$ LowerDepth: Factor w/ 1 level "30": 1 1 1 1 1 1 1 1 1 1 ... .. ..$ Value : num [1:3302] 11.88 3.49 2.32 1.94 1.34 ... .. ..$ Lambda : num [1:3302] 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ...
Exploratory Data Analysis Hereafter soil carbon density will be referred to as Value. Now lets firstly look at some of the summary statistics of SOC > summary(points$Value) Min. 1st Qu. Median Mean 3rd Qu. Max. 0.000 1.005 1.492 1.911 2.244 50.330
Exploratory Data Analysis The observation that the mean and median are not equivalent says that the distribution of this data is not normal. > summary(points$Value) Min. 1st Qu. Median Mean 3rd Qu. Max. 0.000 1.005 1.492 1.911 2.244 50.330
Exploratory Data Analysis The observation that the mean and median are not equivalent says that the distribution of this data seem not normal. To check this statistically, > install.packages("nortest") > install.packages("fBasics") > library(fBasics) > library(nortest) > sampleSKEW(points$Value) SKEW 0.2126149 > sampleKURT(points$Value) KURT 1.500089
Exploratory Data Analysis Here we see that the data is positively skewed.Anderson-Darling Test can be used to test normality. > sampleSKEW(points$Value) SKEW 0.2126149 > sampleKURT(points1$Value) KURT 1.500089 > ad.test(points$Value) Anderson-Darling normality test data: points$Value A = 315.95, p-value < 2.2e-16
Exploratory Data Analysis for normally distributed data the p value should be > than 0.05. This is confirmed when we look at the histogram and qq-plot of this data > par(mfrow = c(1, 2)) > hist(points$Value) > qqnorm(points$Value, plot.it = TRUE, pch = 4, cex = 0.7) > qqline(points$Value, col = "red", lwd = 2)
Exploratory Data Analysis for normally distributed data the p value should be > than 0.05. This is confirmed when we look at the histogram and qq-plot of this data > par(mfrow = c(1, 2)) > hist(points$Value) > qqnorm(points$Value, plot.it = TRUE, pch = 4, cex = 0.7) > qqline(points$Value, col = "red", lwd = 2)
Exploratory Data Analysis Most statistical models assume data is normally distributed. A way to make the data to be more normal is to transform it. Common transformations include the square root, logarithmic, or power transformations. > ad.test(sqrt(points1$Value)) Anderson-Darling normality test data: sqrt(points1$Value) A = 67.687, p-value < 2.2e-16 > sampleKURT(sqrt(points1$Value)) KURT 1.373565 > sampleSKEW(sqrt(points$Value)) SKEW 0.1148215
Exploratory Data Analysis Most statistical models assume data is normally distributed. A way to make the data to be more normal is to transform it. Common transformations include the square root, logarithmic, or power transformations. > ad.test(sqrt(points1$Value)) Anderson-Darling normality test data: sqrt(points1$Value) A = 67.687, p-value < 2.2e-16 > sampleKURT(sqrt(points1$Value)) KURT 1.373565 > sampleSKEW(sqrt(points$Value)) SKEW 0.1148215 We could investigate other data transformations or even investigate the possibility of removing outliers or some such data..

More Related Content

PDF
5. R basics
FAO
 
PDF
11. Linear Models
FAO
 
PDF
Linear models
FAO
 
PDF
Vectors data frames
FAO
 
PDF
Regression kriging
FAO
 
PDF
13. Cubist
FAO
 
PDF
Dplyr and Plyr
Paul Richards
 
PDF
R programming intro with examples
Dennis
 
5. R basics
FAO
 
11. Linear Models
FAO
 
Linear models
FAO
 
Vectors data frames
FAO
 
Regression kriging
FAO
 
13. Cubist
FAO
 
Dplyr and Plyr
Paul Richards
 
R programming intro with examples
Dennis
 

What's hot (20)

PDF
Monad presentation scala as a category
samthemonad
 
PPTX
statistical computation using R- an intro..
Kamarudheen KV
 
PDF
Scala. Introduction to FP. Monads
Kirill Kozlov
 
PDF
Rsplit apply combine
Michelle Darling
 
PDF
Data manipulation with dplyr
Romain Francois
 
PPTX
Rewriting Engine for Process Algebras
Anatolii Kmetiuk
 
PPTX
PCA and LDA in machine learning
Akhilesh Joshi
 
PDF
Data Manipulation Using R (& dplyr)
Ram Narasimhan
 
PDF
4 R Tutorial DPLYR Apply Function
Sakthi Dasans
 
PPTX
Introduction to Monads in Scala (2)
stasimus
 
PDF
Pandas,scipy,numpy cheatsheet
Dr. Volkan OBAN
 
DOCX
CLUSTERGRAM
Dr. Volkan OBAN
 
PDF
Day 4b iteration and functions for-loops.pptx
Adrien Melquiond
 
PDF
R learning by examples
Michelle Darling
 
PPTX
Language R
Girish Khanzode
 
PDF
Pandas Cheat Sheet
ACASH1011
 
PDF
Day 4a iteration and functions.pptx
Adrien Melquiond
 
PPTX
R Language Introduction
Khaled Al-Shamaa
 
PPTX
R: Apply Functions
DataminingTools Inc
 
PDF
R Workshop for Beginners
Metamarkets
 
Monad presentation scala as a category
samthemonad
 
statistical computation using R- an intro..
Kamarudheen KV
 
Scala. Introduction to FP. Monads
Kirill Kozlov
 
Rsplit apply combine
Michelle Darling
 
Data manipulation with dplyr
Romain Francois
 
Rewriting Engine for Process Algebras
Anatolii Kmetiuk
 
PCA and LDA in machine learning
Akhilesh Joshi
 
Data Manipulation Using R (& dplyr)
Ram Narasimhan
 
4 R Tutorial DPLYR Apply Function
Sakthi Dasans
 
Introduction to Monads in Scala (2)
stasimus
 
Pandas,scipy,numpy cheatsheet
Dr. Volkan OBAN
 
CLUSTERGRAM
Dr. Volkan OBAN
 
Day 4b iteration and functions for-loops.pptx
Adrien Melquiond
 
R learning by examples
Michelle Darling
 
Language R
Girish Khanzode
 
Pandas Cheat Sheet
ACASH1011
 
Day 4a iteration and functions.pptx
Adrien Melquiond
 
R Language Introduction
Khaled Al-Shamaa
 
R: Apply Functions
DataminingTools Inc
 
R Workshop for Beginners
Metamarkets
 
Ad

Similar to 10. Getting Spatial (20)

PDF
R getting spatial
FAO
 
PDF
10. R getting spatial
ExternalEvents
 
ODP
Geospatial Data in R
Barry Rowlingson
 
PDF
Unit3 slides
Larefa Bidoun
 
PPT
R Spatial Analysis using SP
tjagger
 
PDF
Spatial_Data_Analysis_with_open_source_softwares[1]
Joachim Nkendeys
 
PDF
Finding Meaning in Points, Areas and Surfaces: Spatial Analysis in R
Revolution Analytics
 
PDF
Spatial Analysis with R - the Good, the Bad, and the Pretty
Noam Ross
 
PDF
Scattered gis handbook
Waleed Liaqat
 
PDF
unitiv-spacialdataanalysis-200423132043.pdf
sumitshrivastav2904
 
PPTX
TYBSC IT PGIS Unit IV Spacial Data Analysis
Arti Parab Academics
 
PPT
Gis Concepts 5/5
CIER Facultad de Agronomía
 
PDF
Data preparation, depth function
FAO
 
PDF
Using python to analyze spatial data
Kudos S.A.S
 
PDF
Introduction to spatial data analysis in r
Richard Wamalwa
 
PDF
Spatial analysis and Analysis Tools
Swapnil Shrivastav
 
PPTX
Geographical information system
Naresh Kumar
 
PDF
Essentials of R
ExternalEvents
 
PDF
Gis basic
Mahbubul Hassan
 
PDF
Geographical Information System (GIS) Georeferencing and Digitization, Bihar ...
Kamlesh Kumar
 
R getting spatial
FAO
 
10. R getting spatial
ExternalEvents
 
Geospatial Data in R
Barry Rowlingson
 
Unit3 slides
Larefa Bidoun
 
R Spatial Analysis using SP
tjagger
 
Spatial_Data_Analysis_with_open_source_softwares[1]
Joachim Nkendeys
 
Finding Meaning in Points, Areas and Surfaces: Spatial Analysis in R
Revolution Analytics
 
Spatial Analysis with R - the Good, the Bad, and the Pretty
Noam Ross
 
Scattered gis handbook
Waleed Liaqat
 
unitiv-spacialdataanalysis-200423132043.pdf
sumitshrivastav2904
 
TYBSC IT PGIS Unit IV Spacial Data Analysis
Arti Parab Academics
 
Gis Concepts 5/5
CIER Facultad de Agronomía
 
Data preparation, depth function
FAO
 
Using python to analyze spatial data
Kudos S.A.S
 
Introduction to spatial data analysis in r
Richard Wamalwa
 
Spatial analysis and Analysis Tools
Swapnil Shrivastav
 
Geographical information system
Naresh Kumar
 
Essentials of R
ExternalEvents
 
Gis basic
Mahbubul Hassan
 
Geographical Information System (GIS) Georeferencing and Digitization, Bihar ...
Kamlesh Kumar
 
Ad

More from FAO (20)

PPTX
Nigeria
FAO
 
PPT
Niger
FAO
 
PPT
Namibia
FAO
 
PPT
Mozambique
FAO
 
PPT
Zimbabwe takesure
FAO
 
PPT
Zimbabwe
FAO
 
PPT
Zambia
FAO
 
PPT
Togo
FAO
 
PPT
Tanzania
FAO
 
PPT
Spal presentation
FAO
 
PPT
Rwanda
FAO
 
PPT
Nigeria uponi
FAO
 
PPTX
The multi-faced role of soil in the NENA regions (part 2)
FAO
 
PPTX
The multi-faced role of soil in the NENA regions (part 1)
FAO
 
PDF
Agenda of the launch of the soil policy brief at the Land&Water Days
FAO
 
PDF
Agenda of the 5th NENA Soil Partnership meeting
FAO
 
PPTX
The Voluntary Guidelines for Sustainable Soil Management
FAO
 
PPTX
GLOSOLAN - Mission, status and way forward
FAO
 
PPTX
Towards a Global Soil Information System (GLOSIS)
FAO
 
PPTX
GSP developments of regional interest in 2019
FAO
 
Nigeria
FAO
 
Niger
FAO
 
Namibia
FAO
 
Mozambique
FAO
 
Zimbabwe takesure
FAO
 
Zimbabwe
FAO
 
Zambia
FAO
 
Togo
FAO
 
Tanzania
FAO
 
Spal presentation
FAO
 
Rwanda
FAO
 
Nigeria uponi
FAO
 
The multi-faced role of soil in the NENA regions (part 2)
FAO
 
The multi-faced role of soil in the NENA regions (part 1)
FAO
 
Agenda of the launch of the soil policy brief at the Land&Water Days
FAO
 
Agenda of the 5th NENA Soil Partnership meeting
FAO
 
The Voluntary Guidelines for Sustainable Soil Management
FAO
 
GLOSOLAN - Mission, status and way forward
FAO
 
Towards a Global Soil Information System (GLOSIS)
FAO
 
GSP developments of regional interest in 2019
FAO
 

Recently uploaded (20)

PDF
International_Financial_Reporting_Standa.pdf
VrindaTayade1
 
PDF
Race Reva University – Shaping Future Leaders in Artificial Intelligence
RACE REVA University
 
PPTX
ELIAS-SEZIURE AND EPilepsy semmioan session.pptx
eyoba2172
 
PDF
Environmental Education MCQ BD2EE - Share Source.pdf
Dr Raja Mohammed T
 
PDF
Uderstanding digital marketing and marketing stratergie for engaging the digi...
afreenpeshmam
 
PDF
Empowerment Technology for Senior High School Guide
solthereseamericandr
 
PDF
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
Demetrio Ccesa Rayme
 
PPTX
Unit 4 Computer Architecture Multicore Processor.pptx
Gunasundari Selvaraj
 
PDF
advance database management system book.pdf
hinamannan364
 
PDF
LIFE & LIVING TRILOGY - PART - (2) THE PURPOSE OF LIFE.pdf
sureshkumarsoni26
 
PPTX
Computer Architecture Input Output Memory.pptx
Gunasundari Selvaraj
 
PDF
FORM 1 BIOLOGY MIND MAPS and their schemes
arcade5
 
PDF
BP 505 T. PHARMACEUTICAL JURISPRUDENCE (UNIT 2).pdf
CMEmpire
 
PPTX
Module on health assessment of CHN. pptx
GurdeepSingh626704
 
PDF
David L Page_DCI Research Study Journey_how Methodology can inform one's prac...
David L Page
 
PPTX
B.Sc. DS Unit 2 Software Engineering.pptx
Dr. Pallawi Bulakh
 
PDF
1.3 FINAL REVISED K-10 PE and Health CG 2023 Grades 4-10 (1).pdf
JohnJerryDalawampu
 
PDF
Skin Care and Cosmetic Ingredients Dictionary ( PDFDrive ).pdf
SahianAlondraPichard
 
PDF
Τίμαιος είναι φιλοσοφικός διάλογος του Πλάτωνα
iliaskessaris
 
PDF
MBA _Common_ 2nd year Syllabus _2021-22_.pdf
DrAnubha4
 
International_Financial_Reporting_Standa.pdf
VrindaTayade1
 
Race Reva University – Shaping Future Leaders in Artificial Intelligence
RACE REVA University
 
ELIAS-SEZIURE AND EPilepsy semmioan session.pptx
eyoba2172
 
Environmental Education MCQ BD2EE - Share Source.pdf
Dr Raja Mohammed T
 
Uderstanding digital marketing and marketing stratergie for engaging the digi...
afreenpeshmam
 
Empowerment Technology for Senior High School Guide
solthereseamericandr
 
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
Demetrio Ccesa Rayme
 
Unit 4 Computer Architecture Multicore Processor.pptx
Gunasundari Selvaraj
 
advance database management system book.pdf
hinamannan364
 
LIFE & LIVING TRILOGY - PART - (2) THE PURPOSE OF LIFE.pdf
sureshkumarsoni26
 
Computer Architecture Input Output Memory.pptx
Gunasundari Selvaraj
 
FORM 1 BIOLOGY MIND MAPS and their schemes
arcade5
 
BP 505 T. PHARMACEUTICAL JURISPRUDENCE (UNIT 2).pdf
CMEmpire
 
Module on health assessment of CHN. pptx
GurdeepSingh626704
 
David L Page_DCI Research Study Journey_how Methodology can inform one's prac...
David L Page
 
B.Sc. DS Unit 2 Software Engineering.pptx
Dr. Pallawi Bulakh
 
1.3 FINAL REVISED K-10 PE and Health CG 2023 Grades 4-10 (1).pdf
JohnJerryDalawampu
 
Skin Care and Cosmetic Ingredients Dictionary ( PDFDrive ).pdf
SahianAlondraPichard
 
Τίμαιος είναι φιλοσοφικός διάλογος του Πλάτωνα
iliaskessaris
 
MBA _Common_ 2nd year Syllabus _2021-22_.pdf
DrAnubha4
 

10. Getting Spatial

  • 1. GSP - Asian Soil Partnership Training Workshop on Soil Organic Carbon Mapping Bangkok, Thailand, 24-29 April 2017 Yusuf YIGINI, PhD - FAO, Land and Water Division (CBL)
  • 2. DAY 4 – 28 April 2017 TIME TOPIC INSTRUCTORS 8:30 - 10:30 GSOC Map Depth Classes, Spline Function Hands-on: Data Manipulation Dr. Yusuf Yigini, FAO Dr. Ate Poortinga Dr. Lucrezia Caon, FAO 10:30 - 11:00 COFFEE BREAK 11:00 - 13:00 Cont. GSOC Map Depth Classes, Spline Function 13:00 - 14:00 LUNCH 14:00 - 16:00 R and Spatial Data Spatial Operations: Points, Rasters Hands-on: Basic Spatial Operations 16:00- 16:30 COFFEE BREAK 16:30 - 17:30 Cont. R and Spatial Data Spatial Operations: Points, Rasters Hands-on: Basic Spatial Operations
  • 3. R - Getting Spatial
  • 4. Sample Data - Points We will be working with a data set of soil information that was collected from Macedonia (FYROM). https://goo.gl/EKKMAF
  • 6. > setwd("C:/mc") > pointdata <- read.csv("mc_profile_data.csv") > View(pointdata) > str(pointdata) 'data.frame': 3302 obs. of 9 variables: $ ID : int 4 7 8 9 10 11 12 13 14 15 ... $ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... $ X : int 7485085 7486492 7485564 7495075 7494798 7492500 7493700 7490922 7489842 7490414 ... $ Y : int 4653725 4653203 4656242 4652933 4651945 4651760 4652388 4651714 4653025 4650948 ... $ UpperDepth: int 0 0 0 0 0 0 0 0 0 0 ... $ LowerDepth: int 30 30 30 30 30 30 30 30 30 30 ... $ Value : num 11.88 3.49 2.32 1.94 1.34 ... $ Lambda : num 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ... $ tsme : num 0.1601 0.00257 0.0026 0.00284 0.00268 ...
  • 7. Required Packages Now we need load the necessary R packages (you may have to install them onto your computer first): > library(sp) > library(raster) > library(rgdal)
  • 8. Coordinates We can use the coordinates() function from the sp package to define which columns in the data frame refer to actual spatial coordinates—here the coordinates are listed in columns X and Y. > coordinates(pointdata) <- ~X + Y
  • 9. Coordinates > coordinates(pointdata) <- ~X + Y > str(pointdata) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: .. ..$ ID : int [1:3302] 4 7 8 9 10 11 12 13 14 15 ... .. ..$ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... .. ..$ UpperDepth: int [1:3302] 0 0 0 0 0 0 0 0 0 0 ... .. ..$ LowerDepth: int [1:3302] 30 30 30 30 30 30 30 30 30 30 ... .. ..$ Value : num [1:3302] 11.88 3.49 2.32 1.94 1.34 ... .. ..$ Lambda : num [1:3302] 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ... .. ..$ tsme : num [1:3302] 0.1601 0.00257 0.0026 0.00284 0.00268 ... ..@ coords.nrs : int [1:2] 3 4 ..@ coords : num [1:3302, 1:2] 7485085 7486492 7485564 7495075 7494798 ... .. ..- attr(*, "dimnames")=List of 2 .. .. ..$ : chr [1:3302] "1" "2" "3" "4" ... .. .. ..$ : chr [1:2] "X" "Y" ..@ bbox : num [1:2, 1:2] 7455723 4526565 7667660 4691342 .. ..- attr(*, "dimnames")=List of 2 .. .. ..$ : chr [1:2] "X" "Y" .. .. ..$ : chr [1:2] "min" "max"
  • 10. Coordinates > coordinates(pointdata) <- ~X + Y > str(pointdata) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: .. ..$ ID : int [1:3302] 4 7 8 9 10 11 12 13 14 15 ... .. ..$ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... .. ..$ UpperDepth: int [1:3302] 0 0 0 0 0 0 0 0 0 0 ... .. ..$ LowerDepth: int [1:3302] 30 30 30 30 30 30 30 30 30 30 ... .. ..$ Value : num [1:3302] 11.88 3.49 2.32 1.94 1.34 ... .. ..$ Lambda : num [1:3302] 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ... .. ..$ tsme : num [1:3302] 0.1601 0.00257 0.0026 0.00284 0.00268 ... ..@ coords.nrs : int [1:2] 3 4 ..@ coords : num [1:3302, 1:2] 7485085 7486492 7485564 7495075 7494798 ... .. ..- attr(*, "dimnames")=List of 2 .. .. ..$ : chr [1:3302] "1" "2" "3" "4" ... .. .. ..$ : chr [1:2] "X" "Y" ..@ bbox : num [1:2, 1:2] 7455723 4526565 7667660 4691342 .. ..- attr(*, "dimnames")=List of 2 .. .. ..$ : chr [1:2] "X" "Y" .. .. ..$ : chr [1:2] "min" "max" Note that by using the str function, the class of pointdata has now changed from a dataframe to a SpatialPointsDataFrame. We can do a spatial plot of these points using the spplot plotting function in the sp package.
  • 11. > setwd("C:/mc") > pointdata <- read.csv("mc_profile_data.csv") > View(pointdata) > str(pointdata) 'data.frame': 3302 obs. of 9 variables: $ ID : int 4 7 8 9 10 11 12 13 14 15 ... $ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... $ X : int 7485085 7486492 7485564 7495075 7494798 7492500 7493700 7490922 7489842 7490414 ... $ Y : int 4653725 4653203 4656242 4652933 4651945 4651760 4652388 4651714 4653025 4650948 ... $ UpperDepth: int 0 0 0 0 0 0 0 0 0 0 ... $ LowerDepth: int 30 30 30 30 30 30 30 30 30 30 ... $ Value : num 11.88 3.49 2.32 1.94 1.34 ... $ Lambda : num 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ... $ tsme : num 0.1601 0.00257 0.0026 0.00284 0.00268 ...
  • 12. Spatial Data Frame spplot(pointdata, "Value", scales = list(draw = T), cuts = 5, col.regions = bpy.colors(cutoff.tails = 0.1,alpha = 1), cex = 1) There are other plotting options available, so it will be helpful to consult the help file (?). Here, we are plotting the SOC concentration measured at each location
  • 13. Spatial Data Frame spplot(pointdata, "Value", scales = list(draw = T), cuts = 5, col.regions = bpy.colors(cutoff.tails = 0.1,alpha = 1), cex = 1) There are other plotting options available, so it will be helpful to consult the help file (?). Here, we are plotting the SOC concentration measured at each location
  • 14. Spatial Data Frame SpatialPointsDataFrame structure is essentially the same data frame, except that additional “spatial” elements have been added or partitioned into slots. Some important ones being the bounding box (sort of like the spatial extent of the data), and the coordinate reference system proj4string(), which we need to define for the sample dataset. To define the CRS, we must know where our data are from, and what was the corresponding CRS used when recording the spatial information in the field. For this data set the CRS used was: Macedonia_State_Coordinate_System_zone_7
  • 15. Coordinate Reference System To clearly tell R this information we define the CRS which describes a reference system in a way understood by the PROJ.4 projection library http://trac.osgeo.org/proj/. An interface to the PROJ.4 library is available in the rgdal package. Alternative to using Proj4 character strings, we can use the corresponding yet simpler EPSG code (European Petroleum Survey Group). rgdal also recognizes these codes. If you are unsure of the Proj4 or EPSG code for the spatial data that you have, but know the CRS, you should consult http://spatialreference.org/ for assistance.
  • 16. Spatial Data Frame > proj4string(pointdata) <- CRS("+init=epsg:6316") > > pointdata@proj4string CRS arguments: +init=epsg:6316 +proj=tmerc +lat_0=0 +lon_0=21 +k=0.9999 +x_0=7500000 +y_0=0 +ellps=bessel +towgs84=682,-203,480,0,0,0,0 +units=m +no_defs First we need to define the CRS and then we can perform any sort of spatial analysis.
  • 17. Spatial Data Frame > writeOGR(pointdata, ".", "pointdata-shape", "ESRI Shapefile") # Check your working directory for presence of this file For example, we may want to use these data in other GIS environments such as ArcGIS, QGIS, SAGA GIS etc. This means we need to export the SpatialPointsDataFrame of pointdata to an appropriate spatial data format such as a shapefile. rgdal is again used for this via the writeOGR() function. To export the data set as a shapefile:
  • 18. Spatial Data Frame > writeOGR(pointdata, ".", "pointdata-shape", "ESRI Shapefile") # Check your working directory for presence of this file For example, we may want to use these data in other GIS environments such as ArcGIS, QGIS, SAGA GIS etc. This means we need to export the SpatialPointsDataFrame of pointdata to an appropriate spatial data format such as a shapefile. rgdal is again used for this via the writeOGR() function. To export the data set as a shapefile: Note that the object we need to export needs to be a spatial points data frame. You should try opening this exported shapefile in your GIS software (ArcGIS, SAGA, QGIS...=).
  • 19. Coordinate Transformation > pointdata.kml <- spTransform(pointdata, CRS("+init=epsg:4326")) > writeOGR(pointdata.kml, "pointdata.kml", "ID", "KML") To look at the locations of the data in Google Earth, we first need to make sure the data is in the WGS84 geographic CRS. If the data is not in this CRS (which is the case for our data), then we need to perform a transformation. This is done by using the spTransform function in sp. The EPSG code for WGS84 geographic is: 4326. We can then export out our transformed pointdata data set to a KML file and visualize it in Google Earth.
  • 20. > pointdata.kml <- spTransform(pointdata, CRS("+init=epsg:4326")) > writeOGR(pointdata.kml, "pointdata.kml", "ID", "KML") To look at the locations of the data in Google Earth, we first need to make sure the data is in the WGS84 geographic CRS. If the data is not in this CRS (which is the case for our data), then we need to perform a transformation. This is done by using the spTransform function in sp. The EPSG code for WGS84 geographic is: 4326. We can then export out our transformed pointdata data set to a KML file and visualize it in Google Earth.
  • 21. KML’s > pointdata.kml <- spTransform(pointdata, CRS("+init=epsg:4326")) To look at the locations of the data in Google Earth, we first need to make sure the data is in the WGS84 geographic CRS. If the data is not in this CRS (which is the case for our data), then we need to perform a transformation. This is done by using the spTransform function in sp. The EPSG code for WGS84 geographic is: 4326. We can then export out our transformed pointdata data set to a KML file and visualize it in Google Earth.
  • 22. Coordinate Transformation > pointdata.kml <- spTransform(pointdata, CRS("+init=epsg:4326")) To look at the locations of the data in Google Earth, we first need to make sure the data is in the WGS84 geographic CRS. If the data is not in this CRS (which is the case for our data), then we need to perform a transformation. This is done by using the spTransform function in sp. The EPSG code for WGS84 geographic is: 4326. We can then export out our transformed pointdata data set to a KML file and visualize it in Google Earth.
  • 23. Sometimes to conduct further analysis of spatial data, we may just want to import it into R directly. For example, read in a shapefile (this includes both points and polygons). Now read in that shapefile that was created just before and saved to the working directory “pointdata-shape.shp”: Read Shape Files in R > pointshape <- readOGR("pointdata-shape.shp") OGR data source with driver: ESRI Shapefile Source: "pointdata-shape.shp", layer: "pointdata-shape" with 3302 features It has 7 fields
  • 24. The imported shapefile is now a SpatialPointsDataFrame, just like the pointdata data that was worked on before, and is ready for further analysis. Read Shape Files in R > pointshape@proj4string CRS arguments: +proj=tmerc +lat_0=0 +lon_0=21 +k=0.9999 +x_0=7500000 +y_0=0 +ellps=bessel +units=m +no_defs
  • 25. The imported shapefile is now a SpatialPointsDataFrame, just like the pointdata data that was worked on before, and is ready for further analysis. Read Shape Files in R > str(pointshape) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: ...
  • 26. The imported shapefile is now a SpatialPointsDataFrame, just like the pointdata data that was worked on before, and is ready for further analysis. Read Shape Files in R > str(pointshape) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: ...
  • 28. Rasters Most of the functions for handling raster data are available in the raster package. There are functions for reading and writing raster files from and to different formats. In digital soil mapping we mostly work with data in table format and then rasterise this data so that we can make a continuous map. For doing this in R environment, we will load raster data in a data frame. This data is a digital elevation model provided by ISRIC for FYROM.
  • 29. Rasters Most of the functions for handling raster data are available in the raster package. There are functions for reading and writing raster files from and to different formats. In digital soil mapping we mostly work with data in table format and then rasterise this data so that we can make a continuous map.
  • 30. For doing this in R environment, we will load raster data in a data frame. This data is a digital elevation model provided by ISRIC for FYROM. Read Rasters in R > mac.dem <- raster("covs/dem1.tif") > points <- readOGR("covs/pointshape.shp")
  • 31. For doing this in R environment, we will load raster data in a data frame. This data is a digital elevation model provided by ISRIC for FYROM. Read Rasters in R > str(mac.dem) Formal class 'RasterLayer' [package "raster"] with 12 slots ..@ file :Formal class '.RasterFile' [package "raster"] with 13 slots .. .. ..@ name : chr "C:mccovsdem1.tif" .. .. ..@ datanotation: chr "INT2S" .. .. ..@ byteorder : chr "little" .. .. ..@ nodatavalue : num -Inf .. .. ..@ NAchanged : logi FALSE .. .. ..@ nbands : int 1
  • 32. So lets do a quick plot of this raster and overlay the point locations Read Rasters in R plot(mac.dem) points(points, pch = 20)
  • 33. So lets do a quick plot of this raster and overlay the point locations Read Rasters in R plot(mac.dem) points(points, pch = 20)
  • 34. So you may want to export this raster to a suitable format to work in a standard GIS environment. See the help file for writeRaster to get information regarding the supported grid types that data can be exported. Here, we will export our raster to ESRI Ascii, as it is a common and universal raster format. Write Raster in R writeRaster(mac.dem, filename = "mac-dem.asc",format = "ascii", overwrite = TRUE)
  • 35. We may also want to export our mac.dem to KML file using the KML function. Note that we need to reproject our data to WGS84 geographic. The raster re-projection is performed using the projectRaster function. Look at the help file for this! KML is a handy function from raster for exporting grids to kml format. Write Raster in R writeRaster(mac.dem, filename = "mac-dem.asc",format = "ascii", overwrite = TRUE)
  • 36. We may also want to export our mac.dem to KML file using the KML function. Note that we need to reproject our data to WGS84 geographic. The raster re-projection is performed using the projectRaster function. Look at the help file for this! KML is a handy function from raster for exporting grids to kml format. Export Raster in KML > KML(mac.dem, "macdem.kml", col = rev(terrain.colors(255)), overwrite = TRUE)
  • 37. We may also want to export our mac.dem to KML file using the KML function. Note that we need to reproject our data to WGS84 geographic. The raster re-projection is performed using the projectRaster function. Look at the help file for this! KML is a handy function from raster for exporting grids to kml format. Export Raster in KML > KML(mac.dem, "macdem.kml", col = rev(terrain.colors(255)), overwrite = TRUE) Check your working space for presence of the kml file!
  • 38. Now visualize this in Google Earth and overlay this map with the points that we created created before Export Raster in KML
  • 39. The other useful procedure we can perform is to import rasters directly into R so we can perform further analyses. rgdal interfaces with the GDAL library, which means that there are many supported grid formats that can be read into R. Import Rasters http://www.gdal.org/formats_list.html
  • 40. Here we will load in the our .asc raster that was made just before. Import Rasters > read.grid <- readGDAL("mac-dem.asc") mac-dem.asc has GDAL driver AAIGrid and has 304 rows and 344 columns
  • 41. The imported raster read.grid is a SpatialGridDataFrame, which is a class of the sp package. To be able to use the raster functions from raster we need to convert it to the RasterLayer class. Import Rasters > str(grid.dem) Formal class 'RasterLayer' [package "raster"] with 12 slots ..@ file :Formal class '.RasterFile' [package "raster"] with 13 slots .. .. ..@ name : chr "" .. .. ..@ datanotation: chr "FLT4S" .. .. ..@ byteorder : chr "little" .. .. ..@ nodatavalue : num -Inf .. .. ..@ NAchanged : logi FALSE .. .. ..@ nbands : int 1 .. .. ..@ bandorder : chr "BIL" .. .. ..@ offset : int 0 .. .. ..@ toptobottom : logi TRUE .. .. ..@ blockrows : int 0
  • 42. It should be noted that R generated data source is loaded into memory. This is fine for small size data but can become a problem when working with very large rasters. A really useful feature of the raster package is the ability to point to the location of a raster file without loading it into the memory. Import Rasters grid.dem <- raster(paste(paste(getwd(), "/", sep = ""),"mac-dem.asc", sep = "")) > grid.dem class : RasterLayer dimensions : 304, 344, 104576 (nrow, ncol, ncell) resolution : 0.008327968, 0.008327968 (x, y) extent : 20.27042, 23.13524, 40.24997, 42.78167 (xmin, xmax, ymin, ymax) coord. ref. : NA data source : C:mcmac-dem.asc names : mac.dem
  • 44. Overlaying Soil Point Observations with Environmental Covariates
  • 45. Data Preparation for DSM In order to carry out digital soil mapping techniques for evaluating the significance of environmental variables in explaining the spatial variation of the target soil variable (for example SOC) , we need to link both sets of data together and extract raster values from covariates at the locations of the soil point data.
  • 46. Data Preparation for DSM > points class : SpatialPointsDataFrame features : 3302 extent : 20.46948, 23.01584, 40.88197, 42.3589 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 variables : 7 names : ID, ProfID, UpperDepth, LowerDepth, Value, Lambda, tsme min values : 10, P0004, 0, 30, 0.00000000, 0.1, 0.002250115 max values : 999, P6539, 0, 30, 50.33234687, 0.1, 0.160096433 > mac.dem class : RasterLayer dimensions : 304, 344, 104576 (nrow, ncol, ncell) resolution : 0.008327968, 0.008327968 (x, y) extent : 20.27042, 23.13524, 40.24997, 42.78167 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 data source : C:mccovsdem1.tif names : dem1 values : 16, 2684 (min, max)
  • 47. Data Preparation for DSM > points class : SpatialPointsDataFrame features : 3302 extent : 20.46948, 23.01584, 40.88197, 42.3589 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 variables : 7 names : ID, ProfID, UpperDepth, LowerDepth, Value, Lambda, tsme min values : 10, P0004, 0, 30, 0.00000000, 0.1, 0.002250115 max values : 999, P6539, 0, 30, 50.33234687, 0.1, 0.160096433 > mac.dem class : RasterLayer dimensions : 304, 344, 104576 (nrow, ncol, ncell) resolution : 0.008327968, 0.008327968 (x, y) extent : 20.27042, 23.13524, 40.24997, 42.78167 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 data source : C:mccovsdem1.tif names : dem1 values : 16, 2684 (min, max)
  • 48. Data Preparation for DSM > DSM_table <- extract(mac.dem, points, sp = 1,method = "simple") > DSM_table class : SpatialPointsDataFrame features : 3302 extent : 20.46948, 23.01584, 40.88197, 42.3589 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 variables : 8 names : ID, ProfID, UpperDepth, LowerDepth, Value, Lambda, tsme, dem1 min values : 10, P0004, 0, 30, 0.00000000, 0.1, 0.002250115, 45 max values : 999, P6539, 0, 30, 50.33234687, 0.1, 0.160096433, 2442 The sp parameter set to 1 means that the extracted covariate data gets appended to the existing SpatialPointsDataFrame object. While the method object specifies the extraction method which in our case is “simple” which likened to get the covariate value nearest to the points
  • 49. Data Preparation for DSM > DSM_table <- as.data.frame(DSM_table) > write.table(DSM_table, "DSM_table.TXT", col.names = T, row.names = FALSE, sep = ",") The sp parameter set to 1 means that the extracted covariate data gets appended to the existing SpatialPointsDataFrame object. While the method object specifies the extraction method which in our case is “simple” which likened to get the covariate value nearest to the points
  • 50. Data Preparation for DSM > DSM_table <- as.data.frame(DSM_table) > write.table(DSM_table, "DSM_table.TXT", col.names = T, row.names = FALSE, sep = ",") The sp parameter set to 1 means that the extracted covariate data gets appended to the existing SpatialPointsDataFrame object. While the method object specifies the extraction method which in our case is “simple” which likened to get the covariate value nearest to the points
  • 51. Using Covariates from Disc > list.files(path = "C:/mc/covs", pattern = ".tif$", + full.names = TRUE) [1] "C:/mc/covs/dem.tif" "C:/mc/covs/dem1.tif" "C:/mc/covs/prec.tif" "C:/mc/covs/slp.tif" > list.files(path = "C:/mc/covs") [1] "dem.tif" "dem1.tfw" "dem1.tif" "dem1.tif.aux.xml" "dem1.tif.ovr" [6] "desktop.ini" "pointshape.cpg" "pointshape.dbf" "pointshape.prj" "pointshape.sbn" [11] "pointshape.sbx" "pointshape.shp" "pointshape.shx" "prec.tif" "slp.tif" This utility is obviously a very handy feature when we are working with large or large number of rasters. The work function we need is list.files. For example:
  • 52. Using Covariates from Disc > list.files(path = "C:/mc/covs", pattern = ".tif$", + full.names = TRUE) [1] "C:/mc/covs/dem.tif" "C:/mc/covs/dem1.tif" "C:/mc/covs/prec.tif" "C:/mc/covs/slp.tif" > list.files(path = "C:/mc/covs") [1] "dem.tif" "dem1.tfw" "dem1.tif" "dem1.tif.aux.xml" "dem1.tif.ovr" [6] "desktop.ini" "pointshape.cpg" "pointshape.dbf" "pointshape.prj" "pointshape.sbn" [11] "pointshape.sbx" "pointshape.shp" "pointshape.shx" "prec.tif" "slp.tif" This utility is obviously a very handy feature when we are working with large or large number of rasters. The work function we need is list.files. For example:
  • 53. Using Covariates from Disc Covs <- list.files(path = "C:/mc/covs", pattern = ".tif$",full.names = TRUE) > Covs [1] "C:/mc/covs/dem.tif" "C:/mc/covs/dem1.tif" "C:/mc/covs/prec.tif" "C:/mc/covs/slp.tif" > covStack <- stack(Covs) > covStack Error in compareRaster(rasters) : different extent When the covariates in common resolution and extent, rather than working with each raster independently it is more efficient to stack them all into a single object. The stack function from raster is ready-made for this, and is simple as follow,
  • 54. Using Covariates from Disc Covs <- list.files(path = "C:/mc/covs", pattern = ".tif$",full.names = TRUE) > Covs [1] "C:/mc/covs/dem.tif" "C:/mc/covs/dem1.tif" "C:/mc/covs/prec.tif" "C:/mc/covs/slp.tif" > covStack <- stack(Covs) > covStack Error in compareRaster(rasters) : different extent If the rasters are not in same resolution and extent you will find the other raster package functions resample and projectRaster as invaluable methods for harmonizing all your different raster layers.
  • 56. Exploratory Data Analysis We will continue using the DSM_table object that we created in the previous section. As the data set was saved to file you will also find it in your working directory. > str(points) Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots ..@ data :'data.frame': 3302 obs. of 7 variables: .. ..$ ID : Factor w/ 3228 levels "10","100","1000",..: 1896 3083 3136 3172 1 66 117 141 144 179 ... .. ..$ ProfID : Factor w/ 3228 levels "P0004","P0007",..: 1 2 3 4 5 6 7 8 9 10 ... .. ..$ UpperDepth: Factor w/ 1 level "0": 1 1 1 1 1 1 1 1 1 1 ... .. ..$ LowerDepth: Factor w/ 1 level "30": 1 1 1 1 1 1 1 1 1 1 ... .. ..$ Value : num [1:3302] 11.88 3.49 2.32 1.94 1.34 ... .. ..$ Lambda : num [1:3302] 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 ...
  • 57. Exploratory Data Analysis Hereafter soil carbon density will be referred to as Value. Now lets firstly look at some of the summary statistics of SOC > summary(points$Value) Min. 1st Qu. Median Mean 3rd Qu. Max. 0.000 1.005 1.492 1.911 2.244 50.330
  • 58. Exploratory Data Analysis The observation that the mean and median are not equivalent says that the distribution of this data is not normal. > summary(points$Value) Min. 1st Qu. Median Mean 3rd Qu. Max. 0.000 1.005 1.492 1.911 2.244 50.330
  • 59. Exploratory Data Analysis The observation that the mean and median are not equivalent says that the distribution of this data seem not normal. To check this statistically, > install.packages("nortest") > install.packages("fBasics") > library(fBasics) > library(nortest) > sampleSKEW(points$Value) SKEW 0.2126149 > sampleKURT(points$Value) KURT 1.500089
  • 60. Exploratory Data Analysis Here we see that the data is positively skewed.Anderson-Darling Test can be used to test normality. > sampleSKEW(points$Value) SKEW 0.2126149 > sampleKURT(points1$Value) KURT 1.500089 > ad.test(points$Value) Anderson-Darling normality test data: points$Value A = 315.95, p-value < 2.2e-16
  • 61. Exploratory Data Analysis for normally distributed data the p value should be > than 0.05. This is confirmed when we look at the histogram and qq-plot of this data > par(mfrow = c(1, 2)) > hist(points$Value) > qqnorm(points$Value, plot.it = TRUE, pch = 4, cex = 0.7) > qqline(points$Value, col = "red", lwd = 2)
  • 62. Exploratory Data Analysis for normally distributed data the p value should be > than 0.05. This is confirmed when we look at the histogram and qq-plot of this data > par(mfrow = c(1, 2)) > hist(points$Value) > qqnorm(points$Value, plot.it = TRUE, pch = 4, cex = 0.7) > qqline(points$Value, col = "red", lwd = 2)
  • 63. Exploratory Data Analysis Most statistical models assume data is normally distributed. A way to make the data to be more normal is to transform it. Common transformations include the square root, logarithmic, or power transformations. > ad.test(sqrt(points1$Value)) Anderson-Darling normality test data: sqrt(points1$Value) A = 67.687, p-value < 2.2e-16 > sampleKURT(sqrt(points1$Value)) KURT 1.373565 > sampleSKEW(sqrt(points$Value)) SKEW 0.1148215
  • 64. Exploratory Data Analysis Most statistical models assume data is normally distributed. A way to make the data to be more normal is to transform it. Common transformations include the square root, logarithmic, or power transformations. > ad.test(sqrt(points1$Value)) Anderson-Darling normality test data: sqrt(points1$Value) A = 67.687, p-value < 2.2e-16 > sampleKURT(sqrt(points1$Value)) KURT 1.373565 > sampleSKEW(sqrt(points$Value)) SKEW 0.1148215 We could investigate other data transformations or even investigate the possibility of removing outliers or some such data..