GEOSTAT13 Spatial Data Analysis Tutorial: Raster Operations in R Tuesday AM

Transcription

1 GEOSTAT13 Spatial Data Analysis Tutorial: Raster Operations in R Tuesday AM In this tutorial we shall experiment with the R package 'raster'. We will learn how to read and plot raster objects, learn how to do calculation on cells and on groups of raster layers, learn about high level spatial operation such as aggregate, overlay, reclassify... and in section 4 will we do an worked example of constraint mapping in the form of a database query. A raster object is defined as a spatial (geographic) data structure that divides an area into rectangles. Such a data structure is also referred to as a 'grid' and is often contrasted with 'vector' data (which represents points, lines, and polygons). The raster package has functions for reading, manipulating, and writing raster data. The package is used for (1) general lowlevel raster data manipulation functions (e.g. read raster values by row, stacking images...); (2) to provide 'high level' functions for raster data manipulation that are common in other spatial data analysis software (often referred to as 'GIS') (e.g. overlay, reclassification...); (3) to work with these functions on very large raster datasets that cannot be directly loaded into memory; and (4) to provide a raster algebra implementation. Most functions in this package operate on objects of the RasterLayer class. A RasterLayer describes a single-variable raster dataset. A RasterLayer object always has values for a number of fundamental parameters such as the number of columns and rows, the coordinates of the spatial extent ('bounding box'), and the coordinate reference system (the 'map projection'), and some other parameters such as a name. In addition, a RasterLayer can store information about the name where the raster cell values are stored (if there is such a file),and it can store some or all of the raster cell values. Because a RasterLayer is a singlevariable dataset it can easily be used for raster algebra (arithmetic and other mathematical operations). A RasterStack is a collection of RasterLayer objects with the same spatial extent and resolution. In essence it is a like a list of individual RasterLayer objects. A RasterStack can easily be formed form a collection of files in different locations and mixed with RasterLayer objects that only exist in memory. There are a number of methods (functions) available for RasterStack objects. These include calc, which lets you compute summary statistics (e.g., sum or mean) for cells across all layers or perform overlay operations. A RasterLayer can easily be created from scratch using the function raster. The default settings will create a global raster data structure with a longitude/latitude coordinate reference system and 1 by 1 degree cells. You can change these settings by providing additional arguments such as nrow, ncol, to the function. These parameters can also be changed after creating the raster object. Much of this tutorial material is taken or modified from the help pages of the package RASTER that was originally written by Robert Hijmans and Jacob van Etten. The package is maintained by Robert Hijmans, University of California-Davis <r.hijmans@gmail.com>..

2 1. Initial introduction to the RASTER package Before we begin set your path to your working directory such as: setwd("c:/geostat_quebec/raster_gis/tut_4")#this should be where your ## data path Task 1: Load the library raster. library(raster) These statements generates a raster image (layer) from scratch - just try this. Things will be more clear as we proceed. rr <- raster(nrows=30, ncols=30, xmn=0, xmx=10,ymn=0,ymx=10) rr # file description rr[] <- runif(ncell(rr)) # generate a set of random values for # ncell(rr) rr[] # display file values plot(rr) # produces plot with legend It is more common; however, to create a RasterLayer objects from existing files. The raster package can read raster files in several formats, including some 'native' supported formats and other formats via the rgdal package, including ESRI (shp & grid) and GeoTiff. Task 2: Load the library and read in a tiff file: rdr <- raster("drelief.tif",native=t) # from your working directory > #rd <- writeraster(rdr,"rdrelief.tif",format='gtiff')#writes a tiff The above statement can be used if you want to write the file in a tiff format. Just remove the comment symbols if you want to write this file to your work directory. Finding information about the file is easy as is plotting the data. Task 3: Find the summary information about the structure of the file and use the generic methods plot and image to display the file. rdr plot(rdr) # plots a legend with terrain color palette #or image(rdr,asp=1) # no legend by default - notice I made the aspect = 1 Also read in the other file we shall need: rs <- raster('dsoils.tif') # reading in map of soils data There are times when we want to operate on many files or layers at a time. This is when we would use the raster object RasterStack. Create a new file by using the raster function:

3 r3 <- raster(rdr) r3[] <- runif(ncell(r3)) # generate a set of random values for ncell(r3) Task 4: Generate a raster class from a file by using another class(stack) st <- stack(r3) # setting the class stack now adding the same file multiple times, in most cases you would add different files st <- stack(st, st, st) plot(st) Not much here but it shows the stack function; now add other layers to the stack st <- addlayer(st, c(rdr, rs)) we can drop layers: (c(3,5) is a vector of the indices of the layer(s) to remove) st st <- droplayer(st, c(3, 5)) list the number of layers nlayers(st) plot(st) You can see with stacks having multiple layers or files all the layers are plotted. 2. Examples of simple calculations Many generic functions have been implemented for RasterLayer objects (however, not as many for multilayer objects), including the normal algebraic and logical operators and functions such as abs, round, ceiling, sqrt, log, log10, exp, cos, sin, max, min, range, prod, sum, any, all. These functions allows for simple and elegant raster algebra. In these functions you can mix RasterLayer objects with numbers, as long as the argument is a RasterLayer. To summarize all cells within a single RasterLayer, see cellstats and maxvalue and minvalue Summary functions (min, max, mean, prod, sum, Median, cv, range, any, all) can also be used with a RasterStack as arguments. In raster algebra, the result of a computation is always a RasterLayer. This is probably obvious when multiplying two RasterLayer objects, but perhaps this is not obvious when using functions like min, sum or mean, so use cellstats instead with a RasterLayer if you want a single number summarizing the cell values of a single RasterLayer. 2.1 Obtaining information on columns and row and cell values Task 5: Run the following code to see the various pieces of information which are available on rows, columns and cell numbers for the drelief layer: #row, column and cell number-for other functions of this type see the help page colfromcell(rdr, 110)

4 cellfromrowcol(rdr,35,35) coordinate location: xy <- xyfromcell(rdr, 17375) xy # these are real image coordinates You can also read values using cell numbers or coordinates (xy). # cell numbers cells <- cellfromrowcol(rdr, 50, 135:139) cells You can assess cell values by using getvalues. # using the soil dataset again rs1 <- rs getvalues(rs1, 50)[135:139]# notice use of square brackets Q1: Provide a table of values from the statement above for the drelief data set. In addition, you can use standard R indexing to access values. You can also replace values (or assign new values to cells). If you replace a value in a RasterLayer based on a file, the connection to that file is lost (because it now is different from the original file). Setting raster values for very large files will be very slow with this approach as each time a new (temporary) file, with all the values, are written to disk. #rs1[cells] rs1[124:128] filename(rs1)# in case you forgot the file name! rs1[2:3] <- 10 #change the cell value rs1[1:4] 2.2 Some raster calculation examples Task 6: Perform the following calculations on the raster layers drelief (rdr) rd1 <- rdr # creating a second image of rdr r3 <- rdr + rd1 # adding rdr to rd1 cell by cell r2 <- rdr / 10 # dividing rdr by 10 r3 <- rdr * (r2-1 + rdr^2 / r2)#operations done in sequence plot(r3,asp=1) # notice the change in the legend values reflecting the operations you # performed Examples of using statistics functions. Remember one of the general conditions in raster operations is that you are working with large data files. cellstats(rdr, 'mean') # calculating the file image mean The following summary methods are available for Raster* objects:mean, median, max, min,

5 range, prod, sum, any, all. For the extreme values within in a layer use maxvalue and minvalue. The following 4 rows are just for generating some sample data to use. r1 <- raster(nrow=10, ncol=10)#create file r1 <- setvalues(r1, runif(ncell(r1)))# add values to file r2 <- setvalues(r1, runif(ncell(r1))) r3 <- setvalues(r1, runif(ncell(r1))) Now we determine the maximum value and range for the three layers. rmax <- max(r1, r2, r3) # max values of 3 layers rrang <- range(r1, r2, r3) # range of 3 layers etc. These operation produce a cell by cell value of the method you are using or by building stacks for using with the 3 layers. s <- stack(r1, r2, r3) rs <- mean(s) # calculating the mean cell value plot(rs, main="mean individual cell value for the 3 layer") Examples of frequency tables: Task 7: Obtain information about the values(attribute) of drelief. rc <- rdr # back to our drelief file Frequency is an informative function providing information on the entire image. freq(rc) Examples of histograms where we obtain a plot of our frequency information. hist(rc) The function hist in the raster package when working with large data files takes a 5% sample of the raster cells.(10000 values used here). Also see the R warning! Now check to see again how many cell values (observations) exist in the data file. rc hist(rc, maxpixels=207060) Q2: Are there any differences? 3. Higher level spatial operations In each of the next subsections you will perform tasks that are key ingredients in many spatial analytical applications and projects. 3.1 Aggregate an image aggregate - a RasterLayer to create a new RasterLayer with a lower resolution (larger cells). A new value is computed for the resulting cells according to a specified function (where default value = mean).

6 ra <- aggregate(rdr, fact=9) # aggregating the data 9x/ the new value default is the mean This new aggregated raster file of ra in which its original grid is reduced by a factor of 9 (lower resolution) with the new data cell being its mean value. par(mfcol=c(2,1))# reset graphic device-two plots on page plot(rdr,main="original") plot(ra,main="aggregate 9x on original value") Q3: Do a histogram on both of these images - what difference do you see? This next example does the same amount of aggregation but a function is applied to determine the maximum value within the aggregated neighborhood plot(rdr,main="original") ra <- aggregate(rdr, fact=9, fun=max) # a new aggregated raster, max of the values plot(ra,main="aggregate 9x on max") 3.2 Reclassification of raster layers reclassify - reclassifies groups of values to other values. For example, all values between 1 and 10 become 1, and between 11 and 15 become 2. Reclassification is done with matrix- "rclmat". This matrix must have 3 columns. (You can also supply a vector that can be coerced into a n*3 matrix) The first two columns are "from" "to" of the input values, and the third column has the new value for that range. Reclassification is applied to from <= x <= to. Reclassification is done in the order of the reclass table. Thus there are overlapping rangesthe last range applies. If update=true, reclass can update values that were reclassified according to an earlier row in the reclass table. For example if row 1 has 1, 10, 15 and row 2 has 11, 20, 25, all the values from 1 to 20 will be classified as 25. We'll now generate a new raster file for use in our reclassify example. r <- raster(ncols=36, nrows=18)# new generated file r[] <- runif(ncell(r)) #fill in values to the file Task 8: Reclassify the values of the image r into three groups: 1. all values >= 0 and <= 0.25 become 1 2. all values >= 0.25 and 0.5 become 2 3. all values >= 0.5 and 1 become 3 m <- c(0, 0.25, 1, 0.25, 0.5, 2, 0.5, 1, 3) # create the reclass criteria rclmat <- matrix(m, ncol=3, byrow=true) # generate the reclass matrix rc <- reclassify(r, rclmat) # do the reclass - reclassed image is the same resolution par(mfcol=c(2,1)) plot(r,main="original") plot(rc,main="reclassed image") par(mfcol=c(1,1)) # reset graphic device to default

7 3.3 Overlay layers of raster objects overlay - creates a new RasterLayer object, based on two or more existing RasterLayers (possibly as parts of a RasterStack) of the same extent and resolution. You should supply a function fun to set the way that the RasterLayers are combined. The number of arguments in the function must match the number of RasterLayer objects. For example, if you merge two RasterLayers you could use multiply: #fun=function(x,y){return(x*y)} for percentage: #fun=function(x,y){return(100 * x / y)} #Generate three new layers r <- raster(ncol=10, nrow=10) r1 <- init(r,fun=runif) # making the cell values r2 <- init(r,fun=runif) # add two images r3 <- overlay(r1, r2, fun=function(x,y){return(x+y)}) plot(r3, main="adding two images") # multiplication r4 <- overlay(r1, r2, fun=function(x,y){return(x*y)} ) plot(r4,main="multiply two images") #use a stack s <- stack(r1, r2) r5 <- overlay(s, fun=function(x,y){return(x*y)} ) # use a single RasterLayer (same as calc function) r6 <- overlay(r1, fun=function(x){return(sqrt(x))} ) plot(r6,main="performing function on original images then overlay") # multiplication with more than two layers (make sure the number of # RasterLayers matches the arguments of 'fun' r7 <- overlay(r1, r2, r3, r4, fun=function(a,b,c,d){return(a*b+c*d)} ) plot(r7,main="multiply and adding with more than two images") # equivalent function, efficient if values can be loaded in memory r8 <- r1 * r2 + r3 * r4 plot(r8) 3.4 Examples for calculating distance from points or grids distance - calculate distance from specified point. This function calculates the distance for all cells that are NA, to the nearest cell that is not NA. The distance units are in meters if the RasterLayer is not projected (+proj=longlat) and in map units (typically also meters) when it is projected. r1 <- raster(ncol=36,nrow=18)

8 r1[] <- NA r1[350:355] <- 1 distmap <- distance(r1) plot(distmap) Q4: From the plot of the distance map-what is the approximate value of the farthest location from the original region? griddistance - calculates the distance to cells of a RasterLayer when the path has to follow the center of a neighbouring raster cells (currently only implemented as a 'queen' case). Origin cells should have a value=true (or 1), destination cells should be NA and cells that can not be traversed should be set to FALSE (or 0). r <- raster(ncol=36,nrow=18) r[] <- NA r[500] <- 1 r[48] <- 2 r[66:68] <- 3 disras <- griddistance(r,origin=2,omit=3) plot(disras) 3.5 Example of changing the geographic extent of images crop - returns a geographic subset of the input RasterLayer(changes the boundaries) specified by an Extent object (or object from which an extent object can be extracted or created). Areas in the bounding box outside of the input RasterLayer are ignored, and the extent is aligned to the cells of the new input RasterLayer. # we'll crop this new image r <- raster(nrow=45, ncol=90) r[] <- 1:ncell(r) r bb <- extent(c(-160, 10, 30, 60)) # using extent set the new dimensions of the image rc <- crop(r, bb) # crop the original image # this is a type of subsetting on location par(mfcol=c(2,1)) plot(r,main="original") plot(rc,main="cropped") The size of the plots is the same but look at the longitude and latitude of each. There are many, many other functions for use in the raster package. Please check the help pages to see the variety of operations and their use. 4. A worked example This example performs a database query/overlay as an starting point to cartographic modelling. This is the similar example that was shown in the lecture.

9 It is an example of a query by attribute Where are all the locations that have a particular attribute? We want to answer the question-how many hectares or square kilometres of land are suitable for agriculture? This involves satisfying two conditions and maybe a third but for now just two: 1. Where is the clay soil? 2. Where is the area located in the normal flood zone? # read in the two datasets used in the example. You have already read in both data sets but we'll do it again. rs <- raster('dsoils.tif') # reading in the soils data rdr <- raster("drelief.tif") This is the legend code for the soils map: 0 : No Data 1 : Heavy Clays 2 : Clays 3 : Sandy Clays 4 : Levee 5 : Stony what the data look like (again): par(mfcol=c(2,1)) plot(rdr,asp=1,main="elevation") plot(rs,col=rev(rainbow(5)),asp=1,main="soil type") par(mfcol=c(1,1)) In doing this type of analysis we have to set some criteria or constraints based on some critical numbers. Our criteria are: 1. Since this is floodplain agriculture than any local elevation greater than 9 meters is unsatisfactory. 2. Agriculture is only productive on clay soils code 2 reclassification-creating boolean images m <- c(0,9,1, 9,Inf, 0) # set up classes-elevation rcl <- reclassify(rdr,m) #reclass elevation ms <- c(0,2,0, 2,3,1, 3,999,0) #set up classes-soil rcls <- reclassify(rs,ms) #reclass soils Q5: Why is the reclassification done? now overlay ro <- rcl * rcls # overlay two boolean images(intersection) grouping the pixels for contiguous regions rclp <- clump(ro) # grouping the regions freq(rclp) # frequency cell count of grouped data

10 An attempt at a constraint map: plot(rclp,col=rev(rainbow(4)),asp=1,main="the Final Map(?)") However, we can do more - perform a zonal operation - sum the number of cells in each group ra <- zonal(ro, rclp,'sum') # summing the cells of each group then calculate the area this function calculate the area of the group grid cells #areakm <- function(x){return(x * dx * dy / 1e+06)}a function for calculating #the area in sq. KM areakm <- function(x, xro){ return (x * xres(xro) * yres(xro) / 1e+06) } ra.k <- areakm(ra[,2],ro) # calculate area in km for each group Prints out a table of areas using the cat method-not very elegant cat(paste("area for each group in KM^2"), "\n",paste("{",1:nrow(ra),"}:",sep=""),"\n",paste(ra.k),"\n") So now we could add another criteria such as an area constraint. ra1 <- as.data.frame(cbind(ra, ra.k))# merge & convert to dataframe rcarea = subs(rclp, ra1, by='zone', which='ra.k')#create map of area amounts ra.tb <- as.data.frame(ra1$ra.k[ra.k>0.25]) colnames(ra.tb) <- c("area>1/4km^2") ra.tb #reclass so final criteria is met - only plot >.25 km^2 m <- c(0, 0.25, 0, , 10, 1) rclas3 <- matrix(m, ncol=3, byrow=true) rcfin <- reclassify(rcarea, rclas3) rcfin[is.na(rcfin)] <- 0 image(rcfin, col = c("white", "black" ),main="final map-four regions") legend( "bottomleft", legend = c ("NO", "YES"),fill = c("white", "black")) With the map above and the area information from the area criteria just printed we have completed our objectives in the database query example. Q6: List the the regions which satisfy our search with their respective area(trivial)? ** Optional section: Coercion to sp objects: Although the raster package defines its own set of classes, it is easy to coerce objects of these classes to objects of the 'spatial' family defined in the sp package. This allows for using functions defined by sp (e.g. spplot) and for using other packages that expect *spatial* objects. To create a RasterLayer from variable n in a *SpatialGrid* x, use raster(x, n) or stack(x). Vice versa use as(, "spatial object")

11 > r1 <- raster(ncol=36, nrow=18) > r2 <- r1 > r1[] <- runif(ncell(r1)) > r2[] <- runif(ncell(r1)) > s <- stack(r1, r2) > sgdf <- as(s, 'SpatialGridDataFrame') > new_r2 <- raster(sgdf, 2) > new_s <- stack(sgdf) #From Robert for space time type operations in Raster: "When I have to do this kind of thing I take my stack and turn it into an array. Then I write a function to do what I want and apply that function over the array. Then remake the raster." > bar <- as.array(my.raster.stack) > foo <- apply(bar,c(1,2),function(x) { > stuff(x) > }) > foo <- raster(foo) > extent(foo) <- extent(my.raster.stack) > projection(foo) <- projection(my.raster.stack) We, also, have not mentioned any examples of using logical operation with the raster package. This is a short section covering this topic that provide some examples of logical operators. The following logical (boolean) operators are available for computations with RasterLayer objects: &,, and! In addition, these functions are available with the RasterLayer arguments: is.na, is.nan, is.finite, is.infinite. These operations return values of TRUE or FALSE. > r <- raster(ncols=10, nrows=10) > r[ ] <- runif(ncell(r)) * 10 > r1 <- r < 3 r > 6 # where r is less than 3 or greater 6 > r2 <-!r1 # the value of r1 is not r2 > r3 <- r >= 3 & r <= 6 # where r is equal to or greater than 3 and equal to or less than 6 > r4 <- r2 == r3 # r4 will be any time r2 is equal to r3 > r[r>2] <- NA # any r greater than 3 is set to NA > r5 <- is.na(r) # checking all of r for NAs (missing values) and creates variable r5 # Now we'll look at some of the results > r[1:5] > r1[1:5] > r2[1:5] > r3[1:5] > r4[1:5]

12 John Lewis > r5[1:5]