To motivate the notion of a variogram for a covariance stationary process, { Ys ( ): s R}

Transcription

1 4. Variograms Te covariogram and its normalized form, te correlogram, are by far te most intuitive metods for summarizing te structure of spatial dependencies in a covariance stationary process. However, from an estimation viewpoint suc functions present certain difficulties (as will be discussed furter in Section 4.0 below). Hence it is convenient to introduce a closely related function nown as te variogram, wic is widely used for estimation purposes. 4. Expected Squared Differences To motivate te notion of a variogram for a covariance stationary process, { Ys ( ): s R}, we begin by considering any pair of component variables, Ys Y( s) and Yv Y( v), and computing teir expected squared difference: (4..) E Y Y E Y YY Y E Y E YY E Y [( s v) ] [ s s v v] ( s) ( s v) ( v) To relate tis to covariograms, note tat if s v, ten by (3..3) and (3..4), (4..) C Y Y E Y Y EYY Y Y ( ) cov( s, v) [( s )( v )] [ s v s v ] EYY ( ) EY ( ) EY ( ) s v s v EYY ( ) EYY ( ) s v s v EYY ( ) C ( ) Exactly te same argument wit s s v v sows tat (4..3) E Y C E Y ( s ) (0) ( v ) Hence by substituting (4..) and (4..3) into (4..) we see tat expected squared differences for all sv, R wit s v can be expressed entirely in terms of te covariogram, C, as (4..4) E Y Y C C [( s v) ] [ (0) ( )] To obtain a sligtly simpler relation, it is convenient to suppress te factor by defining te associated quantity, (4..5) E Y Y sv ( ) [( s v) ], ESE 50 II.4- Tony E. Smit

2 and observing from (4..4) tat wit tis definition we obtain te following simple identity for all distances, : (4..6) ( ) C(0) C ( ) C ( ) From (4..6) it is tus evident tat te scaled expected squared differences in (4..5) define a unique function of distance wic is intimately related to te covariogram. For any given covariance stationary process, tis function is designated as te variogram,, of te process. Moreover, it is also evident tat tis variogram is uniquely constructible from te covariogram. But te converse is not true. In particular since (4..6) also implies tat (4..7) C ( ) ( ) it is clear tat in addition to te variogram,, one must also now te variance,, in order to construct te covariogram. Hence tis variance will become an important parameter to be estimated in all models of variograms developed below. Before proceeded furter wit our analysis of variograms it is important to stress tat te above terminology is not completely standard. In particular, te expected squared difference function in (4..4) is often designated as te variogram of te process, and its scaled version in (4..5) is called te semivariogram [as for example in Cressie (993, p.58-59) and Gotway and Waller (004, p.74)]. (Tis same convention is used in te Geostatistical Analyst extension in ARCMAP.) But since te scaled version in (4..5) is te only form used in practice [because of te simple identity in (4..7)] it seems most natural to use te simple term variogram for tis function, as for example in [BG, p.6]. 4. Te Standard Model of Spatial Dependence To illustrate te relation in (4..7) it is most convenient to begin wit te simplest and most commonly employed model of spatial dependence. Recall from te Ocean Dept Example in Section 3.3. above, tat te basic ypotesis tere was tat nearby locations tend to experience similar concentration levels of planton, wile tose in more widely separated locations ave little to do wit eac oter. Tis can be formalized most easily in terms of correlograms by simply postulating tat correlations are ig (close to unity) for small distances, and fall monotonely to zero as distance increases. Tis same general ypotesis applies to a wide range of spatial penomena, and sall be referred to ere as te standard model of spatial dependence. Given te relation between correlograms and covariograms in (3.3.3), it follows at once tat covariograms for te standard model, i.e., standard covariograms, must fall monotonely from C(0) toward zero, as illustrated However, assuming tat lim C ( ) 0, it follows from (4..6) tat lim ( ). So is in principle obtainable from as te asymptote (sill) in Figure 4. below. See also te lament regarding tis terminology in Scabenberger and Gotway (005, p.35). ESE 50 II.4- Tony E. Smit

3 in Figure 4. below. Te rigt end of tis curve as intentionally been left rater vague. It may reac zero at some point, in wic case covariances will be exactly zero at all greater distances. On te oter and, tis curve may approac zero only asymptotically, so tat covariance is positive at all distances but becomes arbitrarily small. Bot cases are considered to be possible under te standard model (as will be illustrated in Section 4.6 below by te sperical and exponential variogram models). sill C() () Figure 4.. Standard Covariogram Figure 4.. Standard Variogram On te rigt in Figure 4. is te associated standard variogram, wic by (4..6) above must necessarily start at zero and rise monotonely toward te value. Grapically tis implies tat te standard variogram must eiter reac te dased line in Figure 4., designated as te sill, or must approac tis sill asymptotically. 3 But wile tis matematical correspondence between te standard variogram and covariogram is quite simple, tere are subtle differences in teir interpretation. Te interpretation of standard covariograms is straigtforward, since decreases in (positive) covariance at large distances are naturally associated wit decreases in spatial dependence. But te associated increase in te standard variogram is somewat more difficult to interpret in a simple way. If we recall from (4..5) tat tese variogram values are proportional to expected squared differences, ten is reasonable to conclude tat suc differences sould increase as variables become less similar (i.e., less positively dependent). But as a general rule, it would still appear tat te simplest approac to interpreting variogram beavior is to describe tis beavior in terms of te corresponding covariogram. 4.3 Non-Standard Spatial Dependence Since te analysis to follow will focus almost entirely on te standard model, it is of interest to consider one example of a naturally occurring stationary process tat exibits non-standard beavior. As a more micro version of te Ocean Dept Example in Section 3.3. above, suppose tat one is interested in measuring variations in ocean dept due to wave action on te surface. Figure 4.3 below depicts an idealized measurement sceme 3 As noted by [BG, p.6] te scaling by ½ in (4..5) is precisely to yield a sill wic is associated wit rater tan. ESE 50 II.4-3 Tony E. Smit

4 involving a set of (yellow) cors at locations { si : i,.., n} tat are attaced to vertical measuring rods, allowing tem to bob up and down in te waves. Te set of cor eigts, Hi H( si), on tese n rods at any point of time can be treated as a sample of size n from a spatial stocastic process, { H( s): s R}, of wave eigts defined wit respect to some given ocean region, R. wave crest water level H H n s s s 4 s 6 s n d Figure 4.3. Measurement of Wave Heigts Here te fluctuation beavior of cors sould be essentially te same over time at eac location. Moreover, any dependencies among cor eigts due to te smootness of wave actions sould depend only on te spacing between teir positions in Figure 4.3. Hence te omogeneity and isotropy assumptions of spatial stationarity in Section 3.3. sould apply ere as well, so tat in particular, { H( s): s R} can be treated as a covariance stationary process. But tis process as additional structure implied by te natural spacing of waves. If tis spacing is denoted by d, ten it is clear tat for cors separated by distance d, suc as tose at locations s and s 6 in Figure 4.3, wenever a wave crest (or troug) occurs at one location it will tend to occur at te oter as well. Hence pairs of location separated by a distance d sould exibit a positive correlation in wave eigts, as sown in te covariogram of Figure 4.4 below. However, for locations spaced at around alf tis distance, suc as s and s 4 in Figure 4.3, te opposite sould be true: wenever a crest (or troug) occurs at one location, a wave troug (or crest) will tend to occur at te oter. Hence te wave eigts at suc locations can be expected to exibit negative correlation, as is also illustrated by te covariogram in Figure 4.4. Finally, it sould be clear tat distances between wave crests are temselves subject to some random variation (so tat distance d in Figure 4.3 sould be regarded as te expected distance between wave crests). Tus, in a manner similar to te standard model, one can expect tat wave eigts a distant locations will be statistically unrelated. Tis in turn implies tat te positive and negative correlation effects above will gradually ESE 50 II.4-4 Tony E. Smit

5 dampen as distance increases. Hence tis process sould be well represented by te damped sine wave covariogram sown in Figure d d Figure 4.4. Wave Covariogram Figure 4.5. Wave Variogram Finally, te associated variogram for tis process [as defined by (4..6)] is illustrated in Figure 4.5 for sae of comparison. If te variance,, in Figure 4.4 is again tae to define te appropriate sill for tis variogram (as sown by te orizontal dased line in Figure 4.5) ten it is clear tat te values of tis variogram now oscillate around te sill rater tat approac it monotonely. Hence tis sill is only meaningful at larger distances, were wave eigts no longer exibit any significant correlation. 4.4 Pure Spatial Independence A second example of a covariance stationary process, { Ys ( ): s R}, wic is far more extreme, is te case of pure spatial independence, in wic distinct random components, Ys () and Yv, () ave no relation to eac oter no matter ow close tey are in space. Matematically tis implies tat cov[ Y( s), Y( v)] 0 for all distinct s and v. But since cov[ Y( s), Y( s)] 0 for all s, tis in turn implies tat te covariogram, C, for suc a process must exibit a discontinuity at te origin, as sown on te left in Figures 4.6. ( ) C ( ) Figure 4.6. Pure Spatial Independence 4 A matematical model of tis type of covariogram is given in expression below. ESE 50 II.4-5 Tony E. Smit

6 Hence by definition, te corresponding variogram,, for pure spatial spatial independence (sown on te rigt in Figure 4.6) must also exibit a discontinuity at te origin, since (0) 0 and ( ) 0 for all 0. Suc processes are of course only matematical idealizations, since literally all pysical processes must exibit some degree of smootness (even at small scales). But if independence olds at least approximately at sufficiently small scales ten tis idealization may be reasonable. For example, if one considers a sandy desert region, R, and lets Ds () denote te dept of sand at any location, s R, ten tis migt well constitute a smoot covariance stationary process, { Ds ( ): s R}, wic is quite consistent wit te standard model of Section 3.5 (or peraps even te wave model of Section 3.6 if wind effects tend to ripple te sand). But in contrast to tis, suppose tat one considers an alternative process { W( s): s R} in wic W() s now denotes te weigt of te topmost grain of sand at location s (or peraps te diameter or quartz content of tis grain). Ten wile is it reasonable to suppose tat te distribution of tese weigts is te same at eac location s (and is tus a omogeneous process as in Section 3.3. above), tere need be little relation watsoever between te specific weigts of adjacent grains of sand. So at tis scale, te process { W( s) : s R} is well modeled by pure spatial independence. 4.5 Te Combined Model Te standard model in Section 4. and te model of pure spatial independence in Section 4.4 can be viewed as two extremes: one wit continuous positive dependence gradually falling to zero, and te oter wit zero dependence at all positive distances. However, many actual processes are well represented by a mixture of te two. Tis can be illustrated by a furter refinement of te Ocean Dept Example in Section Observe tat wile mobile organisms lie zooplanton ave some ability to cluster in response to various stimuli, te ocean also contains a ost of inert debree (dust particles from te atmospere, and seletal remains of organisms, etc.) wic bear little relation to eac oter. Hence in addition to te spatially correlated errors in sonar dept measurements created by zooplanton, tere is a general level of bacground noise created by debree particles tat is best described in terms of spatially independent errors. If tese two types of measurement errors at location s are denoted repectively by () s and () s, ten a natural refinement of te dept measurement model in (3.3.) would be to postulate tat total measurement error, ( s), is te sum of tese two components. (4.5.) () s () s () s, s R Moreover, it is also reasonable to assume tat tese error components are independent (i.e., tat te distribution of zooplanton is not influenced by te presence or absence of debree particles). More formally, it may be assumed tat () s and () v are independent random variables for every pair of locations, sv, R. Wit tis assumption it ten ESE 50 II.4-6 Tony E. Smit

7 follows (see section A. in Appendix A) tat te covariogram, C, of error process must be te sum of te separate covariograms, C and C, for component processes and, i.e., tat for any 0, (4.5.) C ( ) C( ) C( ) More generally, any covariance stationary process, { Y( s) : s R}, wit covariogram of te form (4.5.) will be said to satisfy te combined model of covariance stationary processes. Covariogram C ten represents te spatially dependent component of tis process, and covariogram C represents its spatially independent component. 5 To see te grapical form of tis combined model, observe first tat by setting 0 in (4.5.) it also follows tat (4.5.3) C(0) C (0) C (0) were and are te corresponding variances for te spatially dependent and independent components, respectively. Hence te covariogram for te combined process in (4.5.) is given by Figure 4.7 below: + = nugget effect C C C Figure 4.7. Covariogram for Combined Model In tis grapical form it is clear tat te covariogram for te combined model is essentially te same as tat of te standard model, except tat tere is now a discontinuity at te origin. Tis local discontinuity is called te nugget effect in te combined model, 6 and te magnitude of tis effect (wic is simply te variance,, of te pure independent component) is called te nugget. Note tat by definition te ratio, /, 5 Tis combined model is an instance of te more general decomposition in Cressie (993, pp.-3) wit C reflecting te smoot component, W, and C reflecting te noise components,. 6 Tis term originally arose in mining applications were tere are often microscale variations in ore deposits due to te presence of occasional nuggets of ore [as discussed in more detail by Cressie (993,p.59)]. In te present context, suc a nugget effect would be modeled as an independent micro component of a larger (covariance stationary) process describing ore deposits. ESE 50 II.4-7 Tony E. Smit

8 gives te relative magnitude of tis effect, and is designated as te relative nugget effect. For example, if te relative nugget effect for a given covariogram is say.75, ten tis would indicate tat te underlying process exibits relatively little spatial dependence. Next we consider te associated variogram for te combined model. If denotes te variogram of te combined process in (4.5.) ten we see from (4..6) togeter wit (4.5.) and (4.5.3) tat (4.5.4) ( ) C( ) ( ) [ C ( ) C ( )] [ C ( )] [ C ( )] ( ) ( ) ( ) were and are te variograms for te spatially dependent and independent components, respectively. Hence it follows tat variograms add as well, and yield a corresponding combined variogram as sown in Figure 4.8 below: sill C ( ) ( ) nugget Figure 4.8. Summary of te Combined Model 4.6 Explicit Models of Variograms Wile te combined model above provides a useful conceptual framewor for variograms and covariograms, it is not sufficiently explicit to be estimated statistically. We require explicit matematical models tat are (i) qualitatively consistent wit te combined model, and (ii) are specified in terms of a small number of parameters tat can be estimated. 7 7 Tere is an additional tecnical requirement covariograms yield well-defined covariance matrices, as detailed furter in te Appendix to Part III (Corollary.p.A3-70). ESE 50 II.4-8 Tony E. Smit

9 4.6.. Te Sperical Model Te simplest and most widely used variogram model is te sperical variogram, defined for all 0 by: (4.6.) 0, ( rsa ;,, ) a( sa), 0 r 3 r r s, r Here parameters (,, rsa ) of are assumed to satisfy rs, 0, a 0 wit s a. [Note tat te argument,, of function is separated from it parameters, (,, rsa, ) by a semicolon 8 ] To interpret tese parameters, it is useful to consider te sperical variogram sown in Figure 4.9 below wit ( r 6, s4, a ) : s s a a r Figure 4.9. Sperical Variogram Figure 4.0. Sperical Covariogram r A comparison of Figure 4.9 wit te rigt and side of Figure 4.8 sows tat parameter, s, corresponds to te sill of te variogram and parameter, a, corresponds to te nugget [as can also be seen by letting approac zero in te (4.6.)]. So for tis particular example te relative nugget effect is a/ s /4. Note finally tat since te sperical variogram reaces te sill at value, r [as can also be seen by setting r in (4.6.)], tis implies tat te corresponding covariogram in Figure 4.0 falls to zero at r. Hence te parameter, r, denotes te maximum range of positive spatial dependencies, and is 8 More generally te expression, f ( x,.., x ;,.., ), is taen to denote a function, f, wit arguments (,.., ) x x and parameters n (,.., ). n ESE 50 II.4-9 Tony E. Smit

10 designated simply as te range of te variogram (and corresponding covariogram). Tese same notational conventions for range, sill and nugget will be used trougout. 9 Te formal sperical covariogram corresponding to expression (4.6.) is immediately obtainable from (4..7) [wit s ], and is given by: (4.6.) s, C ( ; rsa,, ) ( sa), 0 r 3 r r 0, r Togeter, (4.6.) and (4.6.) will be called te sperical model. One can gain furter insigt into te nature of tis model by differentiating (4.6.) in te interval, 0 r, to obtain: (4.6.3) dc ( sa) ( s a) 3 d r r r r Hence we see tat dc (4.6.3) 0 d Moreover, by differentiating once more we see tat r (4.6.4) 3 ( sa) 0 d r r dc wenever te sill is greater tan te nugget (i.e., s a 0 ). Tus, except for te extreme case of pure independence, tis function is always bowl saped on te interval 0 r, and as a unique differentiable minimum at r. Hence tis sperical covariogram yields a combined-model form wit finite range tat falls smootly to zero. Tese properties (togeter wit its matematical simplicity) account for te popularity of te sperical model. All explicit variogram applications in tese notes will employ tis sperical model. However, it is of interest at tis point to consider one alternative model wic is also in wide use. 9 Note tat te use of s to denote sill sould not be confused wit te use of s ( s, s ) to denote spatial locations. Also, since te symbol, n, is used to denote sample size, we coose to denote te nugget by a rater tan n. ESE 50 II.4-0 Tony E. Smit

11 4.6. Te Exponential Model Wile te sperical model is smoot in te sense of continuous differentiability, it maes te implicit assumption tat correlations are exactly zero at all sufficiently large distances. But in some cases it may be more appropriate to assume tat wile correlations may become arbitrarily small at large distances, tey never vanis. Te simplest model wit tis property is te exponential variogram, defined for all 0 by, (4.6.5) 0, 0 ( ; r, s, a) 3 / r a ( s a) e, 0 wit corresponding exponential covariogram, defined for all 0 by, s, 0 (4.6.6) C ( ; rsa,, ) 3 / ( s a) e r, 0 Togeter, tis variogram-covariogram pair is designated as te exponential model, and is illustrated in Figures 4. and 4. below, using te same set of parameter values ( r 6, s4, a ) as for te sperical model above. s - a s s s - a a r r Figure 4.. Exponential Variogram Figure 4.. Exponential Covariogram Here it is clear tat te sill, s, and nugget, a, play te same role as in te sperical model. However, te range parameter, r, is more difficult to interpret in tis case since spatial dependencies never fall to zero. To motivate te interpretation of tis parameter, ESE 50 II.4- Tony E. Smit

12 observe first tat since spatial dependencies are only meaningful at positive distances, it is natural to regard te quantity s a in Figure 4. as te maximal covariance for te underlying process. 0 In tese terms, te practical range of spatial dependency is typically defined to be te smallest distance, r, beyond wic covariances are no more tan 5% of te maximal covariance. To see tat r in (4.6.6) in indeed te practical range x for tis covariogram, observe simply tat since e.05 xln(.05) , it follows tat (4.6.7) 3 / r r e r s a.05 ( ) ( )(.05) Note finally tat in terms of te corresponding variogram (wic plays te primary role in statistical estimation of te exponential model), te quantity s a in Figure 4. is usually called te partial sill Te Wave Model Finally, it is of interest to consider a matematical model of te nonstandard wave dependence example in Section 4.3 above. Here it is not surprising tat te appropriate variogram for tis wave model is given by a damped sin wave as follows, (4.6.8) 0, 0 ( rsa ;,, ) sin( / w) a ( sa) w, 0 were te parameter, w, denotes te wave intensity. Here te corresponding covariogram is given by: (4.6.9) s, 0 Crsa ( ;,, ) sin( / w) ( s a) w, 0 Te wave covariogram and variograms sown in Figures 4.4 and 4.5 above are in fact te instances of tis wave model wit ( w 0.6, a0, s 0.6). 0 More generally, tis maximal covariance for any combined model in Figure 4.7 is seen to be given by te variance,, of te (continuous) spatially dependent component. Indeed tis quantity plays suc a central role tat variograms are often defined wit te partial sill as an explicit parameter rater tan te sill itself. See for example te sperical and exponential (semi) variogram models in Cressie (993, p.6). See also te Geostatistical Analyst example in Section 4.9. below. Tis is also referred to as te ole-effect model [as in Cressie (993, p.63)], and in particular, is given tis designation in te Geostatistical Analyst riging option of ARCMAP. ESE 50 II.4- Tony E. Smit

13 4.7 Fitting Variogram Models to Data Tere are many approaces to fitting possible variogram models to spatial data sets, as discussed at lengt in Cressie (993, section.4) and Scabenberger and Gotway (004, sections ). Here we consider only te standard two-stage approac most commonly used in practice (as for example in Geostatistical Analyst). Te basic idea of tis approac is to begin by constructing a direct model-independent estimate of te variogram called te empirical variogram. Tis empirical variogram is ten used as intermediate data to fit specific variogram models. We consider eac of tese steps in turn Empirical Variograms An examination of (4..5) suggests tat for any given set of spatial data y( si ) : i,.., n and distance,, tere is an obvious estimator of te variogram value, ( ), namely alf te average value of ys ( i) ys ( j) for all pairs of locations s i and s j separated by distance. However, one problem wit tis estimator is tat (unlie K-functions) te value ( ) refers to point pairs wit distance si sj exactly equal to. Since in any finite sample tere will generally be at most one pair tat are separated by a given distance (except for data points on regular grids, as discussed below), one must necessarily aggregate point pairs ( si, s j) wit similar distances and ence estimate ( ) at only a small number of representative distances for eac aggregate. Te simplest way to do so is to partition distances into intervals, called bins, and tae te average distance,, in eac bin to be te appropriate representative distances, called lag distances, as sown in Figure 4.3 below: 0 bins 3 4 max lag lag distances Figure 4.3. Lag Distances and Bins More formally, if N denotes te set of distance pairs, ( si, s j), in bin, [wit te size (number of pairs) in N denoted by N ], and if te distance between eac suc pair is denoted by ij si sj, ten te lag distance,, for bin is defined to be ESE 50 II.4-3 Tony E. Smit

14 (4.7.) N ( si, sj) N ij To determine te size of eac bin, te most common approac is to mae all bins te same size, in order to insure a uniform approximation of lag distances witin eac bin. However tere is an implicit tradeoff ere between approximation of lag distances and te number of point pairs used to estimate te variogram at eac lag distance. Here te standard rule of tumb is tat eac bin sould contain at least 30 point pairs, 3 i.e., tat (4.7.) N 30 Next observe tat te coice of te maximum lag distance (max-lag),, (in Figure 4.3) also involves some implicit restrictions. First, for any given set of sample points, { si : i,.., n} R, one cannot consider lag distances greater tan te maximum pairwise distance, (4.7.3) max max si sj : i j n in tis sample since no observations are available. Moreover, practical experience as sown tat even for lag distances close to max te resulting variogram estimates tend to be unstable [Cressie (985, p.575)]. Hence, in a manner completely analogous to te rule of tumb for K-functions [expression (4.5.) of Part I], it is common practice to restrict to be no greater tan alf of max, i.e., max (4.7.4) Hence our basic rule for constructing bins is coose a system of bins { N :,.., } of uniform size, suc tat te max-lag,, is as large as possible subject to (4.7.3) and (4.7.4). More formally, if te biggest distance in eac bin is denoted by d d, ten our procedure (in te MATLAB program variogram.m max( s i, s j ) N ij discussed below) is to coose a maximum bin number,, and maximum distance (maxdist), d, suc tat 4 (4.7.5) N N 30 (4.7.6) d d 3 Notice tat tis rule of tumb is reminiscent of tat for te Central Limit Teorem used in te Clar- Evans test of Section 3.. in Part I (and in Section 3..3 above). Note also tat some autors recommend tere be at least 50 pairs in eac bin [as for example in Scabenberger and Gotway (005, p.53)]. 4 Tis is essentially a variation on te practical rule suggested by Cressie (985, p.575). ESE 50 II.4-4 Tony E. Smit

15 (Here te default value of d is max / and te default value of is 00 bins.) Wit tese rules for constructing bins and associated lag distances, it ten follows from (4..5) tat for any given set of sample points, { si : i,.., n} R, wit associated data, { y( s ): i,.., n}, an appropriate estimate of te variogram value, ( ), at eac lag i distance, all point pairs ( si, s j) in N, i.e.,, is given by alf te average squared differences ys ys ( ) ( ) over i j (4.7.7) ˆ( ) y( s ) y( s ) i j ( si, sj) N N Tis set of estimates at eac lag distance is designated as te empirical variogram. 5 More formally, if for any given set of (ordered) lag distances, { :,.., }, te associated variogram estimates in (4.7.7) are denoted simply by ˆ ˆ( ), ten te empirical variogram is given by te set of pairs {(, ˆ ) :,.., }. An scematic example of tis empirical variogram construction is given in Figure 4.4 below: ys ( ) ( ) i ysj ˆ ˆ ij Figure 4.4 Empirical Variogram Construction Here te blue dots correspond to squared-difference pairs, ys ( ) ys ( ) ESE 50 II.4-5 Tony E. Smit i j, plotted against distances, ij si sj, for eac point pair, ( si, s j), [as illustrated for one point in te lower left corner of te figure]. Te vertical lines separate te bins, as sown for bins 5 Te empirical variogram is also nown as Materon s estimator, in onor of its originator [Scabenberger and Gotway (005, Section 4.4.)].

16 and +. So in bin, for example, tere is one blue dot for every point pair, ( si, sj) N. Te red dot in te middle of tese points denotes te pair of average values, (, ˆ ), representing all points in tat bin. Hence te empirical variogram consists of all tese average points, one for eac bin of points. [Scematics of suc empirical variograms are sown (as blue dots) in Figure 4.5 below. An actual example of an empirical variogram is sown in Figure 4.9 below.] Wile tis empirical variogram will be used to fit all variograms in tese notes, it sould be mentioned tat a number of modifications are possible. First of all, wile te use of average distances,, in eac bin as certain statistical advantages (to be discussed below), one can also use te median distance, or simply te midpoint of te distance range. Similarly, wile uniformity of bin sizes in (4.7.5) will also turn out to ave certain statistical advantages for fitting variograms in our framewor (as discussed below), one can alternatively require uniform widts of bins. In addition, it as been observed by Cressie and Hawins (980) [also Cressie (993, Section.4.3)] tat estimates involving squared values suc as (4.8.7) are often dominated by a few large values, and are tus sensitive to outliers. Hence tese autors propose several robust alternatives to (4.7.7) based on square roots and median values of absolute differences. Finally it sould be noted tat a number of fitting procedures in use actually drop tis initial stage altogeter, and fit variogram models directly in terms of te original data, { y( si ): i,.., n}. 6 In suc approaces, te empirical variogram is essentially replaced by a completely disaggregated version called te variogram cloud, were eac point pair (, ) s s is estimated by te si s j is treated as a separate bin, and were ij i j ˆ ij ys ( i) ys ( j ). 7 Wile tis approac can in many cases be more powerful statistically, it generally requires stronger modeling assumptions. Moreover, it turns out tat suc metods are not only very sensitive to tese modeling assumptions, but can also be less stable for smaller data sets. Finally, and most important from practical viewpoint, plots of te empirical variogram tend to be visually muc more informative tat plots of te entire variogram cloud, and in particular, can often elp to suggest appropriate model forms for te variogram itself. [An example is given in Figure 4.0 below.] Hence we coose to focus on te classical empirical-variogram approac. 8 single sample, 6 Most prominent among tese is te metod of maximum lieliood, as detailed for example in Scabenberger and Gotway (005, Section 4.5.). [Tis general metod of estimation will also be developed in more detail in Part III of tese notes for fitting spatial regression models.] 7 An example is given in Figure 4.9 below. 8 For additional discussion see te section on Binning versus Not Binning in Scabenberger and Gotway (005, Section ). See also te excellent discussion in Reilly and Gelman (007). ESE 50 II.4-6 Tony E. Smit

17 4.7. Least-Squares Fitting Procedure Given an empirical variogram, {(, ˆ ) :,.., }, togeter wit a candidate variogram model, ( ; r, s, a) [suc as te sperical model in (4.7.)], te tas remaining is to find parameter values, (,, rsa, ˆˆˆ) for tis model tat yield a best fit to te empirical variogram data. Te simplest and most natural approac is to adopt a least squares strategy, i.e., to see parameter values, (,, rsa, ˆˆˆ) tat solve te following (nonlinear) least-squares problem: (4.7.8) ˆ min ( ; r, s, a) ( rsa,, ) Wile tis procedure will be used to fit all variograms in tese notes, it is important to note some sortcomings of tis approac. First of all, since squared deviations are being used in (4.7.8), it again follows tat tis least-squares procedure is sensitive to outliers. As wit all least-squares procedures, one can attempt to mitigate tis problem by using an appropriate weigting sceme, i.e., by considering te more general weigted leastsquares problem: (4.7.9) ˆ min w ( ; r, s, a) ( rsa,, ) for some set of appropriate nonnegative weigts w :,.., for tese weigts [first proposed by Cressie (985)] is to set: 9 N (4.7.0) w,,.., ( ; r, s, a). A very popular coice Here te numerator simply places more weigt on tose terms wit more samples. Te denominator is approximately proportional to te variance of te estimates, ˆ, 0 so tat te effect of bot te numerator and denominator is to place more weigt on tose terms for wic te estimates, ˆ, are most reliable. However, it as been pointed out by oters tat te inclusion of te unnown parameters ( rsa,, ) in tese weigts can create certain instabilities in te estimation procedure [see for example Zang et al. (995) and Müller (999, Section 4)]. Moreover, since our constant bin sizes in (4.7.5) eliminate variation in te sample weigts, we coose to use te simpler unweigted least-squares procedure in (4.7.8). 9 In particular, tis is te weigted least-squares procedure used in Geostatistical Analyst. 0 Tis approximation is based on te important case of normally distributed spatial data. ESE 50 II.4-7 Tony E. Smit

18 Finally it sould also be noted tat tis least-square procedure is implicitly a constrained minimization problem since it is required tat (i) r 0 and (ii) s a 0. In te present setting, owever, nonnegativity of bot r and s is essentially guaranteed by te nonnegativity of te empirical variogram itself. But nonnegativity of te nugget, a, is muc more problematic, and can in some cases fail to old. Tis is illustrated by te scematic example sown on te left in Figure 4.5 below, were a sperical variogram model (red curve) as been fitted to a set of ypotetical empirical variogram data (blue dots). Here it is clear tat te best fitting sperical variogram does indeed involve a negative value for te estimated nugget, â. â Figure 4.5. Negative Nugget Problem Hence in suc cases, it is natural to impose te additional constraint tat a 0, and ten solve te reduced minimization problem in te remaining unnown parameters, ( rs, ) : (4.7.) ˆ min ( ; r, s,0) ( rs, ) Te solution to tis reduced problem, sown scematically above will yield te closest approximation to te solution of (4.8.8) wit a feasible value for te nugget, a. It is tis two-stage fitting procedure tat will be used (implicitly) wenever nuggets are negative. 4.8 Te Constant-Mean Model Our next objective is to develop a practical illustration of variogram estimation. But to do so, it is important to begin by recalling tat covariance stationarity was originally motivated in te context of our general modeling framewor in Section. above, were it was assumed tat spatial random variables are of te form: (4.8.) Ys () () s () s, s R ESE 50 II.4-8 Tony E. Smit

19 and were covariance stationarity is actually a property of te unobserved residual process, (): s s R. Hence variogram estimation for any given set of spatial data, ys ( i) : i,.., n, must generally been done as part of a larger modeling effort in wic bot te variogram and te spatial trend function (): s s R are modeled explicitly. One can ten consider iterative fitting procedures in wic te spatial trend function is first fitted from te data, say by ˆ( s ) : i,.., n, to yield residual estimates, (4.8.) ˆ ( s ) y( s ) ˆ ( s ), i,.., n i i i i tat are in turn used to fit te variogram model. Muc of te present section on Continuous Spatial Data Analysis will be devoted to tis larger modeling-and-estimation problem. Hence to develop a meaningful example of variogram estimation at tis point, it is necessary to mae stronger assumptions about te general framewor in (4.9.) above. In particular, we now assume tat te entire process Y(): s s R is itself covariance stationary. By (3..6) troug (3..8) tis equivalent to assuming tat in addition to covariance stationarity of te residual process in te second term of (4.8.), te spatial trend function in te first term is constant, so tat (4.8.3) Ys ( ) ( s), s R for some (possibly unnown) scalar,. Under tese conditions it follows at once tat (4.8.4) E Y() s Y() v E () s () v E () s () v for all sv, R, so tat by definition te variograms for te Y -process and te -process are identical: (4.8.5) ( Y ) ( ), 0 Hence, under tese assumptions we see tat for any given spatial data, y( si ) : i,.., n, te residual variogram,, can be estimated directly in terms of te empirical variogram, ˆ ( ) y( s ) y( s ),,.., ( si, sj) N( ) N ( ) (4.8.6) Y i j for te observable Y -process. Tis approac will be illustrated in te following example. ESE 50 II.4-9 Tony E. Smit

20 4.9 Example: Nicel Deposits on Vancouver Island Te following example is taen from [BG, pp.50-5] and is based on sample data from Vancouver Island in Britis Columbia collected by te Geological Survey of Canada. Tis data set [contained in te ARCMAP file ( \projects\nicel\nicel.mxd)], extends over te area at te nortern tip of te island sown in Figure 4.6 below. Te area outlined in red denotes te full extent of te data site. For purposes of tis illustration, a smaller set of 436 sample sites was selected, as sown by te dots in Figure m Figure 4.6. Vancouver Sample Area Figure 4.7. Vancouver Sample Area Note te curvilinear patterns of tese sample points. As wit many geocemical surveys, samples are ere taen mainly along stream beds and lae sores, were minerals deposits are more liely to be found. In particular, samples of five different ore types were collected. Te present application will focus on deposits of Nicel ore. [In class Assignments 3 and 4 you will study deposits of Cobalt and Manganese at sligtly different site selections.] Tis Nicel data is sown in te enlarged map below, were Nicel concentration in water samples is measured in parts per million (ppm). Nicel (ppm) Figure 4.8. Nicel Data ESE 50 II.4-0 Tony E. Smit

21 Since te mapped data exibits strong similarities between neigboring values (at tis pysical scale), we can expect to find a substantial range of spatial dependence in tis data. Notice owever tat te covariance-stationarity assumption of Isotropy in (3.3.5) [and (3.3.3)] is muc more questionable for tis data. Indeed tere appear to be diagonal waves of ig and low values rippling troug te site. An examination of Figure 4.6 above sows tat tese waves are rougly parallel to te Pacific coastline, and would seem to reflect te istory of continental drift in tis region. Hence our present assumption of covariance stationarity is clearly an over-simplification of tis spatial data pattern. We sall see tis more clearly in te variogram estimation procedure to follow Empirical Variogram Estimation Given tese n 436 sites ( si : i,.., n) togeter wit teir corresponding nicel measurements, yi ys ( i), our first objective is to construct an empirical variogram for tis data as in (4.8.5) above. Tis procedure is operationalized in te MATLAB program, variogram_plot.m. To use tis program, te data from Nicel.mxd as been imported to te MATLAB worspace file, nicel.mat. Te 436 x 3 matrix, nicel, contains te coordinate + nicel data ( si, si, y i) for eac location i,.., n. By opening te program, variogram_plot.m, it can be seen tat a matrix of tis form is te first required input. Next, recall from Section 4.7. tat along wit tis data, tere are two inputs for defining an appropriate set of distance bins, namely te maximum bin number,, and te maximum distance (max-dist), d. Tese parameter options are specified in an opts structure (similar to tat in te program clust_sim.m of Section 3.5 in Part I). Here we sall start wit te default values, 00, and d / 48,03 max meters, so tat tere is no need to specify tis structure. Hence by typing te simple command: >> variogram_plot(nicel); one obtains a plot of te empirical variogram, as sown in Figure 4.9 below x 0 4 Figure 4.9. Empirical Variogram Figure 4.0. Variogram Cloud In fact tese waves are almost mirror images of te Cascadia subduction zone tat follows te coastline immediately to te west of Vancouver Island. ESE 50 II.4- Tony E. Smit

22 Here te point scatter does rise toward a sill, as in te classical case illustrated in Figure 4.8 above. So it appears tat one sould obtain a reasonable fit using te sperical model in Figure 4.9 [from expression (4.6.)]. But before fitting tis model, tere are a number of additional observations to be made. First, for purposes of comparison, te corresponding variogram cloud is plotted in Figure 4.0. Notice first tat wile te orizontal (distance) scales of tese two figures are te same, te vertical (squared difference) scales are very different. In order to include te full point scatter in te variogram cloud, te maximum squared-difference value as been 4 increased from 000 in Figure 4.9 to around 0,000 ( 0 ) in Figure 4.0. For visual comparison, te value 000 is sown by a red arrow in bot figures. So wile te empirical variogram does indeed loo classical in nature, it is difficult to draw many inferences about te sape of te true variogram from te wider scatter of points exibited by te variogram cloud. Te reason for tis is tat wile te empirical variogram sows mean estimates of te variogram at 00 selected lag distances, te variogram cloud contains te squared y-differences for eac of te 70,687 individual pairs, ( si, s j), wit dij d. Hence about all tat can be seen from tis cloud of points is tat tere are a considerable number of outliers tat are very muc larger tan te mean values at eac distance. But fortunately tis pattern of outliers is fairly uniform across te distance spectrum, and ence sould not seriously bias te final result in tis particular case. On te oter and, if outliers were more concentrated in certain distance ranges (as is often typical for te larger distance values), ten tis migt indicate te need to trim some of tese outliers before proceeding. In sort, wile te variogram cloud may provide certain useful diagnostic information, te empirical variogram is usually far more informative in terms of te possible sapes of te true variogram. Next, it sould be noted tat in addition to te variogram plot, one obtains te following screen output MAXDIST = wic is precisely d above. To compare tis wit te max-lag distance,, note first tat tere are a number of optional outputs for tis program as well. First, te actual values of te empirical variogram, {(, ˆ ) :,.., }, are contained in te matrix, DAT, were eac row contains one (, ˆ ) pair. Tis can be seen by running te full command, >> [DAT,maxdist,bin_size,bin_last] = variogram_plot(nicel); and ten clicing on te matrix, DAT, in te worspace to display te empirical variogram. In particular, te value corresponds to te last element of te first column and can be obtained wit te command [ >> DAT(end,) ] yielding Tis is smaller tan d since is somewere in te middle of te last bin (as in Figure 4.3 above), and d is by definition te outer edge, d, of tis last bin. Tis was constructed using te MATLAB program, variogram_cloud_plot.m. ESE 50 II.4- Tony E. Smit

23 As for te additional outputs, maxdist is precisely te screen output above, and te value, bin_size = 707, tells you ow many point pairs tere are in eac bin [as in condition (4.7.5) above]. In tis application tere are many more tan 30 point pairs in eac bin, so tat te maximum number of bins, 00, is precisely te number realized. However, if te number of sample points ad been sufficiently small, ten bin_size = 30, would be a binding constraint in (4.7.5), and tere could well be fewer tan 00 bins. 3 Finally, te value, bin_last, is simply a count of points in te last bin, to cec weter it is significantly smaller tan te rest. Tis will only occur if d is cosen to be very close to te maximum pairwise distance, max, and ence will rarely occur in practice. 4 As one last observation, recall from te wave pattern in Figure 4.7 above tat one may as weter tis effect is piced up by te empirical variogram at larger distances. By using te measurement tool in ARCMAP and tracing a diagonal line in te direction of tese waves (from lower left to upper rigt), it appears tat a reasonable value of maxdist to try is d 80,000 meters. To do so, we can run te program wit tis option as follows: >> opts.maxdist = 80000; >> variogram_plot(nicel,opts); We ten obtain te empirical variogram in Figure 4.b, were te previous variogram as been repeated in Figure 4.a for ease of comparison: x x 0 4 Figure 4.a. Max Distance = 48,03 Figure 4.b. Max Distance = 80,000 3 For example if n = 50 so tat te number of distinct point pairs is 50(49)/ = 56 < 30(00), ten tere would surely be fewer tan 00 bins. 4 For example, if one were to set opts.maxdist = 95000, wic is very close to in te present max example, ten te last bin will indeed ave fewer points tan te rest. ESE 50 II.4-3 Tony E. Smit

24 Notice tat wile te vertical (squared difference) scales for tese two figures are te same, te orizontal distance scales are now different (reflecting te different maximum distances specified). Moreover, wile te segment of Figure 4.b up to 50,000 4 ( 5 0 ) meters is qualitatively similar to Figure 4.0a, te bins and corresponding lag distances are not te same as in Figure 4.a. Hence it is more convenient to sow separate plots of tese two empirical variograms rater tan try to superimpose tem on te same scale. Given tis scale difference, it is noneteless clear tat te sligt dip in te empirical variogram on te left, starting at about 40,000 meters, becomes muc more pronounced at te larger lag distances sown on te rigt. Recall (from te corresponding covariograms) tat tis can be interpreted to mean tat pairs of y-values (nicel measurements) separated by more tan 40,000 meters tend to be more similar (positively correlated) tan tose separated by sligtly smaller distances. Finally, by again using te measurement tool in ARCMAP, it can be seen tat te spacing of successive waves is about 40,000 meters. So it does appear tat tis effect is being reflected in te empirical variogram. As a final caveat owever, it sould be empasized tat te most extreme dip in Figure 4.b occurs at lag distances close to max, were variogram estimates tend to be very unreliable. In addition, tere are edge effects created by tis rectangular sample region tat may add to te unreliability of comparisons at larger distances Fitting a Sperical Variogram Recall from Section 4.6. above tat all variogram applications in tese notes (as well as te class assignments) will involve fitting sperical variogram models to empiricalvariogram data. [Oter models can easily be fitted using te Geostatistical Analyst (GA) extension in ARCMAP, as illustrated below.] For purposes of te present application, we sall adere to te restriction in (4.7.4) tat d not exceed max /, and ence sall use only te empirical variogram in Figure 4.9 (and 4.a) constructed under tis condition. To fit a sperical variogram model to tis empirical-variogram data, we sall use te simple nonlinear least-squares procedure in (4.7.8) above. Fitting Procedure using MATLAB Tis is operationalized in te MATLAB program, var_sper_plot.m. 5 Since tis program uses exactly te same inputs as tose detailed for variogram_plot.m in Section 4.9. above, tere is no need for furter discussion of inputs. Hence a sperical variogram model can be fitted in te present application wit te command: >> var_sper_plot(nicel); Te first output of tis fitting procedure is te sperical variogram plot sown in Figure 4. below, were te blue dots are te empirical variogram points, and te estimated 5 One can also use te weigted nonlinear least-squares procedure in (4.8.9) and (4.8.0) above, wic is programmed in var_sper_wtd_plot.m. ESE 50 II.4-4 Tony E. Smit

25 sperical variograms is sown in red. If you clic Enter again you will see te associated covariogram plot, as sown in Figure 4.3 below VARIOGRAM PLOT COVARIOGRAM PLOT x Fitted Sperical Variogram 4.3 Derived Sperical Covariogram x 0 4 Here it must be empasized tat tis covariogram is not being directly estimated. Rater, te estimates (,, rsa ˆˆˆ) obtained for te sperical variogram are substituted into (4.6.) in order to obtain te corresponding covariogram. Hence it is more properly designated as te derived sperical covariogram. Similarly, te blue dots sown in tis figure are simply an inverted reflection of te empirical variogram sown in Figure 4.. However, tey can indeed be similarly interpreted as te derived empirical covariogram corresponding to te empirical variogram in Figure 4.. To do so, recall first from (4..7) tat for all distances,, it must be true tat C ( ) ( ). But since eac empirical variogram point ( ˆ, ) by definition yields an estimate of ( ), namely ˆ ˆ( ), and since te sill value, ŝ, is by definition an estimate of, i.e., sˆ ˆ, it is natural to use (4..7) to estimate te covariogram at distance by (4.9.) ˆ ˆ ˆ ˆ ˆ C ( ) ( ) s ( ) Hence by letting C ˆ ˆ( C ), it follows tat te set of points, {(, ˆ C) :,.., }, obtained is precisely te derived empirical covariogram in Figure 4.3 corresponding to te empirical variogram, {(, ˆ ) :,.., }, in Figure As mentioned earlier, te advantage of displaying tis derived covariogram is tat it is muc easier to interpret tat te estimated variogram. To do so, we begin by noting tat in addition to tese two diagrams, te program var_sper_plot.m also yields a screen 6 In particular, te vertical component,, of eac variogram point ˆ new value, C ˆ sˆ ˆ. (, ˆ ) as simply been sifted to te ESE 50 II.4-5 Tony E. Smit

26 display of te parameter estimates (,, rsa ˆˆˆ) [along wit maxdist, d, and te number of iterations in te optimization procedure 7 ], as sown in Figure 4.4 below. SPHERICAL VARIOGRAM: RANGE SILL NUGGET MAXDIST ITERATIONS = 6 Figure 4.4. Parameter Estimates In particular, te RANGE ( rˆ meters) denotes te distance beyond wic tere is estimated to be no statistical correlation between nicel values. 8 In Figure 4., tis corresponds to te distance at wic te variogram first reaces te sill. But tis offers little in te way of statistical intuition. In Figure 4.3 on te oter and, it is clear tat tis is te distance at wic covariance (and ence correlation) first falls to zero. Tis is te ey difference between tese two representations. Notice also tat te vertical axis in Figure 4.3 as been sifted relative to Figure 4., in order to depict te negative covariance values in te cluster of values around te zero line. Turning to te oter estimated parameters, note first from Figure 4.3 tat te SILL ( sˆ ) is seen to be precisely te estimated variance of individual nicel values (i.e., te estimated covariance at zero distance ). Similarly, te NUGGET ( aˆ ) is seen to be tat part of te individual variance tat not related to spatial dependence among neigbors. Since in tis case te relative nugget effect, ( = / ), is well below 0.5, it is evident tat tere is a substantial degree of local spatial dependence among nicel values. So in summary, it sould be clear tat wile te variogram model is useful for obtaining tese parameter estimates, (,, rsa, ˆˆˆ) te derived covariogram model is far more useful for interpreting tem. Fitting Procedure using ARCMAP Before proceeding, it is of interest to compare tis estimated sperical variogram wit te fitting procedure used in te Geostatistical Analyst (GA) extension in ARCMAP (Version 0). Te results of tis procedure applied to te nicel data in te ARCMAP file, nicel.mxd, are sown in Figure 4.5 below. 7 Note tat if ITERATIONS exceeds 600, you will get an error message telling you tat te algoritm failed to converge in 600 iterations (wic is te default maximum number of iterations allowed). 8 Notice also tat tis RANGE value is considerably below te MAXDIST ( meters), indicating tat te range of spatial dependence among nicel values is well captured by tis empirical variogram ESE 50 II.4-6 Tony E. Smit