Applied Spatial Statistics: Lecture 6 Multivariate Normal

Transcription

1 Applied Spatial Statistics: Lecture 6 Multivariate Nrmal Duglas Nychka, Natinal Center fr Atmspheric Research Supprted by the Natinal Science Fundatin Bulder, Spring 2013

2 Outline additive mdel Multivariate nrmal by a linear transfrmatin Likelihd and lg Likelihd clsed frms fr maximizing ver ρ and d cnditinal density functins applicatin t the spatial statistics prblem. 2

3 Estimating a curve r surface. The additive statistical mdel: Given n pairs f bservatins (x i, y i ), i = 1,..., n y = f + ɛ ɛ i s are randm errrs E[ɛ] = 0 COV( ɛ) = σ 2 I and f is an unknwn smth functin: f(x) = p j=1 φ j (x)d j + g(x) {φ j } knwn basis functins based n spatial lcatins e.g. plynimials r tpgraphy. d fixed cefficients but nt knwn.. g(x) a Gaussian prcess expected value f 0 and a cvariance functin ρk θ (x 1, x 2 ) frm f k is knwn expect fr sme parameter values: ρ and θ that are unknwn. 3

4 A minimum set f parameters: fixed effect linear parameters d. nugget, errr variance σ 2 sill, prcess variance ρ ther cvariance parameters such as range, scale θ The gal is t identify the statistical likelihd s we can estimate these (and pssibly thers). Flklre: d is easy just regressin λ = σ 2 /ρ and σ 2 are mre difficult with n clsed frmulas ρ mre sensitive t data and cvariance shape. θ This is hard t estimate with small sample sizes. 4

5 Multivariate nrmal cnstructin u: a vectr f N independent N(0,1)s u = u 1, u 2,..., u N independent u k Nrmal (0, 1). jint prbability density functin: h(u) = 1 (2π) N/2e (1/2) Nk=1 u 2 k = 1 u 2 (2π) N/2e ut Def 1 v = Au + µ is multivariate nrmal with expectatin µ and cvariance Σ = AA T Def 2 prbability density functin fr v is h(v) = 1 (2π) N/2e (v µ) T Σ 1 (v µ) 2 Σ 1/2 BTW This result is if and nly if Def 1 Def 2 and Def 1 Def2 5

6 Why des this wrk? In general if T (v) = u is any transfrmatin f a randm vectr then the prbability density functin fr v is h(t (v))j(v) J is the Jacbian term, the determinant (indicated by. ) f the matrix f partial derivatives: J(v) = T (u 1 ) T (u 1 ) v 1 v... 2 T (u 2 ) T (u 2 ) v 1 v T (u N ) T (u N ) v 1 v... 2 T (u 1 ) v N T (u 2 ) v N T (u 2 ) v N Stats students suffer thrugh devilishly hard change f variables prblems fr their qualifiers where T is cmplex and nnlinear When T is linear: T (v) = A 1 (v µ) = u the Jacbian is filled with cnstant elements and the determinant is simplified by sme identities. (1) 6

7 Multivariate nrmal density in R A 2-d example but reviewing matrix/vectr techniques in R. mu<- matrix( c(.1, -.5), ncl=1, nrw=2) Sigma<- rbind( c(1,.8), c(.8,1) ) v.grid<- make.surface.grid( list( x= seq( -3,3,,100), y = seq( -3,3,,100) )) z<- rep( NA, 100*100) fr( k in 1:(100*100)){ v<- v.grid[k,] # a vectr with 2 rws z[k] <- ( 1/ (2*pi)^( 2/2) ) * exp( - t(v - mu) %*% slve( Sigma) %*% (v - mu)/2 ) * det( Sigma)^(-2/2) } surface( as.surface( v.grid, z)) 7

8 the density surface y x 8

9 Likelihd fr cvariance parameters Cmbine the randm surface with the measurement errr cmpnent y MN(Xd, ρk θ + σ 2 ) X the fixed part f the mdel K cvariance matrix fr the bservatins based n cvariance functin : K i,j = k θ (x i, x j ) e.g. K i,j = ρe x i x j /θ fr the expnential prbability density functin fr y h(y) = 1 Xd)T exp (y (2π) N/2 ( ρkθ + σ 2 I ) 1 (y Xd) ρk θ + σ 2 I 1/2 2 9

10 lg Likelihd lgl(y, ρ, σ, θ, d) = (N/2) lg(2π) (y Xd) T ( ρk θ + σ 2 I ) 1 (y Xd) 2 (1/2) lg ρk θ + σ 2 I First maximize ver d clsed frm expressin: ˆd generalized least squares estimatr. Given ˆd maximize ver the ther parameters keeping in mind fr every new set f parameter values ˆd needs t be recmputed. 10

11 simplificatins First maximize ver d clsed frm expressin: ˆd generalized least squares estimatr. Given ˆd maximize ver the ther parameters keeping in mind fr every new set f parameter values ˆd needs t be recmputed. It is als pssible t maximize ver the ρ parameter in clsed frm alng with d. (1/ρ) lgl(y, ρ, λ, θ, d) = (N/2) lg(2π) [ (y Xd) T (K θ + λi) 1 (y Xd) (1/2) lg (K θ + λi) (N/2) lg(ρ) 2 ] Fr fixed λ and θ easy t shw ˆρ = (1/N) [ (y Xˆd) T (K θ + λi) 1 (y Xˆd) ] 11

12 The prfiled likelihd Substituting the prfile MLEs in fr d and ρ leaves lgl(y, ˆρ, λ, θ, ˆd) = (N/2) lg(2π) N/2 (1/2) lg K θ + λi (N/2) lg(ˆρ) Keep in mind that ˆρ and ˆd are implicitly functins f the remaining parameters. 12

13 Even when we knw the parameters we still have t d spatial predictin... Big Idea Find the cnditinal distributin f the spatial field given the bservatins. The best estimate is the cnditinal mean, uncertainty the cnditinal variance. We will get the same answer as Kriging! 13

14 Cnditinal distributins f(x, y) the jint pdf fr (X, Y )and suppse that g(x) is the pdf just fr X. f(x, y) f(y x) = g(x) The distributin f Y given the value fr X. Here X = x is bserved ( fixed) and we have a distributin fr Y. Nte that the cnditinal density is prprtinal t the jint but with the cnditining variable held fixed. A useful prperty f Multivariate nrmals is that the cnditinal distributins are als nrmal. 14

15 The tw variable nrmal Fr the bivariate nrmal v 1, v 2 with expected value µ 1, µ 2, variances σ 2 1, σ2 2 crrelatin α. f(v 2 v 1 ) is Nrmal mean : µ 2 + ασ 2 /σ 1 (v 1 µ 1 ), variance: σ 2 2 (1 α2 ) (with sme simplificatin) 15

16 Sme useful ntatin fr pdfs: [Y ] the pdf fr the randm variable Y [X, Y ] pdf fr jint distributin f X and Y [Y X] cnditinal pdf fr Y given X S the frmula fr the cnditinal is: [Y X] = [X, Y ]/[X] Als nte that [X, Y ] = [Y X][X] 16

17 Bayes Therem Bayes Therem gives a way f inverting the cnditinal infrmatin. In bracket ntatin it is just [Y X] = [X Y ][Y ] [X] The prf fllws by definitins: [Y X] = [X, Y ] [X] = [X Y ][Y ] [X] Nte that [Y X] is simply prprtinal t the jint density where the nrmalizatin depends n the values f X. (But in many cases the nrmalizatin is difficult t find.) Will cme back t this in sme later lectures 17

18 Bulder/Fraser data Bulder Bulder Fraser Bulder Fraser Z

19 Fraser jint pdf Slicing the surface Fraser Bulder Z Bulder Fraser 19

20 Cnditinal densities fr Bulder/Fraser (Y is Fraser temps and X is Bulder) Frequency [Y X=64.5] [Y X=67.5] [Y] Degrees F 20

21 Cnditinal nrmal general case Divide v int tw parts (e.g. what we bserve and what we want t predict.) v = ( v1 v 2 ) µ = ( µ1 µ 2 ) Σ = ( Σ11 Σ 12 Σ 21 Σ 22 ) Σ is divided up s cvariance fr v 1 is Σ 11, cvariance fr v 2 is Σ 22 Cnditinal frmula: [v 2 v 1 ] = N(µ 2 + Σ 2,1 Σ 1 1,1 (v 1 µ 1 ), distributin f v 2 given v 1 Σ 2,2 Σ 2,1 Σ 1 1,1 Σ 1,2) cnditinal mean depends n v 1 cnditinal cvariance des nt depend n v 1 21

22 Our applicatin is v 1 = y (the Data), µ 1 = Xd and v 2 = {g(x 1 ),...g(x N )} (pints t predict) µ 2 = 0 spatial field values where we wuld like t predict. These can be either at the bservatin lcatins r at ther pints. Srting thrugh pieces f Σ Σ 1,1 = cv(y, y) = ρk + σ 2 I i, j element f Σ 2,1 = cv(y, g) is ρk(x i, x k ) j, k element f Σ 2,2 = cv(g, g) is ρk(x j, x k ) Substituting these int the cnditinal mean and cvariance frmulas gives us the Kriging estimatr 22

23 Summary Identify a fixed part f the spatial mdel and the cvariance. Estimate the parameters by maximum likelihd pssibly prfiling t make cmputatins easier Find cnditinal mean fr g(x ) using MLE fr parameters Find predictin fr fixed part f mdel e.g. j φ(x ) ˆd j Add tw tgether fr spatial predictin at x Use Kriging cvariance as the uncertainty includes uncertainty fr ˆd 23

24 Clrad minimum MAM temps. Expnential cvariance search ver λ and θ lg Prfile Likelihd with 95% cnfidence regin (grey) 355 lg theta lg lambda MLEs: θ = 0.867, λ = and frm prfiling ρ = 0.296, σ =

25 Spatial estimate fr Lyns, CO (See R script t find the d and c cefficients explicitly) Recall ˆd fund by GLS and ĉ fund by Kriging GLS residuals. x.star<- rbind( c( , ) ) # Lyns, CO data(rmelevatin) elev.star<- interp.surface(rmelevatin, x.star) X.star<- matrix(c( 1, x.star, elev.star), nrw=1) # fixed.part<- X.star%*%d.MLE ghat.x.star<- t( exp( -rdist( xhw1,x.star)/theta.mle))%*%c.mle #NOTE: lambda frmat s left ut rh in frmula Lyns.predictin <- fixed.part + ghat.x.star Estimate: degrees C 25

26 Using fields as a black bx bj<- mkrig(x = xhw1, y = yhw1, lambda = lambda.mle, Z = elevhw1, theta = theta.mle) predict( bj, x= x.star, Z= elev.star) 26