An explicit link between Gaussian fields and Gaussian Markov random fields; the stochastic partial differential equation approach

Transcription

1 Intro B, W, M, & R SPDE/GMRF Example End An explicit link between Gaussian fields and Gaussian Markov random fields; the stochastic partial differential equation approach Finn Lindgren 1 Håvard Rue 1 Johan Lindström 2 1 Department of Mathematical Sciences Norwegian University of Science and Technology, Trondheim, Norway 2 Mathematical Statistics Centre for Mathematical Sciences, Lund University, Lund, Sweden RSS, London, 16th March 2011

2 Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Computationally efficent spatial models Spatial random field models Dense covariance: x N (µ, Σ) = expensive computations Sparse precision: x N (µ, Q 1 ) = efficient computations Leukaemia survival in NW England

3 Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations lattice CAR models We need models with well-defined continuous interpretation

4 Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations triangular graph We need models with well-defined continuous interpretation

5 Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations refined triangulation We need models with well-defined continuous interpretation

6 Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations smooth triangulation We need models with well-defined continuous interpretation

7 Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations continuous space We need models with well-defined continuous interpretation

8 Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle Conditional autoregression models on lattices GMRF Covariance CAR(2) κ u K 1 (κ u ) Whittle (1954) CAR(1) 1 2π K 0(κ u ) Besag (1981) ICAR(1) 1 2π log( u ) Besag & Mondal (2005)

9 Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle Matérn covariances (Bertil Matérn) The Matérn covariance family Cov(x(0), x(u)) = σ 2 21 ν Γ(ν) (κ u )ν K ν (κ u ), u R d Scale κ > 0, smoothness ν > 0, variance σ 2 > 0 Are any other discrete domain Markov fields close to Matérn fields? Are any Matérn fields continuous domain Markov fields?

10 Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle The continuous domain Markov property S is a separating set for A and B: x(a) x(b) x(s) A S B

11 Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle Identifying a Markov field by its spectrum Spectral representation: x(u) = exp(iu k) x(k) dk R d Spectrum: S x (k) = E( x(k) 2 ) Rozanov (1977) x(u) is Markov S x (k) P(k) 1 where P(k) is a non-negative symmetric polynomial What is the spectrum of a Matérn field? c MFO

12 Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle The Whittle-Matérn connection Whittle (1954, 1963) Matérn fields on R d are stationary solutions to the SPDE ( κ 2 ) α/2 x(u) = W(u), α = ν + d/2 The spectrum is S x (k) = (2π) d (κ 2 + k 2 ) α Therefore Matérn fields are Markov when ν + d/2 is an integer. Laplacian: = d 2 u 2 i=1 i

13 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Piecewise linear representations x(u) = cos(u 1 ) + sin(u 2 ) x(u) = k ψ k(u) x k Guiding principle Attack the SPDE with local finite dimensional representations instead of covariances or kernels (subsets of Green s functions)!

14 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond The best piecewise linear approximation k ψ k(u) x k Projection of the SPDE: Linear systems of equations (α = 2) j (κ 2 ψ i, ψ j + ψ i, ψ j } {{ } } {{ } C ij G ij )x j D = ψ i, W jointly for all i. C and G are as sparse as the triangulation neighbourhood Constructing the precision matrices K = κ 2 C + G α = 1 α = 2 α = 3, 4,... Kx N (0, K) N (0, C) ( ) N 0, CQ 1 x,α 2 C Q x,α K K T C 1 K K T C 1 Q x,α 2 C 1 K

15 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Markov approximation C 1 is dense, which leads to non-markov models! The diagonal matrix C, C i,i = ψ i, 1 = the local cell area, gives a sparse precision with the total energy preserved.

16 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Lattice on R 2, node distance h, regular triangulation Order α = 1: generalised covariance 1 2π K 0(κ u ) 1 κ 2 h Order α = 2: covariance 1 4πκ 2 κ u K 1 (κ u ) κ 4 h κ h

17 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 ) α/2 (τ x(u)) = W(u), u R d

18 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 e iπθ ) α/2 (τ x(u)) = W(u), u R d

19 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 + m M ) α/2 (τx(u)) = W(u), u R d

20 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u R d

21 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω

22 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω

23 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω

24 Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. ( t + κ2 u,t + m u,t M u,t ) (τ u,t x(u, t)) = E(u, t), (u, t) Ω R

25 Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Large spatial problems: covariances are impractical Data points n Kriging locations m Estimation: time O(n 3 ), storage O(n 2 ) 20Gbytes Kriging: time O(mn + n 3 ), storage O(mn + n 2 ) 130Gbytes This is not a huge data set.

26 Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Practical triangulation based GMRFs Data points n Basis functions m Estimation+Kriging: time O(m 3/2 ), storage O(m + n) 50Mbytes

27 Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Global temperature analysis Monthly average temperatures On average 2500 locations per year Covariate data (elevation, land use, etc.) for 7000 stations Model for yearly mean temperatures Observations : temperature elevation + climate + anomaly Climate : µ u, τ u, κ 2 u = k ψ k (u)θ k Anomaly : ( κ 2 u ) (τ u x t (u)) = W(u) Weather : µ(u) + x t (u)

28 Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Code outline for Bayesian inference with R-INLA mesh = inla.mesh(loc=station.locations, ## optional cutoff=10/earth.radius, refine=list(max.edge=500/earth.radius))... spde = inla.spde(mesh, model="matern",... param=list(basis.t=b.tau,...))... formula = temperature ~ elevation + climate.mu + f(anomaly, model=spde, replicate=year)... result = inla(formula, data, family="gaussian",...)

29 Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Estimated climate covariance properties Approximate range Standard deviations km C Latitude Latitude

30 Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Empirical Climate (C) Latitude

31 Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Std dev for Climate (C) Latitude

32 Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Empirical Anomaly 1980 (C) Latitude

33 Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Std dev for Anomaly 1980 (C) Latitude

34 Intro B, W, M, & R SPDE/GMRF Example End Conclusion SPDE/GMRF The connection allows for consistent local specifications of general spatio-temporal models that allow efficient practical computations. Unavoidably beyond the traditional computational statistics toolbox Combines standard concepts from different areas in applied mathematics and engineering Detailed knowledge of SPDEs is not needed for practical use Open source software available:

35 Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 ) α/2 (τ x(u)) = W(u), u R d

36 Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 e iπθ ) α/2 (τ x(u)) = W(u), u R d

37 Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 + m M ) α/2 (τx(u)) = W(u), u R d

38 Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u R d

39 Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω

40 Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω

41 Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω