Intro B, W, M, & R SPDE/GMRF Example End An explicit link between Gaussian fields and Gaussian Markov random fields; the stochastic partial differential equation approach Finn Lindgren 1 Håvard Rue 1 Johan Lindström 2 1 Department of Mathematical Sciences Norwegian University of Science and Technology, Trondheim, Norway 2 Mathematical Statistics Centre for Mathematical Sciences, Lund University, Lund, Sweden RSS, London, 16th March 2011
Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Computationally efficent spatial models Spatial random field models Dense covariance: x N (µ, Σ) = expensive computations Sparse precision: x N (µ, Q 1 ) = efficient computations Leukaemia survival in NW England
Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations lattice CAR models We need models with well-defined continuous interpretation
Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations triangular graph We need models with well-defined continuous interpretation
Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations refined triangulation We need models with well-defined continuous interpretation
Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations smooth triangulation We need models with well-defined continuous interpretation
Intro B, W, M, & R SPDE/GMRF Example End Efficiency Gaussian Markov random fields Irregular discrete data locations continuous space We need models with well-defined continuous interpretation
Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle Conditional autoregression models on lattices GMRF Covariance CAR(2) κ u K 1 (κ u ) Whittle (1954) CAR(1) 1 2π K 0(κ u ) Besag (1981) ICAR(1) 1 2π log( u ) Besag & Mondal (2005)
Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle Matérn covariances (Bertil Matérn) The Matérn covariance family Cov(x(0), x(u)) = σ 2 21 ν Γ(ν) (κ u )ν K ν (κ u ), u R d Scale κ > 0, smoothness ν > 0, variance σ 2 > 0 Are any other discrete domain Markov fields close to Matérn fields? Are any Matérn fields continuous domain Markov fields?
Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle The continuous domain Markov property S is a separating set for A and B: x(a) x(b) x(s) A S B
Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle Identifying a Markov field by its spectrum Spectral representation: x(u) = exp(iu k) x(k) dk R d Spectrum: S x (k) = E( x(k) 2 ) Rozanov (1977) x(u) is Markov S x (k) P(k) 1 where P(k) is a non-negative symmetric polynomial What is the spectrum of a Matérn field? c MFO
Intro B, W, M, & R SPDE/GMRF Example End CAR Matérn Markov Whittle The Whittle-Matérn connection Whittle (1954, 1963) Matérn fields on R d are stationary solutions to the SPDE ( κ 2 ) α/2 x(u) = W(u), α = ν + d/2 The spectrum is S x (k) = (2π) d (κ 2 + k 2 ) α Therefore Matérn fields are Markov when ν + d/2 is an integer. Laplacian: = d 2 u 2 i=1 i
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Piecewise linear representations x(u) = cos(u 1 ) + sin(u 2 ) x(u) = k ψ k(u) x k Guiding principle Attack the SPDE with local finite dimensional representations instead of covariances or kernels (subsets of Green s functions)!
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond The best piecewise linear approximation k ψ k(u) x k Projection of the SPDE: Linear systems of equations (α = 2) j (κ 2 ψ i, ψ j + ψ i, ψ j } {{ } } {{ } C ij G ij )x j D = ψ i, W jointly for all i. C and G are as sparse as the triangulation neighbourhood Constructing the precision matrices K = κ 2 C + G α = 1 α = 2 α = 3, 4,... Kx N (0, K) N (0, C) ( ) N 0, CQ 1 x,α 2 C Q x,α K K T C 1 K K T C 1 Q x,α 2 C 1 K
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Markov approximation C 1 is dense, which leads to non-markov models! The diagonal matrix C, C i,i = ψ i, 1 = the local cell area, gives a sparse precision with the total energy preserved.
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Lattice on R 2, node distance h, regular triangulation Order α = 1: generalised covariance 1 2π K 0(κ u ) 1 κ 2 h 2 1 + 1 4 1 1 Order α = 2: covariance 1 4πκ 2 κ u K 1 (κ u ) κ 4 h 2 1 1 + 2κ 2 1 4 1 + 1 h 1 2 1 2 8 2 1 8 20 8 1 2 8 2 1
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 ) α/2 (τ x(u)) = W(u), u R d
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 e iπθ ) α/2 (τ x(u)) = W(u), u R d
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 + m M ) α/2 (τx(u)) = W(u), u R d
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u R d
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω
Intro B, W, M, & R SPDE/GMRF Example End Finite Projection Markov Lattices... and beyond Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. ( t + κ2 u,t + m u,t M u,t ) (τ u,t x(u, t)) = E(u, t), (u, t) Ω R
Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Large spatial problems: covariances are impractical Data points n 2500 20 Kriging locations m 15000 20 Estimation: time O(n 3 ), storage O(n 2 ) 20Gbytes Kriging: time O(mn + n 3 ), storage O(mn + n 2 ) 130Gbytes This is not a huge data set.
Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Practical triangulation based GMRFs Data points n 2500 20 Basis functions m 15000 20 Estimation+Kriging: time O(m 3/2 ), storage O(m + n) 50Mbytes
Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Global temperature analysis http://www.ncdc.noaa.gov/ghcn/ghcn.html Monthly average temperatures On average 2500 locations per year Covariate data (elevation, land use, etc.) for 7000 stations Model for yearly mean temperatures Observations : temperature elevation + climate + anomaly Climate : µ u, τ u, κ 2 u = k ψ k (u)θ k Anomaly : ( κ 2 u ) (τ u x t (u)) = W(u) Weather : µ(u) + x t (u)
Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Code outline for Bayesian inference with R-INLA mesh = inla.mesh(loc=station.locations, ## optional cutoff=10/earth.radius, refine=list(max.edge=500/earth.radius))... spde = inla.spde(mesh, model="matern",... param=list(basis.t=b.tau,...))... formula = temperature ~ elevation + climate.mu + f(anomaly, model=spde, replicate=year)... result = inla(formula, data, family="gaussian",...)
Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Estimated climate covariance properties 1970-1989 Approximate range Standard deviations km 0 1000 2000 3000 C 0.0 1.0 2.0 3.0 90 60 30 0 30 60 90 Latitude 90 60 30 0 30 60 90 Latitude
Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Empirical Climate 1970 1989 (C) 30 20 10 0 10 20 30 Latitude 90 60 45 30 15 0 15 30 45 60 90 180 135 90 45 0 45 90 135 180
Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Std dev for Climate 1970 1989 (C) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Latitude 90 60 45 30 15 0 15 30 45 60 90 180 135 90 45 0 45 90 135 180
Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Empirical Anomaly 1980 (C) 2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0 Latitude 90 60 45 30 15 0 15 30 45 60 90 180 135 90 45 0 45 90 135 180
Intro B, W, M, & R SPDE/GMRF Example End Large problems GHCN INLA Climate Weather Std dev for Anomaly 1980 (C) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 Latitude 90 60 45 30 15 0 15 30 45 60 90 180 135 90 45 0 45 90 135 180
Intro B, W, M, & R SPDE/GMRF Example End Conclusion SPDE/GMRF The connection allows for consistent local specifications of general spatio-temporal models that allow efficient practical computations. Unavoidably beyond the traditional computational statistics toolbox Combines standard concepts from different areas in applied mathematics and engineering Detailed knowledge of SPDEs is not needed for practical use Open source software available: http://www.r-inla.org/
Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 ) α/2 (τ x(u)) = W(u), u R d
Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 e iπθ ) α/2 (τ x(u)) = W(u), u R d
Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 + m M ) α/2 (τx(u)) = W(u), u R d
Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u R d
Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω
Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω
Intro B, W, M, & R SPDE/GMRF Example End Covariances for Beyond classical Matérn models The approach can in a straightforward way be extended to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u ) α/2 (τ u x(u)) = W(u), u Ω