Bayesian analysis of non-gaussian Long-Range Dependent processes

Transcription

1 Bayesian analysis of non-gaussian Long-Range Dependent processes Tim Graves 1, Christian Franzke 2, Nick Watkins 2, and Bobby Gramacy 3 Statistical Laboratory, University of Cambridge, Cambridge, UK British Antarctic Survey, Cambridge, UK Booth School of Business, The University of Chicago, Chicago, USA May 18, 2012

2 Outline 1 Introduction 2 Exact Bayesian analysis for Gaussian case 3 Approximate Bayesian analysis for general case

3 Outline 1 Introduction 2 Exact Bayesian analysis for Gaussian case 3 Approximate Bayesian analysis for general case

4 Long Range Dependence Definition A Long Range Dependent process is a stationary process for which k= ρ(k) =.

5 Long Range Dependence Definition A Long Range Dependent process is a stationary process for which k= ρ(k) =.... [T]he stationary long memory processes form a layer among the stationary processes that is near the boundary with non-stationary processes, or, alternatively, as the layer separating the non-stationary processes from the usual stationary processes. [Samorodnitsky, 2006]

6 Long Range Dependence Definition A Long Range Dependent process is a stationary process for which ρ(k) ck 2d 1, 0 < d < 1 2.

7 Long Range Dependence Definition A Long Range Dependent process is a stationary process for which ρ(k) ck 2d 1, 0 < d < 1 2. Definition A Long Range Dependent process is a stationary process for which X t = ψ k ɛ t k, k=0 ψ k ck d 1, 0 < d < 1 2.

8 ARFIMA processes Definition A process {X t } is an ARFIMA(p, d, q) process if it is the solution to: Φ(B)(1 B) d X t = Θ(B)ε t, p where Φ(z) = 1 + φ j z j and Θ(z) = 1 + j=1 q θ j z j, and the innovations {ε t } are iid with 0 mean and variance σ 2 <. We say that {X t } is an ARFIMA(p, d, q) process with mean µ, if {X t µ} is an ARFIMA(p, d, q) process. j=1

9 ARFIMA parameters µ location parameter σ scale parameter d long memory parameter (long memory process iff 0 < d < 0.5 ) φ p-dimensional short memory parameter θ q-dimensional short memory parameter Which parameters are of interest? When considering long memory processes, we are usually primarily interested in the parameter d (and possibly µ). The parameters σ, φ, θ (and even p, q) are essentially nuisance parameters.

10 ARFIMA parameters µ location parameter σ scale parameter d long memory parameter (long memory process iff 0 < d < 0.5 ) φ p-dimensional short memory parameter θ q-dimensional short memory parameter Which parameters are of interest? When considering long memory processes, we are usually primarily interested in the parameter d (and possibly µ). The parameters σ, φ, θ (and even p, q) are essentially nuisance parameters.

11 ARFIMA parameters µ location parameter σ scale parameter d long memory parameter (long memory process iff 0 < d < 0.5 ) φ p-dimensional short memory parameter θ q-dimensional short memory parameter Which parameters are of interest? When considering long memory processes, we are usually primarily interested in the parameter d (and possibly µ). The parameters σ, φ, θ (and even p, q) are essentially nuisance parameters.

12 ARFIMA parameters µ location parameter σ scale parameter d long memory parameter (long memory process iff 0 < d < 0.5 ) φ p-dimensional short memory parameter θ q-dimensional short memory parameter Which parameters are of interest? When considering long memory processes, we are usually primarily interested in the parameter d (and possibly µ). The parameters σ, φ, θ (and even p, q) are essentially nuisance parameters.

13 ARFIMA parameters µ location parameter σ scale parameter d long memory parameter (long memory process iff 0 < d < 0.5 ) φ p-dimensional short memory parameter θ q-dimensional short memory parameter Which parameters are of interest? When considering long memory processes, we are usually primarily interested in the parameter d (and possibly µ). The parameters σ, φ, θ (and even p, q) are essentially nuisance parameters.

14 Outline 1 Introduction 2 Exact Bayesian analysis for Gaussian case 3 Approximate Bayesian analysis for general case

15 Outline of problem Assume Gaussian distribution for the innovations Bayesian: use flat priors for µ, log(σ), and d...

16 Outline of problem Assume Gaussian distribution for the innovations Bayesian: use flat priors for µ, log(σ), and d but can use any set of (independent) priors if desired.

17 Outline of problem Assume Gaussian distribution for the innovations Bayesian: use flat priors for µ, log(σ), and d but can use any set of (independent) priors if desired. Even assuming Gaussianity, the likelihood for d is very complex impossible to find analytic posterior

18 Outline of problem Assume Gaussian distribution for the innovations Bayesian: use flat priors for µ, log(σ), and d but can use any set of (independent) priors if desired. Even assuming Gaussianity, the likelihood for d is very complex impossible to find analytic posterior Must resort to MCMC methods in order to obtain samples from the posterior

19 Outline of problem Assume Gaussian distribution for the innovations Bayesian: use flat priors for µ, log(σ), and d but can use any set of (independent) priors if desired. Even assuming Gaussianity, the likelihood for d is very complex impossible to find analytic posterior Must resort to MCMC methods in order to obtain samples from the posterior Don t want to assume form of short memory (i.e. p, q) must use Reversible-Jump (RJ) MCMC [Green, 1995]

20 Outline of method Re-parameterisation of model to enforce stationarity constraints on Φ and Θ

21 Outline of method Re-parameterisation of model to enforce stationarity constraints on Φ and Θ Efficient calculation of Gaussian likelihood (long memory correlation structure prevents use of standard quick methods)

22 Outline of method Re-parameterisation of model to enforce stationarity constraints on Φ and Θ Efficient calculation of Gaussian likelihood (long memory correlation structure prevents use of standard quick methods) Necessary use of Metropolis Hastings algorithm requires careful selection of proposal distributions

23 Outline of method Re-parameterisation of model to enforce stationarity constraints on Φ and Θ Efficient calculation of Gaussian likelihood (long memory correlation structure prevents use of standard quick methods) Necessary use of Metropolis Hastings algorithm requires careful selection of proposal distributions Correlation between parameters (e.g. φ and d) requires blocking.

24 Example: Pure Gaussian Long Range Dependence (1 B) 0.25 X t = ε t

25 Example: Pure Gaussian Long Range Dependence Density estimate of π(d) d

26 Example: Pure Gaussian Long Range Dependence Density estimate of π(d) d Similarly good results for µ and σ The posterior model probability for the (0, d, 0) model was 70%

27 Example: Corrupted Gaussian Long Range Dependence ( B)(1 B) 0.25 X t = ( B)ε t

28 Example: Corrupted Gaussian Long Range Dependence Density estimate of π(d) d

29 Example: Corrupted Gaussian Long Range Dependence Density estimate of π(d) d The posterior model probability for the (1, d, 1) model was 77% The posterior model probability for the (0, d, 0) model was 0%

30 Dependence of posterior variance on n

31 Dependence of posterior variance on n σ d σ d n n

32 Dependence of posterior variance on n σ d σ d n n σ d n 1/2

33 Outline 1 Introduction 2 Exact Bayesian analysis for Gaussian case 3 Approximate Bayesian analysis for general case

34 Assumptions and general method Drop the Gaussianity assumption Replace with a more general distribution (e.g. α-stable) Seek joint inference about d and α

35 Assumptions and general method Drop the Gaussianity assumption Replace with a more general distribution (e.g. α-stable) Seek joint inference about d and α Initially (for simplicity) we assume no short memory, i.e. we assume a (0, d, 0) model

36 Assumptions and general method Drop the Gaussianity assumption Replace with a more general distribution (e.g. α-stable) Seek joint inference about d and α Initially (for simplicity) we assume no short memory, i.e. we assume a (0, d, 0) model Infinite variance means that auto-covariance approach is no longer sound

37 Assumptions and general method Drop the Gaussianity assumption Replace with a more general distribution (e.g. α-stable) Seek joint inference about d and α Initially (for simplicity) we assume no short memory, i.e. we assume a (0, d, 0) model Infinite variance means that auto-covariance approach is no longer sound Lack of closed form for α-stable density implies lack of closed form for likelihood

38 Solution Approximate the long memory process as a very high order AR process

39 Solution Approximate the long memory process as a very high order AR process Construct the likelihood sequentially and evaluate using specialised efficient methods

40 Solution Approximate the long memory process as a very high order AR process Construct the likelihood sequentially and evaluate using specialised efficient methods f (x 1,..., x t H) = f (x t x t 1,..., x 1, H)f (x t 1,..., x 1 H) where H is the finite recent history of the process x 0, x 1,..., x n

41 Solution Approximate the long memory process as a very high order AR process Construct the likelihood sequentially and evaluate using specialised efficient methods f (x 1,..., x t H) = f (x t x t 1,..., x 1, H)f (x t 1,..., x 1 H) where H is the finite recent history of the process x 0, x 1,..., x n Use auxiliary variables to integrate out the (unknown) history H In practice, setting H = x,..., x suffices, providing enormous computational saving.

42 Example: Pure symmetric α-stable long memory (1 B) 0.15 X t = ε t, α = 1.5

43 Example: Pure symmetric α-stable long memory Density estimate of π(d) d

44 Example: Pure symmetric α-stable long memory Density estimate of π(α) α

45 Example: Pure symmetric α-stable long memory α d

46 Example: Pure symmetric α-stable long memory α d Good estimation of all parameters The posteriors of d and α are independent

47 Dependence of posterior variance on n

48 Dependence of posterior variance on n σ d 5e 04 1e 03 2e 03 5e 03 1e 02 2e 02 5e 02 1e n σ d n 1/2

49 Example: Pure asymmetric α-stable long memory (1 B) 0.1 X t = ε t, α = 1.5 β = 0.5

50 Example: Pure asymmetric α-stable long memory Density estimate of π(β) β

51 Example: Pure asymmetric α-stable long memory Density estimate of π(β) β Good estimation of all other parameters

52 References Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82(4), Samorodnitsky, G. (2006). Long range dependence. Foundations and Trends in Stochastic Systems, 1(3),