11. Time series and dynamic linear models

Transcription

1 11. Time series and dynamic linear models Objective To introduce the Bayesian approach to the modeling and forecasting of time series. Recommended reading West, M. and Harrison, J. (1997). models, (2 nd ed.). Springer. Bayesian forecasting and dynamic Pole, A.,West, M. and Harrison, J. (1994). Applied Bayesian forecasting and time series analysis. Chapman and Hall. Bauwens, L., Lubrano, M. and Richard, J.F. (2000). Bayesian inference in dynamic econometric models. Oxford University Press.

2 Dynamic linear models West The first Bayesian approach to forecasting stems from Harrison and Stevens (1976) and is based on the dynamic linear model. For a full discussion, see West and Harrison (1997).

3 The general DLM Definition 29 The general (univariate) dynamic linear model is Y t = F T t θ t + ν t θ t = G t θ t 1 + ω t where ν t and ω t are zero mean measurement errors and state innovations. These models are linear state space models, where x t = F T t θ t represents the signal, θ t is the state vector, F t is a regression vector and G t is a state matrix. The usual features of a time series such as trend and seasonality can be modeled within this format. In some cases, F and G are supposed independent of t. Then the model is a time series DLM. If V and W are also time independent then the DLM is constant.

4 Examples Example 80 A slowly varying level model is y t = θ t + ν t θ t = θ t 1 + ω t The observations fluctuate around a mean which varies according to a random walk. Example 81 A dynamic linear regression model is given by y t = F T t θ t + ν t θ t = θ t 1 + ω t

5 Bayesian analysis of DLM s If the error terms, ν t and ω t are normally distributed, with known variances, e.g. ν t N (0, V t ), ω t N (0, W t ), then a straightforward Bayesian analysis can be carried out. Example 82 In Example 80, suppose that at time t 1, the current accumulated information is D t 1 = {y 1, y 2,..., y t 1 } and assume that the distribution for θ t 1 is θ t 1 D t 1 N (m t 1, C t 1 ). and that the error distributions are ν t N (0, V t ) and ω t N (0, W t ). Then, we have: 1. The prior distribution for θ t is: θ t D t 1 N (m t 1, R t ) where R t = C t 1 + W t

6 2. The one step ahead predictive distribution for y t is: y t D t 1 N (m t 1, Q t ) where Q t = R t + V t 3. The joint distribution of θ t and y t is θ t y t D t 1 N ( mt 1 m t 1, ( )) Rt R t r t Q t 4. The posterior distribution for θ t given D t = {D t 1, y t } is θ t D t N (m t, C t ) where m t = m t 1 + A t e t A t = R t /Q t e t = y t m t 1 C t = R t A 2 tq t.

7 Proof and observations Proof The first three steps of the proof are straightforward just by going through the observation and system equations. The posterior distribution follows from property iv) of the multivariate normal distribution as given in Definition 22. In the formula for the posterior mean, e t is simply a prediction error term. This formula could also be rewritten as a weighted average in the usual way for normal models: m t = (1 A t )m t 1 + A t y t.

8 The following diagram illustrates the one step ahead predictions for the sales data from Pole et al (1994) assuming a model with constant observation and state error variances and a non-informative prior. An interesting feature to note is that the predictive variance approaches a fixed constant for this model as the number of observed data increases. See West and Harrison (1997) for more details.

9 Example 83 In Example 81, suppose that we have ν t N (0, V t ) and ω t N (0, W t ) with distribution θ t D t 1 N (m t 1, C t 1 ). Then: 1. The prior distribution for θ t is: θ t D t 1 N (m t 1, R t ) where R t = C t 1 + W t. 2. The one step ahead predictive distribution for y t is: y t D t 1 N (f t, Q t ) where f t = F T t m t 1 Q t = F T t R t F t + V t

10 3. The joint distribution of θ t and y t is θ t y t D t 1 N ( ( mt 1 Rt F, T t R t f t R t F t Q t )) 4. The posterior distribution for θ t given D t = {D t 1, y t } is θ t D t N (m t, C t ) where m t = m t 1 + A t e t C t = R t A t A T t Q t A t = R t F t Q 1 t e t = y t f t. Proof Exercise.

11 The following plot shows sales against price. Thus, a dynamic, simple linear regression model would seem appropriate.

12 The following diagram, assuming a constant variance model as earlier illustrates the improved fit of this model.

13 Theorem 43 For the general, univariate DLM, The general theorem for DLM s Y t = F T t θ t + ν t θ t = G t θ t 1 + ω t where ν t N (0, V t ) and ω t N (0, W t ), assuming the prior distribution θ t 1 D t 1 N (m t 1, C t 1 ), we have 1. Prior distribution for θ t : θ t D t 1 N (a t, R t ) where a t = G t m t 1 R t = G t C t 1 G T t + W t

14 2. One step ahead prediction: y t D t 1 N (f t, Q t ) where f t = F T t a t Q t = F T t R t F t + V t. 3. Posterior distribution for θ t D t : θ t D t N (m t, C t ) where m t = a t + A t e t C t = R t R T t Q t A t = R t F t Q 1 t e t = y t f t. Proof Exercise.

15 DLM s and the Kalman filter The updating equations in the general theorem are essentially those used in the Kalman filter developed in Kalman (1960) and Kalman and Bucy (1961). For more details, see

16 Superposition of models Many time series exhibit various different components. For example, as well as the regression component we have already fitted, it may well be that the sales series exhibits a seasonal component. In such cases, we may often wish to combine these components in a single model. In such cases, we may write y t = y 1t y kt + ν t where y jt = F T jtθ jt and This leads to a combined model θ jt = G jt θ j,t 1 + ω jt for j = 1,..., k. F t = y t = F T t θ + ν t θ t = G t θ t 1 + ω t where F 1t., G t = G 1t F kt 0 0 G kt.

17

18 Discount factors Thus far, we have not considered how to model the uncertainty in the unknown variances. It is possible to model the uncertainty in the observation variances analytically in the usual way (via inverse gamma priors). However, the treatment of the system variances is more complex. In this case, discount factors can be used. The idea is based on information discounting. As information ages, it becomes less useful and so its value should diminish. Thus, in our problem, with system equation θ t = G t θ t 1 + ω t, ω t N (0, W t ) then given that V [θ t 1 D t 1 ] = C t 1, we have where R t = V [θ t D t 1 ] = P t + W t P t = V [G t θ t 1 D t 1 ] = G t C t 1 G T t and W t = R t P t.

19 If we define δ such that R t = P t /δ, then we can interpret δ as the percentage of information that passes from time t 1 to time t and in this case, W t = P t ( δ 1 1 ). Typical values for systems without abrupt changes are usually around δ = 0.9. Small values of δ (below 0.8) imply large levels of uncertainty and lead to predictions with very wide bounds. High values represent more smoothly changing systems, and in the limit, when δ = 1, we have a static system with no information loss. The following diagrams show the effects of fitting a trend model with discount factors 0.8, 0.9 and 1 to the sales data. We can see that the higher the discount factor, the higher the degree of smoothing.

20

21 The forward filtering backward sampling algorithm This algorithm, developed in Carter and Kohn (1994) and Frühwirth-Schnatter (1994) allows for the implementation of an MCMC approach to DLM s. The forward filtering step is the standard normal linear analysis to give p(θ t D t ) at each t, for t = 1,..., n. The backward sampling step uses the Markov property and samples θ n from p(θ n D n ) and then, for t = 1,..., n 1, samples θ t from p(θ t D t, θ t+1). Thus, a sample from the posterior parameter structure is generated.

22 Example: the AR(p) model with time varying coefficients Example 84 The AR(p) model with time varying coefficients takes the form y t = θ 0t + θ 1t θ pt y t p + ν t θ it = θ i,t 1 + ω it where we shall assume that the error terms are independent normals: ν it N (0, V ) and ω it N (0, λ i V ). Then this model can be expressed in state space form by setting θ t = (θ 0t,..., θ pt ) T F = (1, y t 1,..., y t p ) T G = I p+1 W = V diag(λ)

23 Here diag(λ) represents a diagonal matrix with ii th entry equal to λ i, for i = 1,..., p + 1. Now, given gamma priors for V and for λ 1 i and a normal prior for θ 0, then it is clear that the relevant posterior distributions are all conditionally conjugate. Koop (2003) examines data on the annual percentage change in UK industrial production from 1701 to 1992.

24 Koop (2003) assumes a time varying coefficient AR(1) model and uses proper but relatively uninformative prior distributions. The posterior distributions of λ 0 and λ 1 estimated from running the Gibbs sampler are as follows. The posterior means and deviations of both λ 0 and λ 1 suggest that there is quite high stochastic variation in both θ 0t and θ 1t.

25 Software for fitting DLM s Two general software packages are available. BATS. Pole et al (1994). This (somewhat out of date) package can be used to perform basic analyses and is available from: dlm. Petris (2006). This is a recently produced R package for fitting DLM s, including ARMA models etc. available from

26 Other work in time series ARMA and ARIMA models. Marriot and Newbold (1998). Non linear and non normal state space models. Carlin et al (1992). Latent structure models. Aguilar et al (1998). Stochastic volatility models. Jacquier et al (1994). GARCH and other econometric models. Bauwens et al (2000). Wavelets. Nason and Sapatinas (2002).

27 References Aguilar, O., Huerta, G., Prado, R. and West, M. (1998). Bayesian inference on latent structure in time series. In 6 eds. Bernardo et al. Oxford: University Press. Bauwens, L., Lubrano, M. and Richard, J.F. (2000). econometric models. Oxford University Press. Bayesian inference in dynamic Carlin, B.P., Poison, N.G. and Stoffer, D.S. (1992). A Monte Carlo approach to nonnormal and nonlinear state space modeling. Journal of the American Statistical Association, 87, Carter, C.K. and Kohn, R. (1996). Markov chain Monte Carlo in conditionally Gaussian state space models. Biometrika, 83, Frühwirth-Schnatter, S. (1994). Data augmentation and dynamic linear models. Journal of Time Series Analysis, 15, Harrison, P.J. and Stevens, C.F. (1976). Bayesian forecasting (with discussion). Journal of the Royal Statistical Society Series B, 38, Jacquier, E., Polson, N.G. and Rossi, P.E. (1994). Bayesian Analysis of Stochastic Volatility Models. Journal of Business & Economic Statistics, 12, Kalman, R.E. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering Series D, 82,

28 Kalman, R.E. and Bucy, R.S. (1961). New Results in Linear Filtering and Prediction Theory. Journal of Basic Engineering Series D, 83, Koop, G. (2003). Bayesian Econometrics. New York: Wiley. Marriott, J. and Newbold, P. (1998). Bayesian Comparison of ARIMA and Stationary ARMA Models. International Statistical Review, 66, Nason, G.P. and Sapatinas, T. (2002). Wavelet packet transfer function modelling of nonstationary time series. Statistics and Computing, 12, Pole, A.,West, M. and Harrison, J. (1994). Applied Bayesian forecasting and time series analysis. Chapman and Hall. West, M. and Harrison, J. (1997). Bayesian forecasting and dynamic models, (2 nd ed.). Springer.