Rob J Hyndman. Forecasting using. 11. Dynamic regression OTexts.com/fpp/9/1/ Forecasting using R 1

Transcription

1 Rob J Hyndman Forecasting using 11. Dynamic regression OTexts.com/fpp/9/1/ Forecasting using R 1

2 Outline 1 Regression with ARIMA errors 2 Example: Japanese cars 3 Using Fourier terms for seasonality 4 Example: Sales of petroleum & coal products Forecasting using R Regression with ARIMA errors 2

3 Regression with ARIMA errors Regression models y t = b 0 + b 1 x 1,t + + b k x k,t + n t y t modeled as function of k explanatory variables x 1,t,..., x k,t. Usually, we assume that n t is WN. Now we want to allow n t to be autocorrelated. Example: n t = ARIMA(1,1,1) y t = b 0 + b 1 x 1,t + + b k x k,t + n t where (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t and e t is white noise. Forecasting using R Regression with ARIMA errors 3

4 Regression with ARIMA errors Regression models y t = b 0 + b 1 x 1,t + + b k x k,t + n t y t modeled as function of k explanatory variables x 1,t,..., x k,t. Usually, we assume that n t is WN. Now we want to allow n t to be autocorrelated. Example: n t = ARIMA(1,1,1) y t = b 0 + b 1 x 1,t + + b k x k,t + n t where (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t and e t is white noise. Forecasting using R Regression with ARIMA errors 3

5 Regression with ARIMA errors Regression models y t = b 0 + b 1 x 1,t + + b k x k,t + n t y t modeled as function of k explanatory variables x 1,t,..., x k,t. Usually, we assume that n t is WN. Now we want to allow n t to be autocorrelated. Example: n t = ARIMA(1,1,1) y t = b 0 + b 1 x 1,t + + b k x k,t + n t where (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t and e t is white noise. Forecasting using R Regression with ARIMA errors 3

6 Regression with ARIMA errors Regression models y t = b 0 + b 1 x 1,t + + b k x k,t + n t y t modeled as function of k explanatory variables x 1,t,..., x k,t. Usually, we assume that n t is WN. Now we want to allow n t to be autocorrelated. Example: n t = ARIMA(1,1,1) y t = b 0 + b 1 x 1,t + + b k x k,t + n t where (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t and e t is white noise. Forecasting using R Regression with ARIMA errors 3

7 Regression with ARIMA errors Regression models y t = b 0 + b 1 x 1,t + + b k x k,t + n t y t modeled as function of k explanatory variables x 1,t,..., x k,t. Usually, we assume that n t is WN. Now we want to allow n t to be autocorrelated. Example: n t = ARIMA(1,1,1) y t = b 0 + b 1 x 1,t + + b k x k,t + n t where (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t and e t is white noise. Forecasting using R Regression with ARIMA errors 3

8 Residuals and errors Example: n t = ARIMA(1,1,1) where y t = b 0 + b 1 x 1,t + + b k x k,t + n t (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t Be careful in distinguishing n t from e t. n t are the errors and e t are the residuals. In ordinary regression, n t is assumed to be white noise and so n t = e t. After differencing all variables y t = b 1 x 1,t + + b k x k,t + n t. Now a regression with ARMA(1,1) error Forecasting using R Regression with ARIMA errors 4

9 Residuals and errors Example: n t = ARIMA(1,1,1) where y t = b 0 + b 1 x 1,t + + b k x k,t + n t (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t Be careful in distinguishing n t from e t. n t are the errors and e t are the residuals. In ordinary regression, n t is assumed to be white noise and so n t = e t. After differencing all variables y t = b 1 x 1,t + + b k x k,t + n t. Now a regression with ARMA(1,1) error Forecasting using R Regression with ARIMA errors 4

10 Residuals and errors Example: n t = ARIMA(1,1,1) where y t = b 0 + b 1 x 1,t + + b k x k,t + n t (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t Be careful in distinguishing n t from e t. n t are the errors and e t are the residuals. In ordinary regression, n t is assumed to be white noise and so n t = e t. After differencing all variables y t = b 1 x 1,t + + b k x k,t + n t. Now a regression with ARMA(1,1) error Forecasting using R Regression with ARIMA errors 4

11 Residuals and errors Example: n t = ARIMA(1,1,1) where y t = b 0 + b 1 x 1,t + + b k x k,t + n t (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t Be careful in distinguishing n t from e t. n t are the errors and e t are the residuals. In ordinary regression, n t is assumed to be white noise and so n t = e t. After differencing all variables y t = b 1 x 1,t + + b k x k,t + n t. Now a regression with ARMA(1,1) error Forecasting using R Regression with ARIMA errors 4

12 Residuals and errors Example: n t = ARIMA(1,1,1) where y t = b 0 + b 1 x 1,t + + b k x k,t + n t (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t Be careful in distinguishing n t from e t. n t are the errors and e t are the residuals. In ordinary regression, n t is assumed to be white noise and so n t = e t. After differencing all variables y t = b 1 x 1,t + + b k x k,t + n t. Now a regression with ARMA(1,1) error Forecasting using R Regression with ARIMA errors 4

13 Residuals and errors Example: n t = ARIMA(1,1,1) where y t = b 0 + b 1 x 1,t + + b k x k,t + n t (1 φ 1 B)(1 B)n t = (1 θ 1 B)e t Be careful in distinguishing n t from e t. n t are the errors and e t are the residuals. In ordinary regression, n t is assumed to be white noise and so n t = e t. After differencing all variables y t = b 1 x 1,t + + b k x k,t + n t. Now a regression with ARMA(1,1) error Forecasting using R Regression with ARIMA errors 4

14 Regression with ARIMA errors Any regression with an ARIMA error can be rewritten as a regression with an ARMA error by differencing all variables with the same differencing operator as in the ARIMA model. Original data y t = b 0 + b 1 x 1,t + + b k x k,t + n t where φ(b)(1 B) d n t = θ(b)e t After differencing all variables y t = b 1 x 1,t + + b k x k,t + n t. where φ(b)n t = θ(b)e t and y t = (1 B) d y t, etc. Forecasting using R Regression with ARIMA errors 5

15 Regression with ARIMA errors Any regression with an ARIMA error can be rewritten as a regression with an ARMA error by differencing all variables with the same differencing operator as in the ARIMA model. Original data y t = b 0 + b 1 x 1,t + + b k x k,t + n t where φ(b)(1 B) d n t = θ(b)e t After differencing all variables y t = b 1 x 1,t + + b k x k,t + n t. where φ(b)n t = θ(b)e t and y t = (1 B) d y t, etc. Forecasting using R Regression with ARIMA errors 5

16 Regression with ARIMA errors Any regression with an ARIMA error can be rewritten as a regression with an ARMA error by differencing all variables with the same differencing operator as in the ARIMA model. Original data y t = b 0 + b 1 x 1,t + + b k x k,t + n t where φ(b)(1 B) d n t = θ(b)e t After differencing all variables y t = b 1 x 1,t + + b k x k,t + n t. where φ(b)n t = θ(b)e t and y t = (1 B) d y t, etc. Forecasting using R Regression with ARIMA errors 5

17 Modeling procedure Problems with OLS and autocorrelated errors 1 OLS no longer the best way to compute coefficients as it does not take account of time-relationships in data. 2 Standard errors of coefficients are incorrect most likely too small. This invalidates tests and prediction intervals. Second problem more serious because it can lead to misleading results. If standard errors obtained using OLS too small, some explanatory variables may appear to be significant when, in fact, they are not. This is known as spurious regression. Forecasting using R Regression with ARIMA errors 6

18 Modeling procedure Problems with OLS and autocorrelated errors 1 OLS no longer the best way to compute coefficients as it does not take account of time-relationships in data. 2 Standard errors of coefficients are incorrect most likely too small. This invalidates tests and prediction intervals. Second problem more serious because it can lead to misleading results. If standard errors obtained using OLS too small, some explanatory variables may appear to be significant when, in fact, they are not. This is known as spurious regression. Forecasting using R Regression with ARIMA errors 6

19 Modeling procedure Problems with OLS and autocorrelated errors 1 OLS no longer the best way to compute coefficients as it does not take account of time-relationships in data. 2 Standard errors of coefficients are incorrect most likely too small. This invalidates tests and prediction intervals. Second problem more serious because it can lead to misleading results. If standard errors obtained using OLS too small, some explanatory variables may appear to be significant when, in fact, they are not. This is known as spurious regression. Forecasting using R Regression with ARIMA errors 6

20 Modeling procedure Problems with OLS and autocorrelated errors 1 OLS no longer the best way to compute coefficients as it does not take account of time-relationships in data. 2 Standard errors of coefficients are incorrect most likely too small. This invalidates tests and prediction intervals. Second problem more serious because it can lead to misleading results. If standard errors obtained using OLS too small, some explanatory variables may appear to be significant when, in fact, they are not. This is known as spurious regression. Forecasting using R Regression with ARIMA errors 6

21 Modeling procedure Problems with OLS and autocorrelated errors 1 OLS no longer the best way to compute coefficients as it does not take account of time-relationships in data. 2 Standard errors of coefficients are incorrect most likely too small. This invalidates tests and prediction intervals. Second problem more serious because it can lead to misleading results. If standard errors obtained using OLS too small, some explanatory variables may appear to be significant when, in fact, they are not. This is known as spurious regression. Forecasting using R Regression with ARIMA errors 6

22 Modeling procedure Estimation only works when all predictor variables are deterministic or stationary and the errors are stationary. So difference stochastic variables as required until all variables appear stationary. Then fit model with ARMA errors. auto.arima() will handle order selection and differencing (but only checks that errors are stationary). Forecasting using R Regression with ARIMA errors 7

23 Modeling procedure Estimation only works when all predictor variables are deterministic or stationary and the errors are stationary. So difference stochastic variables as required until all variables appear stationary. Then fit model with ARMA errors. auto.arima() will handle order selection and differencing (but only checks that errors are stationary). Forecasting using R Regression with ARIMA errors 7

24 Modeling procedure Estimation only works when all predictor variables are deterministic or stationary and the errors are stationary. So difference stochastic variables as required until all variables appear stationary. Then fit model with ARMA errors. auto.arima() will handle order selection and differencing (but only checks that errors are stationary). Forecasting using R Regression with ARIMA errors 7

25 Outline 1 Regression with ARIMA errors 2 Example: Japanese cars 3 Using Fourier terms for seasonality 4 Example: Sales of petroleum & coal products Forecasting using R Example: Japanese cars 8

26 Example: Japanese cars Japanese motor vehicle production millions Year Forecasting using R Example: Japanese cars 9

27 Example: Japanese cars Japanese motor vehicle production millions Proposed model: linear trend plus correlated errors. y t = a + bt + n t where n t is ARMA process Year Forecasting using R Example: Japanese cars 9

28 Example: Japanese cars Errors from linear trend model ACF PACF Lag Lag Forecasting using R Example: Japanese cars 10

29 Example: Japanese cars 1 We will fit a linear trend model: y t = a + bx t + n t where x t = t auto.arima chooses an AR(1) model for n t. 3 Full model is y t = a + bx t + n t where n t = φ 1 n t 1 + e t and e t is a white noise series. Forecasting using R Example: Japanese cars 11

30 Example: Japanese cars 1 We will fit a linear trend model: y t = a + bx t + n t where x t = t auto.arima chooses an AR(1) model for n t. 3 Full model is y t = a + bx t + n t where n t = φ 1 n t 1 + e t and e t is a white noise series. Forecasting using R Example: Japanese cars 11

31 Example: Japanese cars 1 We will fit a linear trend model: y t = a + bx t + n t where x t = t auto.arima chooses an AR(1) model for n t. 3 Full model is y t = a + bx t + n t where n t = φ 1 n t 1 + e t and e t is a white noise series. Forecasting using R Example: Japanese cars 11

32 Example: Japanese cars Parameter Estimate Standard Error AR(1) φ Intercept a Slope b We need to check that the residuals (e t ) look like white noise. Forecasting using R Example: Japanese cars 12

33 Example: Japanese cars Parameter Estimate Standard Error AR(1) φ Intercept a Slope b We need to check that the residuals (e t ) look like white noise. Forecasting using R Example: Japanese cars 12

34 Example: Japanese cars Residuals from linear trend + AR(1) model ACF PACF Lag Lag Forecasting using R Example: Japanese cars 13

35 Example: Japanese cars Forecasts from linear trend + AR(1) errors millions Forecasting using R Example: Japanese cars 14

36 Example: Japanese cars > fit2 <- auto.arima(x) ARIMA(0,1,0) with drift Coefficients: drift s.e sigma^2 estimated as : log likelihood = AIC = AICc = 31.9 BIC = 33.8 Forecasting using R Example: Japanese cars 15

37 Example: Japanese cars Forecasts from ARIMA(0,1,0) with drift millions Forecasting using R Example: Japanese cars 16

38 Example: Japanese cars Forecasts from linear trend + AR(1) errors millions Forecasting using R Example: Japanese cars 16

39 Outline 1 Regression with ARIMA errors 2 Example: Japanese cars 3 Using Fourier terms for seasonality 4 Example: Sales of petroleum & coal products Forecasting using R Using Fourier terms for seasonality 17

40 Fourier series Any periodic function of period m can be approximated using K { ( ) ( )} 2πkt 2πkt sin + cos m m k=1 The approximation can be made exact with large K. Each k represents a harmonic. fourier generates a matrix of Fourier terms. Example fit <- auto.arima(y, seasonal=false, xreg=fourier(y, K=3)) Forecasting using R Using Fourier terms for seasonality 18

41 Fourier series Any periodic function of period m can be approximated using K { ( ) ( )} 2πkt 2πkt sin + cos m m k=1 The approximation can be made exact with large K. Each k represents a harmonic. fourier generates a matrix of Fourier terms. Example fit <- auto.arima(y, seasonal=false, xreg=fourier(y, K=3)) Forecasting using R Using Fourier terms for seasonality 18

42 Fourier series Any periodic function of period m can be approximated using K { ( ) ( )} 2πkt 2πkt sin + cos m m k=1 The approximation can be made exact with large K. Each k represents a harmonic. fourier generates a matrix of Fourier terms. Example fit <- auto.arima(y, seasonal=false, xreg=fourier(y, K=3)) Forecasting using R Using Fourier terms for seasonality 18

43 Fourier series Any periodic function of period m can be approximated using K { ( ) ( )} 2πkt 2πkt sin + cos m m k=1 The approximation can be made exact with large K. Each k represents a harmonic. fourier generates a matrix of Fourier terms. Example fit <- auto.arima(y, seasonal=false, xreg=fourier(y, K=3)) Forecasting using R Using Fourier terms for seasonality 18

44 Fourier series Any periodic function of period m can be approximated using K { ( ) ( )} 2πkt 2πkt sin + cos m m k=1 The approximation can be made exact with large K. Each k represents a harmonic. fourier generates a matrix of Fourier terms. Example fit <- auto.arima(y, seasonal=false, xreg=fourier(y, K=3)) Forecasting using R Using Fourier terms for seasonality 18

45 Outline 1 Regression with ARIMA errors 2 Example: Japanese cars 3 Using Fourier terms for seasonality 4 Example: Sales of petroleum & coal products Forecasting using R Example: Sales of petroleum & coal products 19

46 Example: Sales of petroleum & coal products Sales: Petroleum and coal products Sales: Chemicals and allied products Bituminous Coal production Sales: Motor Vehicles and parts U.S. monthly sales: Jan 1971 Dec Forecasting using R Example: Sales of petroleum & coal products 20

47 Example: Sales of petroleum & coal products Log Sales: Petroleum & Coal products Log Sales: Chemicals & allied products Log Bituminous Coal production Log Sales: Motor Vehicles and parts Forecasting using R Example: Sales of petroleum & coal products 21

48 Example: Sales of petroleum & coal products Log Sales: Petroleum & Coal products Log Sales: Chemicals & allied products Log Bituminous Coal production Log Sales: Motor Vehicles and parts Series clearly non-stationary, so difference. Forecasting using R Example: Sales of petroleum & coal products 21

49 Example: Sales of petroleum & coal products y t = b 1 x 1,t + b 2 x 2,t + b 3 x 3,t + n t y t = log petroleum sales x 1,t = log chemical sales x 2,t = log coal production x 3,t = log motor vehicle and parts sales. auto.arima selects an ARIMA(0,0,1)(1,0,2) 12 model for n t. Full model is y t = b 1 x 1,t + b 2 x 2,t + b 3 x 3,t + n t (1 Φ 1 B 12 )(1 B)n t = (1+θ 1 B)(1+Θ 1 B 12 +Θ 2 B 24 )e t. Forecasting using R Example: Sales of petroleum & coal products 22

50 Example: Sales of petroleum & coal products y t = b 1 x 1,t + b 2 x 2,t + b 3 x 3,t + n t y t = log petroleum sales x 1,t = log chemical sales x 2,t = log coal production x 3,t = log motor vehicle and parts sales. auto.arima selects an ARIMA(0,0,1)(1,0,2) 12 model for n t. Full model is y t = b 1 x 1,t + b 2 x 2,t + b 3 x 3,t + n t (1 Φ 1 B 12 )(1 B)n t = (1+θ 1 B)(1+Θ 1 B 12 +Θ 2 B 24 )e t. Forecasting using R Example: Sales of petroleum & coal products 22

51 Example: Sales of petroleum & coal products y t = b 1 x 1,t + b 2 x 2,t + b 3 x 3,t + n t y t = log petroleum sales x 1,t = log chemical sales x 2,t = log coal production x 3,t = log motor vehicle and parts sales. auto.arima selects an ARIMA(0,0,1)(1,0,2) 12 model for n t. Full model is y t = b 1 x 1,t + b 2 x 2,t + b 3 x 3,t + n t (1 Φ 1 B 12 )(1 B)n t = (1+θ 1 B)(1+Θ 1 B 12 +Θ 2 B 24 )e t. Forecasting using R Example: Sales of petroleum & coal products 22

52 Example: Sales of petroleum & coal products Parameter Estimate S.E MA(1) θ Seasonal AR(1) Φ Seasonal MA(1) Θ Seasonal MA(2) Θ Log Chemicals b Log Coal b Log Vehicles b AIC = Consider dropping motor vehicles and parts variable: AIC= Forecasting using R Example: Sales of petroleum & coal products 23

53 Example: Sales of petroleum & coal products Parameter Estimate S.E MA(1) θ Seasonal AR(1) Φ Seasonal MA(1) Θ Seasonal MA(2) Θ Log Chemicals b Log Coal b Log Vehicles b AIC = Consider dropping motor vehicles and parts variable: AIC= Forecasting using R Example: Sales of petroleum & coal products 23

54 Forecasting To forecast a regression model with ARIMA errors, we need to forecast the regression part of the model and the ARIMA part of the model and combine the results. When future predictors are unknown, they need to be forecast first. Forecasts of macroeconomic variables may be obtained from the national statistical offices. Separate forecasting models may be needed for other explanatory variables. Some explanatory variable are known into the future (e.g., time, dummies). Forecasting using R Example: Sales of petroleum & coal products 24

55 Forecasting To forecast a regression model with ARIMA errors, we need to forecast the regression part of the model and the ARIMA part of the model and combine the results. When future predictors are unknown, they need to be forecast first. Forecasts of macroeconomic variables may be obtained from the national statistical offices. Separate forecasting models may be needed for other explanatory variables. Some explanatory variable are known into the future (e.g., time, dummies). Forecasting using R Example: Sales of petroleum & coal products 24

56 Forecasting To forecast a regression model with ARIMA errors, we need to forecast the regression part of the model and the ARIMA part of the model and combine the results. When future predictors are unknown, they need to be forecast first. Forecasts of macroeconomic variables may be obtained from the national statistical offices. Separate forecasting models may be needed for other explanatory variables. Some explanatory variable are known into the future (e.g., time, dummies). Forecasting using R Example: Sales of petroleum & coal products 24

57 Forecasting To forecast a regression model with ARIMA errors, we need to forecast the regression part of the model and the ARIMA part of the model and combine the results. When future predictors are unknown, they need to be forecast first. Forecasts of macroeconomic variables may be obtained from the national statistical offices. Separate forecasting models may be needed for other explanatory variables. Some explanatory variable are known into the future (e.g., time, dummies). Forecasting using R Example: Sales of petroleum & coal products 24

58 Forecasting To forecast a regression model with ARIMA errors, we need to forecast the regression part of the model and the ARIMA part of the model and combine the results. When future predictors are unknown, they need to be forecast first. Forecasts of macroeconomic variables may be obtained from the national statistical offices. Separate forecasting models may be needed for other explanatory variables. Some explanatory variable are known into the future (e.g., time, dummies). Forecasting using R Example: Sales of petroleum & coal products 24