Introduction to Time Series Analysis. Lecture 1. Peter Bartlett 1. Organizational issues. 2. Objectives of time series analysis. Examples. 3. Overview of the course. 4. Time series models. 5. Time series modelling: Chasing stationarity. 1
Organizational Issues Peter Bartlett. bartlett@stat. Office hours: Tue 11-12, Thu 10-11 (Evans 399). Joe Neeman. jneeman@stat. Office hours: Wed 1:30 2:30, Fri 2-3 (Evans???). http://www.stat.berkeley.edu/ bartlett/courses/153-fall2010/ Check it for announcements, assignments, slides,... Text: Time Series Analysis and its Applications. With R Examples, Shumway and Stoffer. 2nd Edition. 2006. 2
Organizational Issues Classroom and Computer Lab Section: Friday 9 11, in 344 Evans. Starting tomorrow, August 27: Sign up for computer accounts. Introduction to R. Assessment: Lab/Homework Assignments (25%): posted on the website. These involve a mix of pen-and-paper and computer exercises. You may use any programming language you choose (R, Splus, Matlab, python). Midterm Exams (30%): scheduled for October 7 and November 9, at the lecture. Project (10%): Analysis of a data set that you choose. Final Exam (35%): scheduled for Friday, December 17. 3
A Time Series 400 350 300 250 200 150 100 50 0 0 1000 2000 3000 4000 5000 6000 7000 4
A Time Series 400 350 300 250 200 150 100 50 0 1960 1965 1970 1975 1980 1985 1990 year 5
A Time Series 400 350 300 250 $ 200 150 100 50 0 1960 1965 1970 1975 1980 1985 1990 year 6
A Time Series 400 SP500: 1960 1990 350 300 250 $ 200 150 100 50 0 1960 1965 1970 1975 1980 1985 1990 year 7
A Time Series 340 SP500: Jan Jun 1987 320 300 $ 280 260 240 220 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year 8
A Time Series 30 SP500 Jan Jun 1987. Histogram 25 20 15 10 5 0 240 250 260 270 280 290 300 310 $ 9
A Time Series 340 SP500: Jan Jun 1987. Permuted. 320 300 $ 280 260 240 220 0 20 40 60 80 100 120 10
Objectives of Time Series Analysis 1. Compact description of data. 2. Interpretation. 3. Forecasting. 4. Control. 5. Hypothesis testing. 6. Simulation. 11
Classical decomposition: An example Monthly sales for a souvenir shop at a beach resort town in Queensland. (Makridakis, Wheelwright and Hyndman, 1998) 12 x 104 10 8 6 4 2 0 0 10 20 30 40 50 60 70 80 90 12
Transformed data 12 11.5 11 10.5 10 9.5 9 8.5 8 7.5 7 0 10 20 30 40 50 60 70 80 90 13
Trend 12 11.5 11 10.5 10 9.5 9 8.5 8 7.5 7 0 10 20 30 40 50 60 70 80 90 14
Residuals 1.5 1 0.5 0 0.5 1 0 10 20 30 40 50 60 70 80 90 15
Trend and seasonal variation 12 11.5 11 10.5 10 9.5 9 8.5 8 7.5 7 0 10 20 30 40 50 60 70 80 90 16
Objectives of Time Series Analysis 1. Compact description of data. Example: Classical decomposition: X t = T t + S t + Y t. 2. Interpretation. Example: Seasonal adjustment. 3. Forecasting. Example: Predict sales. 4. Control. 5. Hypothesis testing. 6. Simulation. 17
Unemployment data Monthly number of unemployed people in Australia. (Hipel and McLeod, 1994) 8 x 105 7.5 7 6.5 6 5.5 5 4.5 4 1983 1984 1985 1986 1987 1988 1989 1990 18
Trend 8 x 105 7.5 7 6.5 6 5.5 5 4.5 4 1983 1984 1985 1986 1987 1988 1989 1990 19
Trend plus seasonal variation 8 x 105 7.5 7 6.5 6 5.5 5 4.5 4 1983 1984 1985 1986 1987 1988 1989 1990 20
Residuals 8 x 104 6 4 2 0 2 4 6 1983 1984 1985 1986 1987 1988 1989 1990 21
Predictions based on a (simulated) variable 8 x 105 7.5 7 6.5 6 5.5 5 4.5 4 1983 1984 1985 1986 1987 1988 1989 1990 22
Objectives of Time Series Analysis 1. Compact description of data: X t = T t + S t + f(y t ) + W t. 2. Interpretation. Example: Seasonal adjustment. 3. Forecasting. Example: Predict unemployment. 4. Control. Example: Impact of monetary policy on unemployment. 5. Hypothesis testing. Example: Global warming. 6. Simulation. Example: Estimate probability of catastrophic events. 23
Overview of the Course 1. Time series models 2. Time domain methods 3. Spectral analysis 4. State space models(?) 24
Overview of the Course 1. Time series models (a) Stationarity. (b) Autocorrelation function. (c) Transforming to stationarity. 2. Time domain methods 3. Spectral analysis 4. State space models(?) 25
Overview of the Course 1. Time series models 2. Time domain methods (a) AR/MA/ARMA models. (b) ACF and partial autocorrelation function. (c) Forecasting (d) Parameter estimation (e) ARIMA models/seasonal ARIMA models 3. Spectral analysis 4. State space models(?) 26
Overview of the Course 1. Time series models 2. Time domain methods 3. Spectral analysis (a) Spectral density (b) Periodogram (c) Spectral estimation 4. State space models(?) 27
Overview of the Course 1. Time series models 2. Time domain methods 3. Spectral analysis 4. State space models(?) (a) ARMAX models. (b) Forecasting, Kalman filter. (c) Parameter estimation. 28
Time Series Models A time series model specifies the joint distribution of the sequence {X t } of random variables. For example: P[X 1 x 1,...,X t x t ] for all t and x 1,...,x t. Notation: X 1, X 2,... is a stochastic process. x 1, x 2,... is a single realization. We ll mostly restrict our attention to second-order properties only: EX t, E(X t1, X t2 ). 29
Time Series Models Example: White noise: X t WN(0, σ 2 ). i.e., {X t } uncorrelated, EX t = 0, VarX t = σ 2. Example: i.i.d. noise: {X t } independent and identically distributed. P[X 1 x 1,...,X t x t ] = P[X 1 x 1 ] P[X t x t ]. Not interesting for forecasting: P[X t x t X 1,...,X t 1 ] = P[X t x t ]. 30
Gaussian white noise P[X t x t ] = Φ(x t ) = 1 2π xt e x2 /2 dx. 2.5 2 1.5 1 0.5 0 0.5 1 1.5 2 2.5 0 5 10 15 20 25 30 35 40 45 50 31
Gaussian white noise 2.5 2 1.5 1 0.5 0 0.5 1 1.5 2 2.5 0 5 10 15 20 25 30 35 40 45 50 32
Time Series Models Example: Binary i.i.d. P[X t = 1] = P[X t = 1] = 1/2. 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 0 5 10 15 20 25 30 35 40 45 50 33
Random walk S t = t i=1 X i. Differences: S t = S t S t 1 = X t. 8 6 4 2 0 2 4 0 5 10 15 20 25 30 35 40 45 50 34
Random walk ES t? VarS t? 10 5 0 5 10 15 0 5 10 15 20 25 30 35 40 45 50 35
Random Walk Recall S&P500 data. (Notice that it s smooth) 340 SP500: Jan Jun 1987 320 300 $ 280 260 240 220 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year 36
Random Walk Differences: S t = S t S t 1 = X t. 10 SP500, Jan Jun 1987. first differences 8 6 4 2 $ 0 2 4 6 8 10 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year 37
Trend and Seasonal Models X t = T t + S t + E t = β 0 + β 1 t + i (β i cos(λ i t) + γ i sin(λ i t)) + E t 6 5.5 5 4.5 4 3.5 3 2.5 0 50 100 150 200 250 38
Trend and Seasonal Models X t = T t + E t = β 0 + β 1 t + E t 6 5.5 5 4.5 4 3.5 3 2.5 0 50 100 150 200 250 39
Trend and Seasonal Models X t = T t + S t + E t = β 0 + β 1 t + i (β i cos(λ i t) + γ i sin(λ i t)) + E t 6 5.5 5 4.5 4 3.5 3 2.5 0 50 100 150 200 250 40
Trend and Seasonal Models: Residuals 0.5 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4 0.5 0 50 100 150 200 250 41
Time Series Modelling 1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract T t, S t. (b) Differencing. (c) Nonlinear transformations (log, ). 3. Fit model to residuals. 42
Nonlinear transformations Recall: Monthly sales. (Makridakis, Wheelwright and Hyndman, 1998) 12 x 104 12 11.5 10 11 10.5 8 10 6 9.5 9 4 8.5 2 8 7.5 0 0 10 20 30 40 50 60 70 80 90 7 0 10 20 30 40 50 60 70 80 90 43
Time Series Modelling 1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract T t, S t. (b) Differencing. (c) Nonlinear transformations (log, ). 3. Fit model to residuals. 44
Differencing Recall: S&P 500 data. 340 SP500: Jan Jun 1987 10 SP500, Jan Jun 1987. first differences 8 320 6 300 4 2 $ 280 $ 0 260 2 4 240 6 8 220 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year 10 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year 45
Differencing and Trend Define the lag-1 difference operator, (think first derivative ) X t = X t X t 1 = (1 B)X t, where B is the backshift operator, BX t = X t 1. If X t = β 0 + β 1 t + Y t, then If X t = k i=0 β it i + Y t, then X t = β 1 + Y t. k X t = k!β k + k Y t, where k X t = ( k 1 X t ) and 1 X t = X t. 46
Differencing and Seasonal Variation Define the lag-s difference operator, s X t = X t X t s = (1 B s )X t, where B s is the backshift operator applied s times, B s X t = B(B s 1 X t ) and B 1 X t = BX t. If X t = T t + S t + Y t, and S t has period s (that is, S t = S t s for all t), then s X t = T t T t s + s Y t. 47
Time Series Modelling 1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract T t, S t. (b) Differencing. (c) Nonlinear transformations (log, ). 3. Fit model to residuals. 48
Outline 1. Objectives of time series analysis. Examples. 2. Overview of the course. 3. Time series models. 4. Time series modelling: Chasing stationarity. 49