COMP6053 lecture: Time series analysis, autocorrelation jn2@ecs.soton.ac.uk
Time series analysis The basic idea of time series analysis is simple: given an observed sequence, how can we build a model that can predict what comes next? Obvious applications in finance, business, ecology, agriculture, demography, etc.
What's different about time series? In most of the contexts we've seen so far, there's an implicit assumption that observations are independent of each other. In other words, the fact that subject 27 is 165cm tall and terrible at basketball says nothing at all about what will happen with subject 28.
What's different about time series? In time series data, this is not true. We're hoping for exactly the opposite: that what happens at time t contains information about what will happen at time t+1. Observations are treated as both outcome and then predictor variables as we move forward in time.
Ways of dealing with time series Despite (or perhaps because of) the practical uses of time series, there is no single universal technique for handling them. Lots of different ways to proceed depending on the implicit theory of data generation we're proposing. Easiest to illustrate with examples...
Example 1: Lake Huron data Our first example data set is a series of annual measurements of the level of Lake Huron, in feet, from 1875 to 1972. It's a built-in data set in R. So we only need data(lakehuron) to access it. R already "knows" that this is a time series.
Example 1: Lake Huron data
Ex. 2: Australian beer production Our second example is data on monthly Australian beer production, in millions of litres. The time series runs from January 1956 to August 1995. The data is available in beer.csv.
Ex. 2: Australian beer production R doesn't yet know that this is a time series: the data comes in as a list of numbers. We use the ts function to specify that something should be interpreted as a time series, optionally specifying the seasonal period. beer = ts(beer[,1],start=1956,freq=12)
Ex. 2: Australian beer production
Two goals in time series modelling We assume there's some structure in the time series data, obscured by random noise. Structure = trends + seasonal variation. The Lake Huron data has no obvious repetitive structure, but possibly a downward trend. The beer data shows clear seasonality and a trend.
Models of data generation The most basic of data generation is to suppose that there is no structure in the time series at all, and that each observation is an independent random variate. An example: white noise. In this case, the best we can do is simply predict the mean value of the data set.
Lake Huron: prediction if observations were independent
Beer production: prediction if observations were independent
Producing these graphs in R png("beermeanpredict.png",width=800,height=400) plot(beer,xlim=c(1956,2000),lw=2,col="blue") lines(predict(nullbeer,n.ahead=50)$pred, lw=2,col="red") lines(predict(nullbeer,n.ahead=50)$pred +1.96*predict(nullBeer,n.ahead=50)$se, lw=2,lty="dotted",col="red") lines(predict(nullbeer,n.ahead=50)$pred -1.96*predict(nullBeer,n.ahead=50)$se, lw=2,lty="dotted",col="red") graphics.off()
Simple approach to trends We could ignore the seasonal variation and the random noise and simply fit a linear or polynomial model to the data. For example: tb = seq(1956,1995.8,length=length(beer)) tb2 = tb^2 polybeer = lm(beer ~ tb + tb2)
Polynomial fit of lake level on time
Polynomial fit of beer production on time
Regression on time a good idea? This is an OK start: it gives us some sense of what the trend line is. But we probably don't believe that beer production or lake level is a function of the calendar date. More likely these things are a function of their own history, and we need methods that can capture that.
Autoregression A better approach is to ask whether the next value in the time series can be predicted as some function of its previous values. This is called autoregression. We want to build a regression model of the current value fitted on one or more previous values (lagged values). But how many?
Autocorrelation and partial autocorrelation We can look directly at the time series and ask how much information there is in previous values that helps predict the current value. The acf function looks at the correlation between now and various points in the past. Partial autocorrelation(pacf) does the same, but "partials out" the other effects to get the unique contribution of each time-lag.
ACF & PACF, Lake Huron data
ACF & PACF, beer data
ACF & PACF plots ACF shows a correlation that fades as we take longer lagged values in the Lake Huron time series. ACF shows periodic structure in the beer time series reflecting its seasonal nature.
ACF & PACF plots But if t[0] is correlated with t[-1], and t[-1] is correlated with t[-2], then t[0] will necessarily be correlated with t[-2] also. So we need to look at the PACF values. We find that only the most recent value is really useful in building an autoregression model for the Lake Huron data, for example.
Autoregression models With the ar command we can fit autoregression models and ask R to use AIC to decide how many lagged values should be included in the model. For example: arb = ar(beer) The Lake Huron model includes only one lagged value; the beer model includes 24.
Autoregression model, lake data, 1 lagged term
Autoregression model, beer data, 24 lagged terms
Automatically separating trends, seasonal effects, and noise The stl procedure uses locally weighted regression to separate out a trend line, and parcels out the seasonal effect. For example: plot(stl(beer,s.window="periodic"), col="blue",lw=2) If things go well, there should be no autocorrelation structure left in the residuals.
Exponential smoothing A reasonable guess about the next value in a series is that it would be an average of previous values, with the most recent values weighted more strongly. This assumption constitutes exponential smoothing: t 0 = α t -1 + α(1-α)t -2 + α(1-α) 2 t -3...
Holt-Winters procedure The logic can be applied to the basic level of the prediction, to the trend term, and to the seasonal term. The Holt-Winters procedure automatically does this for all three; for example: HWB = HoltWinters(beer)
Holt-Winters analysis on beer data
Holt-Winters analysis on lake data The process seems to work well with the seasonal beer data. For the lake data, we have not specified a seasonal period, and we might also drop the trend term, thus: HWLake = HoltWinters(LakeHuron,gamma=FALSE, beta=false)
Holt-Winters analysis on lake data
Holt-Winters analysis on lake data The fitted alpha value is close to 1 (i.e., a very short memory) so the prediction is that the process will stay where it was. What if we put the trend term back in? HWLake = HoltWinters(LakeHuron,gamma=FALSE)
Holt-Winters analysis on lake data Trend is overdoing it (beta = 0.17)?
Differencing Some time series techniques (e.g., ARIMA) are based on the assumption that the series is stationary, i.e., that it has constant mean, variance, and autocorrelation values over time. If we want to use these techniques we may need to work with the differenced values rather than the raw values.
Differencing This just means transforming t[1] into t[1] - t[0], etc. We can use the diff command to make this easy. To plot the beer data as a differenced series: plot(diff(beer),lw=2,col="green")
Differencing
Some housekeeping in R To get access to some relevant ARIMA model fitting functions, we need to download the "forecast" package. install.packages("forecast") library(forecast)
Auto-regressive integrated movingaverage models (ARIMA) ARIMA is a method for putting together all of the techniques we've seen so far. A non-seasonal ARIMA model is specified with p, d, and q parameters. p: no. of autoregression terms. d: no. of difference levels. q: no. of moving-average (smoothing) terms.
Auto-regressive integrated movingaverage models (ARIMA) ARIMA(0,0,0) is simply predicting the mean of the overall time series, i.e., no structure. ARIMA(0,1,0) works with differences, not raw values, and predicts the next value without any autoregression or smoothing. This is therefore a random walk. ARIMA(1,0,0) and ARIMA(24,0,0) are the models we originally fitted to the lake and beer data.
Auto-regressive integrated movingaverage models (ARIMA) We can also have seasonal ARIMA models: three more terms apply to the seasonal effects. The "forecast" library includes a very convenient auto.arima function that uses AIC to find the most parsimonious model in the space of possible models.
ARIMA(1,1,2) model of lake data
ARIMA(2,1,2)(2,0,0)[12] model of beer data
Fourier transforms No time to discuss Fourier transforms... But they're useful when you suspect there are seasonal or cyclic components in the data, but you don't yet know the period of these components. In the beer example, we already knew the seasonal period was 12, of course.
Additional material The beer.csv data set. The R script used to do the analyses. A general intro to time series analysis in R by Walter Zucchini and Oleg Nenadic. An intro to ARIMA models by Robert Nau. Another useful intro to time series analysis.