Applied Time Series Analysis

Transcription

1 Applied Time Series Analysis SS 014 Dr. Marcel Deling Insiue for Daa Analysis and Process Design Zurich Universiy of Applied Sciences CH-8401 Winerhur

2

3 Table of Conens 1 INTRODUCTION PURPOSE 1 1. EXAMPLES 1.3 GOALS IN TIME SERIES ANALYSIS 8 MATHEMATICAL CONCEPTS 11.1 DEFINITION OF A TIME SERIES 11. STATIONARITY 11.3 TESTING STATIONARITY 13 3 TIME SERIES IN R TIME SERIES CLASSES DATES AND TIMES IN R DATA IMPORT 1 4 DESCRIPTIVE ANALYSIS VISUALIZATION 3 4. TRANSFORMATIONS DECOMPOSITION AUTOCORRELATION PARTIAL AUTOCORRELATION 60 5 STATIONARY TIME SERIES MODELS WHITE NOISE ESTIMATING THE CONDITIONAL MEAN AUTOREGRESSIVE MODELS MOVING AVERAGE MODELS ARMA(P,Q) MODELS 87 6 SARIMA AND GARCH MODELS ARIMA MODELS SARIMA MODELS ARCH/GARCH MODELS 98 7 TIME SERIES REGRESSION WHAT IS THE PROBLEM? FINDING CORRELATED ERRORS COCHRANE ORCUTT METHOD 114

4 7.4 GENERALIZED LEAST SQUARES MISSING PREDICTOR VARIABLES 11 8 FORECASTING FORECASTING ARMA EXPONENTIAL SMOOTHING MULTIVARIATE TIME SERIES ANALYSIS PRACTICAL EXAMPLE CROSS CORRELATION PREWHITENING TRANSFER FUNCTION MODELS SPECTRAL ANALYSIS DECOMPOSING IN THE FREQUENCY DOMAIN STATE SPACE MODELS STATE SPACE FORMULATION AR PROCESSES WITH MEASUREMENT NOISE DYNAMIC LINEAR MODELS 167

5 1 Inroducion 1 Inroducion 1.1 Purpose Time series daa, i.e. records which are measured sequenially over ime, are exremely common. They arise in virually every applicaion field, such as e.g.: Business Sales figures, producion numbers, cusomer frequencies,... Economics Sock prices, exchange raes, ineres raes,... Official Saisics Census daa, personal expendiures, road casualies,... Naural Sciences Populaion sizes, sunspo aciviy, chemical process daa,... Environmerics Precipiaion, emperaure or polluion recordings,... In conras o basic daa analysis where he assumpion of idenically and independenly disribued daa is key, ime series are serially correlaed. The purpose of ime series analysis is o visualize and undersand hese dependences in pas daa, and o exploi hem for forecasing fuure values. While some simple descripive echniques do ofen considerably enhance he undersanding of he daa, a full analysis usually involves modeling he sochasic mechanism ha is assumed o be he generaor of he observed ime series. Page 1

6 1 Inroducion Once a good model is found and fied o daa, he analys can use ha model o forecas fuure values and produce predicion inervals, or he can generae simulaions, for example o guide planning decisions. Moreover, fied models are used as a basis for saisical ess: hey allow deermining wheher flucuaions in monhly sales provide evidence of some underlying change, or wheher hey are sill wihin he range of usual random variaion. The dominan main feaures of many ime series are rend and seasonal variaion. These can eiher be modeled deerminisically by mahemaical funcions of ime, or are esimaed using non-parameric smoohing approaches. Ye anoher key feaure of mos ime series is ha adjacen observaions end o be correlaed, i.e. serially dependen. Much of he mehodology in ime series analysis is aimed a explaining his correlaion using appropriae saisical models. While he heory on mahemaically oriened ime series analysis is vas and may be sudied wihou necessarily fiing any models o daa, he focus of our course will be applied and direced owards daa analysis. We sudy some basic properies of ime series processes and models, bu mosly focus on how o visualize and describe ime series daa, on how o fi models o daa correcly, on how o generae forecass, and on how o adequaely draw conclusions from he oupu ha was produced. 1. Examples 1..1 Air Passenger Bookings The numbers of inernaional passenger bookings (in housands) per monh on an airline (PanAm) in he Unied Saes were obained from he Federal Aviaion Adminisraion for he period The company used he daa o predic fuure demand before ordering new aircraf and raining aircrew. The daa are available as a ime series in R. Here, we here show how o access hem, and how o firs gain an impression. > daa(airpassengers) > AirPassengers Jan Feb Mar Apr May Jun Jul Aug Sep Oc Nov Dec Page

7 1 Inroducion Some furher informaion abou his daase can be obained by yping?airpassengers in R. The daa are sored in an R-objec of class s, which is he specific class for ime series daa. However, for furher deails on how ime series are handled in R, we refer o secion 3. One of he mos imporan seps in ime series analysis is o visualize he daa, i.e. creae a ime series plo, where he air passenger bookings are ploed versus he ime of booking. For a ime series objec, his can be done very simply in R, using he generic plo funcion: > plo(airpassengers, ylab="pax", main="passenger Bookings") The resul is displayed on he nex page. There are a number of feaures in he plo which are common o many ime series. For example, i is apparen ha he number of passengers ravelling on he airline is increasing wih ime. In general, a sysemaic change in he mean level of a ime series ha does no appear o be periodic is known as a rend. The simples model for a rend is a linear increase or decrease, an ofen adequae approximaion. We will discuss how o esimae rends, and how o decompose ime series ino rend and oher componens in secion 4.3. The daa also show a repeaing paern wihin each year, i.e. in summer, here are always more passengers han in winer. This is known as a seasonal effec, or seasonaliy. Please noe ha his erm is applied more generally o any repeaing paern over a fixed period, such as for example resauran bookings on differen days of week. Passenger Bookings Pax Time We can naurally aribue he increasing rend of he series o causes such as rising prosperiy, greaer availabiliy of aircraf, cheaper flighs and increasing populaion. The seasonal variaion coincides srongly wih vacaion periods. For his reason, we here consider boh rend and seasonal variaion as deerminisic Page 3

8 1 Inroducion componens. As menioned before, secion 4.3 discusses visualizaion and esimaion of hese componens, while in secion 7, ime series regression models will be specified o allow for underlying causes like hese, and finally secion 8 discusses exploiing hese for predicive purposes. 1.. Lynx Trappings The nex series which we consider here is he annual number of lynx rappings for he years in Canada. We again load he daa and visualize hem using a ime series plo: > daa(lynx) > plo(lynx, ylab="# of Lynx Trapped", main="lynx Trappings") The plo on he nex page shows ha he number of rapped lynx reaches high and low values every abou 10 years, and some even larger figure every abou 40 years. To our knowledge, here is no fixed naural period which suggess hese resuls. Thus, we will aribue his behavior no o a deerminisic periodiciy, bu o a random, sochasic one. Lynx Trappings # of Lynx Trapped Time This leads us o he hear of ime series analysis: while undersanding and modeling rend and seasonal variaion is a very imporan aspec, much of he ime series mehodology is aimed a saionary series, i.e. daa which do no show deerminisic, bu only random (cyclic) variaion. Page 4

9 1 Inroducion 1..3 Lueinizing Hormone Measuremens One of he key feaures of he above lynx rappings series is ha he observaions apparenly do no sem from independen random variables, bu here is some serial correlaion. If he previous value was high (or low, respecively), he nex one is likely o be similar o he previous one. To explore, model and exploi such dependence lies a he roo of ime series analysis. We here show anoher series, where 48 lueinizing hormone levels were recorded from blood samples ha were aken a 10 minue inervals from a human female. This hormone, also called luropin, riggers ovulaion. > daa(lh) > lh Time Series: Sar = 1; End = 48; Frequency = 1 [1] [15] [9] [43] Again, he daa hemselves are of course needed o perform analyses, bu provide lile overview. We can improve his by generaing a ime series plo: > plo(lh, ylab="lh level", main="lueinizing Hormone") Lueinizing Hormone LH level Time For his series, given he way he measuremens were made (i.e. 10 minue inervals), we can almos cerainly exclude any deerminisic seasonal variaion. Bu is here any sochasic cyclic behavior? This quesion is more difficul o answer. Normally, one resors o he simpler quesion of analyzing he correlaion of subsequen records, called auocorrelaions. The auocorrelaion for lag 1 can be visualized by producing a scaerplo of adjacen observaions: Page 5

10 1 Inroducion > plo(lh[1:47], lh[:48], pch=0) > ile("scaerplo of LH Daa wih Lag 1") Scaerplo of LH Daa wih Lag 1 lh[:48] lh[1:47] Besides he (non-sandard) observaion ha here seems o be an inhomogeneiy, i.e. wo disinc groups of daa poins, i is apparen ha here is a posiive correlaion beween successive measuremens. This manifess iself wih he clearly visible fac ha if he previous observaion was above or below he mean, he nex one is more likely o be on he same side. We can even compue he value of he Pearson correlaion coefficien: > cor(lh[1:47], lh[:48]) [1] This figure is an esimae for he so-called auocorrelaion coefficien a lag 1. As we will see in secion 4.4, he idea of considering lagged scaerplos and compuing Pearson correlaion coefficiens serves as a good proxy for a mahemaically more sound mehod. We also noe ha despie he posiive correlaion of +0.58, he series seems o always have he possibiliy of revering o he oher side of he mean, a propery which is common o saionary series an issue ha will be discussed in secion Swiss Marke Index The SMI is he blue chip index of he Swiss sock marke. I summarizes he value of he shares of he 0 mos imporan companies, and currenly conains nearly 90% of he oal marke capializaion. I was inroduced on July 1, 1988 a a basis level of poins. Daily closing daa for 1860 consecuive rading days from are available in R. We observe a more han 4-fold increase during ha period. As a side noe, he value in he second half of 013 is around 8000 poins, indicaing a sidewards movemen over he laes 15 years. Page 6

11 1 Inroducion > daa(eusockmarkes) > EuSockMarkes Time Series: Sar = c(1991, 130) End = c(1998, 169) Frequency = 60 DAX SMI CAC FTSE As we can see, EuSockMarkes is a muliple ime series objec, which also conains daa from he German DAX, he French CAC and UK s FTSE. We will focus on he SMI and hus exrac and plo he series: esm <- EuSockMarkes mp <- EuSockMarkes[,] smi <- s(mp, sar=sar(esm), freq=frequency(esm)) plo(smi, main="smi Daily Closing Value") Because subseing from a muliple ime series objec resuls in a vecor, bu no a ime series objec, we need o regenerae a laer one, sharing he argumens of he original. In he plo we clearly observe ha he series has a rend, i.e. he mean is obviously non-consan over ime. This is ypical for all financial ime series. SMI Daily Closing Value smi Time Such rends in financial ime series are nearly impossible o predic, and difficul o characerize mahemaically. We will no embark in his, bu analyze he so-called log-reurns, i.e. he logged-value of oday s value divided by he one of yeserday: Page 7

12 1 Inroducion > lre.smi <- diff (log(smi)) > plo(lre.smi, main="smi Log-Reurns") SMI Log-Reurns lre.smi Time The SMI log-reurns are a close approximaion o he relaive change (percen values) wih respec o he previous day. As can be seen above, hey do no exhibi a rend anymore, bu show some of he sylized facs ha mos log-reurns of financial ime series share. Using lagged scaerplos or he correlogram (o be discussed laer in secion 4.4), you can convince yourself ha here is no serial correlaion. Thus, here is no direc dependency which could be exploied o predic omorrows reurn based on he one of oday and/or previous days. However, i is visible ha large changes, i.e. log-reurns wih high absolue values, imply ha fuure log-reurns end o be larger han normal, oo. This feaure is also known as volailiy clusering, and financial service providers are rying heir bes o exploi his propery o make profi. Again, you can convince yourself of he volailiy clusering effec by aking he squared log-reurns and analyzing heir serial correlaion, which is differen from zero. 1.3 Goals in Time Series Analysis A firs impression of he purpose and goals in ime series analysis could be gained from he previous examples. We conclude his inroducory secion by explicily summarizing he mos imporan goals Exploraory Analysis Exploraory analysis for ime series mainly involves visualizaion wih ime series plos, decomposiion of he series ino deerminisic and sochasic pars, and sudying he dependency srucure in he daa. Page 8

13 1 Inroducion 1.3. Modeling The formulaion of a sochasic model, as i is for example also done in regression, can and does ofen lead o a deeper undersanding of he series. The formulaion of a suiable model usually arises from a mixure beween background knowledge in he applied field, and insigh from exploraory analysis. Once a suiable model is found, a cenral issue remains, i.e. he esimaion of he parameers, and subsequen model diagnosics and evaluaion Forecasing An ofen-heard moivaion for ime series analysis is he predicion of fuure observaions in he series. This is an ambiious goal, because ime series forecasing relies on exrapolaion, and is generally based on he assumpion ha pas and presen characerisics of he series coninue. I seems obvious ha good forecasing resuls require a very good comprehension of a series properies, be i in a more descripive sense, or in he sense of a fied model Time Series Regression Raher han jus forecasing by exrapolaion, we can ry o undersand he relaion beween a so-idenified response ime series, and one or more explanaory series. If all of hese are observed a he same ime, we can in principle employ he ordinary leas squares (OLS) regression framework. However, he all-o-common assumpion of (serially) uncorrelaed errors in OLS is usually violaed in a ime series seup. We will illusrae how o properly deal wih his siuaion, in order o generae correc confidence and predicion inervals Process Conrol Many producion or oher processes are measured quaniaively for he purpose of opimal managemen and qualiy conrol. This usually resuls in ime series daa, o which a sochasic model is fi. This allows undersanding he signal in he daa, bu also he noise: i becomes feasible o monior which flucuaions in he producion are normal, and which ones require inervenion. Page 9

14

15 Mahemaical Conceps Mahemaical Conceps For performing anyhing else han very basic exploraory ime series analysis, even from a much applied perspecive, i is necessary o inroduce he mahemaical noion of wha a ime series is, and o sudy some basic probabilisic properies, namely he momens and he concep of saionariy..1 Definiion of a Time Series As we have explained in secion 1., observaions ha have been colleced over fixed sampling inervals form a ime series. Following a saisical approach, we consider such series as realizaions of random variables. A sequence of random variables, defined a such fixed sampling inervals, is someimes referred o as a discree-ime sochasic process, hough he shorer names ime series model or ime series process are more popular and will mosly be used in his scripum. I is very imporan o make he disincion beween a ime series, i.e. observed values, and a process, i.e. a probabilisic consruc. Definiion: A ime series process is a se of random variables X, T, where T is he se of imes a which he process was, will or can be observed. We assume ha each random variable X is disribued according some univariae disribuion funcion F. Please noe ha for our enire course and hence scripum, we exclusively consider ime series processes wih equidisan ime inervals, as well as real-valued random variables X. This allows us o enumerae he se of imes, so ha we can wrie T {1,, 3, }. An observed ime series, on he oher hand, is seen as a realizaion of he random vecor X ( X1, X,, X n ), and is denoed wih small leers x ( x1, x,, xn). I is imporan o noe ha in a mulivariae sense, a ime series is only one single realizaion of he n -dimensional random variable X, wih is mulivariae, n -dimensional disribuion funcion F. As we all know, we canno do saisics wih jus a single observaion. As a way ou of his siuaion, we need o impose some condiions on he join disribuion funcion F.. Saionariy The aforemenioned condiion on he join disribuion F will be formulaed as he concep of saionariy. In colloquial language, saionariy means ha he probabilisic characer of he series mus no change over ime, i.e. ha any secion of he ime series is ypical for every oher secion wih he same lengh. More mahemaically, we require ha for any indices s, and k, he observaions x,, x k could have jus as easily occurred a imes s,, s k. If ha is no he case pracically, hen he series is hardly saionary. Page 11

16 Mahemaical Conceps Imposing even more mahemaical rigor, we inroduce he concep of sric saionariy. A ime series is said o be sricly saionary if and only if he ( k 1) -dimensional join disribuion of X,, X k coincides wih he join disribuion of Xs,, Xs k for any combinaion of indices, s and k. For he special case of k 0 and s, his means ha he univariae disribuions F of all X are equal. For sricly saionary ime series, we can hus leave off he index on he disribuion. As he nex sep, we will define he uncondiional momens: Expecaion EX [ ], Variance Var( X ), Covariance () h Cov( X, X ). h In oher words, sricly saionary series have consan (uncondiional) expecaion, consan (uncondiional) variance, and he covariance, i.e. he dependency srucure, depends only on he lag h, which is he ime difference beween he wo observaions. However, he covariance erms are generally differen from 0, and hus, he X are usually dependen. Moreover, he condiional expecaion given he pas of he series, EX [ X 1, X,...] is ypically non-consan, denoed as. In some (rarer, e.g. for financial ime series) cases, even he condiional variance Var( X X 1, X,...) can be non-consan. In pracice however, excep for simulaion sudies, we usually have no explici knowledge of he laen ime series process. Since sric saionariy is defined as a propery of he process join disribuions (all of hem), i is impossible o verify from an observed ime series, i.e. a single daa realizaion. We can, however, ry o verify wheher a ime series process shows consan uncondiional mean and variance, and wheher he dependency only depends on he lag h. This much less rigorous propery is known as weak saionariy. In order o do well-founded saisical analyses wih ime series, weak saionariy is a necessary condiion. I is obvious ha if a series observaions do no have common properies such as consan mean/variance and a sable dependency srucure, i will be impossible o saisically learn from i. On he oher hand, i can be shown ha weak saionariy, along wih he addiional propery of ergodiciy (i.e. he mean of a ime series realizaion converges o he expeced value, independen of he saring poin), is sufficien for mos pracical purposes such as model fiing, forecasing, ec.. We will, however, no furher embark in his subjec. Remarks: From now on, when we speak of saionariy, we sricly mean weak saionariy. The moivaion is ha weak saionariy is sufficien for applied ime series analysis, and sric saionariy is a pracically useless concep. When we analyze ime series daa, we need o verify wheher i migh have arisen from a saionary process or no. Be careful wih he wording: saionariy is always a propery of he process, and never of he daa. Page 1

17 Mahemaical Conceps Moreover, bear in mind ha saionariy is a hypohesis, which needs o be evaluaed for every series. We may be able o rejec his hypohesis wih quie some cerainy if he daa srongly speak agains i. However, we can never prove saionariy wih daa. A bes, i is plausible ha a series originaed from a saionary process. Some obvious violaions of saionariy are rends, non-consan variance, deerminisic seasonal variaion, as well as apparen breaks in he daa, which are indicaors for changing dependency srucure..3 Tesing Saionariy If, as explained above, saionariy is a hypohesis which is esed on daa, sudens and users keep asking if here are any formal ess. The answer o his quesion is yes, and here are even quie a number of ess. This includes he Augmened Dickey-Fuller Tes, he Phillips-Perron Tes, he KPSS Tes, which are all available in R s series package. The urca package includes furher ess such as he Ellio-Rohenberg-Sock, Schmid-Phillips und Zivo-Andrews. However, we will no discuss any of hese ess here for a variey of reasons. Firs and foremos, hey all focus on some very specific non-saionariy aspecs, bu do no es saionariy in a broad sense. While hey may reasonably do heir job in he narrow field hey are aimed for, hey have low power o deec general nonsaionariy and in pracice ofen fail o do so. Addiionally, heory and formalism of hese ess is quie complex, and hus beyond he scope of his course. In summary, hese ess are o be seen as more of a pasime for he mahemaically ineresed, raher han a useful ool for he praciioner. Thus, we here recommend assessing saionariy by visual inspecion. The primary ool for his is he ime series plo, bu also he correlogram (see secion 4.4) can be helpful as a second check. For long ime series, i can also be useful o spli up he series ino several pars for checking wheher mean, variance and dependency are similar over he blocks. Page 13

18

19 3 Time Series in R 3 Time Series in R 3.1 Time Series Classes In R, here are objecs, which are organized in a large number of classes. These classes e.g. include vecors, daa frames, model oupu, funcions, and many more. No surprisingly, here are also several classes for ime series. We sar by presening s, he basic class for regularly spaced ime series. This class is comparably simple, as i can only represen ime series wih fixed inerval records, and only uses numeric ime samps, i.e. (sophisically) enumeraes he index se. However, i will be sufficien for mos, if no all, of wha we do in his course. Then, we also provide an oulook o more complicaed conceps The s Class For defining a ime series of class s, we of course need o provide he daa, bu also he saring ime as argumen sar, and he frequency of measuremens as argumen frequency. If no saring ime is supplied, R uses is defaul value of 1, i.e. enumeraes he imes by he index se 1,..., n, where n is he lengh of he series. The frequency is he number of observaions per uni of ime, e.g. 1 for yearly, 4 for quarerly, or 1 for monhly recordings. Insead of he sar, we could also provide he end of he series, and insead of he frequency, we could supply argumen dela, he fracion of he sampling period beween successive observaions. The following example will illusrae he concep. Example: We here consider a simple and shor series ha holds he number of days per year wih raffic holdups in fron of he Gohard road unnel norh enrance in Swizerland. The daa are available from he Federal Roads Office The sar of his series is in 004. The ime uni is years, and since we have jus one record per year, he frequency of his series is 1. This ells us ha while here may be a rend, here canno be a seasonal effec, as he laer can only be presen in periodic series, i.e. series wih frequency > 1. We now define a s objec in in R. > rawda <- c(88, 76, 11, 109, 91, 98, 139) > s.da <- s(rawda, sar=004, freq=1) > s.da Time Series: Sar = 004, End = 010 Frequency = 1 [1] Page 15

20 3 Time Series in R There are a number of simple bu useful funcions ha exrac basic informaion from objecs of class s, see he following examples: > sar(s.da) [1] > end(s.da) [1] > frequency(s.da) [1] 1 > dela(s.da) [1] 1 Anoher possibiliy is o obain he measuremen imes from a ime series objec. As class s only enumeraes he imes, hey are given as fracions. This can sill be very useful for specialized plos, ec. > ime(s.da) Time Series: Sar = 004 End = 010 Frequency = 1 [1] The nex basic, bu for pracical purposes very useful funcion is window(). I is aimed a selecing a subse from a ime series. Of course, also regular R- subseing such as s.da[:5] does work wih he ime series class. However, his resuls in a vecor raher han a ime series objec, and is hus mosly of less use han he window() command. > window(s.da, sar=006, end=008) Time Series: Sar = 006 End = 008 Frequency = 1 [1] While we here presened he mos imporan basic mehods/funcions for class s, here is a wealh of furher ones. This includes he plo() funcion, and many more, e.g. for esimaing rends, seasonal effecs and dependency srucure, for fiing ime series models and generaing forecass. We will presen hem in he forhcoming chapers of his scripum. To conclude he previous example, we will no do wihou showing he ime series plo of he Gohard road unnel raffic holdup days, see nex page. Because here are a limied number of observaions, i is difficul o give saemens regarding a possible rend and/or sochasic dependency. > plo(s.da, ylab="# of Days", main="traffic Holdups") Page 16

21 3 Time Series in R Traffic Holdups # of Days Time 3.1. Oher Classes Besides he basic s class, here are several more which offer a variey of addiional opions, bu will rarely o never be required during our course. Mos prominenly, his includes he zoo package, which provides infrasrucure for boh regularly and irregularly spaced ime series using arbirary classes for he ime samps. I is designed o be as consisen as possible wih he s class. Coercion from and o zoo is also readily available. Some furher packages which conain classes and mehods for ime series include xs, is, series, fs, imeseries and is. Addiional informaion on heir conen and philosophy can be found on CRAN. 3. Daes and Times in R While for he s class, he handling of imes has been solved very simply and easily by enumeraing, doing ime series analysis in R may someimes also require o explicily working wih dae and ime. There are several opions for dealing wih dae and dae/ime daa. The buil-in as.dae() funcion handles daes ha come wihou imes. The conribued package chron handles daes and imes, bu does no conrol for differen ime zones, whereas he sophisicaed bu complex POSIXc and POSIXl classes allow for daes and imes wih ime zone conrol. As a general rule for dae/ime daa in R, we sugges o use he simples echnique possible. Thus, for dae only daa, as.dae() will mosly be he opimal choice. If handling daes and imes, bu wihou ime-zone informaion, is required, he chron package is he choice. The POSIX classes are especially useful in he relaively rare cases when ime-zone manipulaion is imporan. Page 17

22 3 Time Series in R Apar for he POSIXl class, daes/imes are inernally sored as he number of days or seconds from some reference dae. These daes/imes hus generally have a numeric mode. The POSIXl class, on he oher hand, sores dae/ime values as a lis of componens (hour, min, sec, mon, ec.), making i easy o exrac hese pars. Also he curren dae is accessible by yping Sys.Dae() in he console, and reurns an objec of class Dae The Dae Class As menioned above, he easies soluion for specifying days in R is wih he as.dae() funcion. Using he forma argumen, arbirary dae formas can be read. The defaul, however, is four-digi year, followed by monh and hen day, separaed by dashes or slashes: > as.dae(" ") [1] " " > as.dae("01/0/07") [1] " " If he daes come in non-sandard appearance, we require defining heir forma using some codes. While he mos imporan ones are shown below, we reference o he R help file of funcion srpime() for he full lis. Code Value %d Day of he monh (decimal number) %m Monh (decimal number) %b Monh (characer, abbreviaed) %B Monh (characer, full name) %y Year (decimal, wo digi) %Y Year (decimal, four digi) The following examples illusrae he use of he forma argumen: > as.dae("7.01.1", forma="%d.%m.%y") [1] " " > as.dae("14. Februar, 01", forma="%d. %B, %Y") [1] " " Inernally, Dae objecs are sored as he number of days passed since he 1 s of January in Earlier daes receive negaive numbers. By using he as.numeric() funcion, we can easily find ou how many days are pas since he reference dae. Also back-conversion from a number of pas days o a dae is sraighforward: > myda <- as.dae(" ") > ndays <- as.numeric(myda) > ndays [1] Page 18

23 3 Time Series in R > days < > class(days) <- "Dae" > days [1] " " A very useful feaure is he possibiliy of exracing weekdays, monhs and quarers from Dae objecs, see he examples below. This informaion can be convered o facors. In his form, hey serve for purposes such as visualizaion, decomposiion, or ime series regression. > weekdays(myda) [1] "Diensag" > monhs(myda) [1] "Februar" > quarers(myda) [1] "Q1" Furhermore, some very useful summary saisics can be generaed from Dae objecs: median, mean, min, max, range,... are all available. We can even subrac wo daes, which resuls in a diffime objec, i.e. he ime difference in days. > da <- as.dae(c(" "," "," ")) > da [1] " " " " " " > min(da) [1] " " > max(da) [1] " " > mean(da) [1] " " > median(da) [1] " " > da[3]-da[1] Time difference of 777 days Anoher opion is generaing ime sequences. For example, o generae a vecor of 1 daes, saring on Augus 3, 1985, wih an inerval of one single day beween hem, we simply ype: > seq(as.dae(" "), by="days", lengh=1) [1] " " " " " " " " [5] " " " " " " " " [9] " " " " " " " " The by argumen proves o be very useful. We can supply various unis of ime, and even place an ineger in fron of i. This allows creaing a sequence of daes separaed by wo weeks: Page 19

24 3 Time Series in R > seq(as.dae(" "), by=" weeks", lengh=1) [1] " " " " " " " " [5] " " " " " " " " [9] " " " " " " " " 3.. The chron Package The chron() funcion convers daes and imes o chron objecs. The daes and imes are provided separaely o he chron() funcion, which may well require some inial pre-processing. For such parsing, R-funcions such as subsr() and srspli() can be of grea use. In he chron package, here is no suppor for ime zones and dayligh savings ime, and chron objecs are inernally sored as fracional days since he reference dae of January 1 s, By using he funcion as.numeric(), hese inernal values can be accessed. The following example illusraes he use of chron: > library(chron) > da <- c(" :43:0", " ::40", " :48:40", " :18:50") > ds <- subsr(da, 1, 10) > me <- subsr(da, 1, 19) > fm <- c("y-m-d","h:m:s") > cd <- chron(daes=ds, ime=me, forma=fm) > cd [1] ( :43:0) ( ::40) [3] ( :48:40) ( :18:50) As before, we can again use he enire palee of summary saisic funcions. Of some special ineres are ime differences, which can now be obained as eiher fracion of days, or in weeks, hours, minues, seconds, ec.: > cd[]-cd[1] Time in days: [1] > diffime(cd[], cd[1], unis="secs") Time difference of secs 3..3 POSIX Classes The wo classes POSIXc and POSIXl implemen dae/ime informaion, and in conras o he chron package, also suppor ime zones and dayligh savings ime. We recommend uilizing his funcionaliy only when urgenly needed, because he handling requires quie some care, and may on op of ha be sysem dependen. Furher deails on he use of he POSIX classes can be found on CRAN. As explained above, he POSIXc class also sores daes/imes wih respec o he inernal reference, whereas he POSIXl class sores hem as a lis of componens (hour, min, sec, mon, ec.), making i easy o exrac hese pars. Page 0

25 3 Time Series in R 3.3 Daa Impor We can safely assume ha mos ime series daa are already presen in elecronic form; however, no necessarily in R. Thus, some knowledge on how o impor daa ino R is required. I is be beyond he scope of his scripum o presen he uncouned opions which exis for his ask. Hence, we will resric ourselves o providing a shor overview and some useful hins. The mos common form for sharing ime series daa are cerainly spreadshees, or in paricular, Microsof Excel files. While library(robdc) offers funcionaliy o direcly impor daa from Excel files, we discourage is use. Firs of all, his only works on Windows sysems. More imporanly, i is usually simpler, quicker and more flexible o expor comma- or ab-separaed ex files from Excel, and impor hem via he ubiquious read.able() funcion, respecively he ailored version read.csv() (for comma separaion) and read.delim() (for ab separaion). Wih packages ROBDC and RMySQL, R can also communicae wih SQL daabases, which is he mehod of choice for large scale problems. Furhermore, afer loading library(foreign), i is also possible o read files from Saa, SPSS, Ocave and SAS. Page 1

26

27 4 Descripive Analysis 4 Descripive Analysis As always when working wih a pile of numbers, also known as daa, i is imporan o firs gain an overview. In he field of ime series analysis, his encompasses several aspecs: undersanding he conex of he problem and he daa source making suiable plos, looking for general srucure and ouliers hinking abou daa ransformaions, e.g. o reduce skewness judging saionariy and poenially achieve i by decomposiion We sar by discussing ime series plos, hen discuss ransformaions, focus on he decomposiion of ime series ino rend, seasonal effec and saionary random par and conclude by discussing mehods for visualizing he dependency srucure. 4.1 Visualizaion Time Series Plo The mos imporan means of visualizaion is he ime series plo, where he daa are ploed versus ime/index. There are several examples in secion 1., where we also go acquained wih R s generic plo() funcion. As a general rule, he daa poins are joined by lines in ime series plos. An excepion is when here are missing values. Moreover, he reader expecs ha he axes are well-chosen, labeled and he measuremen unis are given. Anoher issue is he correc aspec raio for ime series plos: if he ime axis ges oo much compressed, i can become difficul o recognize he behavior of a series. Thus, we recommend choosing he aspec raio appropriaely. However, here are no hard and simple rules on how o do his. As a rule of he humb, use he banking o 45 degrees paradigm: increase and decrease in periodic series should no be displayed a angles much higher or lower han 45 degrees. For very long series, his can become difficul on eiher A4 paper or a compuer screen. In his case, we recommend spliing up he series and display i in differen frames. For illusraion, we here show an example, he monhly unemploymen rae in he US sae of Maine, from January 1996 unil Augus 006. The daa are available from a ex file on he web. We can read i direcly ino R, define he daa as an objec of class s and hen do he ime series plo: > www <- "hp://saff.elena.au.ac.nz/paul-cowperwai/s/" > da <- read.able(pase(www,"maine.da",sep="", header=t) > sd <- s(da, sar=c(1996,1), freq=1) > plo(sd, ylab="(%)", main="unemploymen in Maine") Page 3

28 4 Descripive Analysis Unemploymen in Maine (%) Time No surprisingly for monhly economic daa, he series shows boh seasonal variaion and a non-linear rend. Since unemploymen raes are one of he main economic indicaors used by poliicians/decision makers, his series poses a worhwhile forecasing problem Muliple Time Series Plos In applied problems, one is ofen provided wih muliple ime series. Here, we illusrae some basics on impor, definiion and ploing. Our example exhibis he monhly supply of elecriciy (millions of kwh), beer (millions of liers) and chocolae-based producion (onnes) in Ausralia over he period from January 1958 o December These daa are available from he Bureau of Ausralian Saisics and are, in pre-processed form, accessible as a ex-file online. www <- "hp://saff.elena.au.ac.nz/paul-cowperwai/s/" da <- read.able(pase(www,"cbe.da",sep="", header=t) sd <- s(da, sar=1958, freq=1) plo(sd, main="chocolae, Beer & Elecriciy") All hree series show a disinc seasonal paern, along wih a rend. I is also insrucive o know ha he Ausralian populaion increased by a facor of 1.8 during he period where hese hree series were observed. As visible in he bi of code above, ploing muliple series ino differen panels is sraighforward. As a general rule, using differen frames for muliple series is he mos recommended means of visualizaion. However, someimes i can be more insrucive o have hem in he same frame. Of course, his requires ha he series are eiher on he same scale, or have been indexed, resp. sandardized o be so. While R offers funcion s.plo() o include muliple series in he same frame, ha funcion does no allow color coding. For his reason, we prefer doing some manual work. Page 4

29 4 Descripive Analysis Chocolae, Beer & Elecriciy beer choc elec Time ## Indexing he series sd[,1] <- sd[,1]/sd[1,1]*100 sd[,] <- sd[,]/sd[1,]*100 sd[,3] <- sd[,3]/sd[1,3]*100 ## Ploing in one single frame clr <- c("green3", "red3", "blue3") plo.s(sd[,1], ylim=range(sd), ylab="index", col=clr[1]) ile("indexed Chocolae, Beer & Elecriciy") lines(sd[,], col=clr[]); lines(sd[,3], col=clr[3]) ## Legend lx <- names(da) legend("oplef", ly=1, col=clr, legend=lx) Indexed Chocolae, Beer & Elecriciy Index choc beer elec Time Page 5

30 4 Descripive Analysis In he indexed single frame plo above, we can very well judge he relaive developmen of he series over ime. Due o differen scaling, his was nearly impossible wih he muliple frames on he previous page. We observe ha elecriciy producion increased around 8x during 1958 and 1990, whereas for chocolae he muliplier is around 4x, and for beer less han x. Also, he seasonal variaion is mos pronounced for chocolae, followed by elecriciy and hen beer. 4. Transformaions Many popular ime series models are based on he Gaussian disribuion and linear relaions beween he variables. However, daa may exhibi differen behavior. In such cases, we can ofen improve he fi by no using he original daa x,..., 1 x n, bu a ransformed version gx ( 1),..., gx ( n). The mos popular and pracically relevan ransformaion is g() log(). I is indicaed if eiher he variaion in he series grows wih increasing mean, or if he marginal disribuion appears o be righ-skewed. Boh properies ofen appear in daa ha can ake posiive values only, such as he lynx rappings from secion 1... I is easy o spo righskewness by hisograms and QQ-plos: > his(lynx, col="lighblue") > qqnorm(lynx, pch=0); qqline(lynx, col="blue") Hisogram of lynx Normal Q-Q Plo Frequency Sample Quaniles lynx Theoreical Quaniles The lynx daa show some very srong righ-skewness and hence, a logransformaion is indicaed. Of course, i was no wrong o use he original scale for a ime series plo, bu when i comes o esimaing auocorrelaions or esimaing ime series models, i is clearly beer o log-ransform he daa firs. This is very easy in R: > plo(log(lynx)) > ile("logged Lynx Trappings") Page 6

31 4 Descripive Analysis Logged Lynx Trappings log(lynx) Time The daa now follow a more symmerical paern; he exreme upward spikes are all gone. We will use hese ransformed daa o deermine he auocorrelaion and o generae forecass. However, please be aware of he fac ha backransforming fied or prediced (model) values o he original scale by jus aking exp( ) usually leads o biased resuls, unless a correcion facor is used. An indeph discussion of ha issue is conained in chaper Decomposiion The Basics We have learned in secion. ha saionariy is an imporan prerequisie for being able o saisically learn from ime series daa. However, many of he example series exhibi eiher rend and/or seasonal effec, and hus are nonsaionary. In his secion, we will learn how o deal wih ha. I is achieved by using decomposiion models, he easies of which is he simple addiive one: X m s R, where X is he ime series process a ime, m is he rend, s is he seasonal effec, and R is he remainder, i.e. a sequence of usually correlaed random variables wih mean zero. The goal is o find a decomposiion such ha R is a saionary ime series process. Such a model migh be suiable for all he monhlydaa series we go acquained wih so far: air passenger bookings, unemploymen in Maine and Ausralian producion. However, closer inspecion of all hese series exhibis ha he seasonal effec and he random variaion increase as he rend increases. In such cases, a muliplicaive decomposiion model is beer: X ms R Page 7

32 4 Descripive Analysis Empirical experience says ha aking logarihms is beneficial for such daa. Also, some basic mah shows ha his brings us back o he addiive case: log( X ) log( m s R) log( m) log( s ) log( R) m s R For illusraion, we carry ou he log-ransformaion on he air passenger bookings: > plo(log(airpassengers), ylab="log(pax)", main=...) Logged Passenger Bookings log(pax) Time Indeed, seasonal effec and random variaion now seem o be independen of he level of he series. Thus, he muliplicaive model is much more appropriae han he addiive one. However, a furher snag is ha he seasonal effec seems o aler over ime raher han being consan. Tha issue will be addressed laer Differencing A simple approach for removing deerminisic rends and/or seasonal effecs from a ime series is by aking differences. A pracical inerpreaion of aking differences is ha by doing so, he changes in he daa will be moniored, bu no longer he series iself. While his is concepually simple and quick o implemen, he main disadvanage is ha i does no resul in explici esimaes of he rend componen m and he seasonal componen s. We will firs urn our aenion o series wih an addiive rend, bu wihou seasonal variaion. By aking firs-order differences wih lag 1, and assuming a rend wih lile shor-erm changes, i.e. m m 1, we have: X m R Y X X R R 1 1 Page 8

33 4 Descripive Analysis In pracice, his kind of differencing approach mosly works, i.e. manages o reduce presence of a rend in he series in a saisfacory manner. However, he rend is only fully removed if i is exacly linear, i.e. m. Then, we obain: Y X X R R 1 1 Anoher somewha disurbing propery of he differencing approach is ha srong, arificial new dependencies are creaed, meaning ha he auocorrelaion in Y is differen from he one in R. For illusraion, consider a sochasically independen remainder R : he differenced process Y has auocorrelaion! Cov( Y, Y ) Cov( R R, R R ) Cov( R, R ) 1 1 We illusrae how differencing works by using a daase ha shows he raffic developmen on Swiss roads. The daa are available from he federal road office (ASTRA) and show he indexed raffic amoun from We ype in he values and plo he original series: > SwissTraffic <- s(c(100.0, 10.7, 104., 104.6, 106.7, 106.9, 107.6, 109.9, 11.0, 114.3, 117.4, 118.3, 10.9, 13.7, 14.1, 14.6, 15.6, 17.9, 17.4, 130., 131.3), sar=1990, freq=1) > plo(swisstraffic) Swiss Traffic Index Index Value Time There is a clear rend, which is close o linear, hus he simple approach should work well here. Taking firs-order differences wih lag 1 shows he yearly changes in he Swiss Traffic Index, which mus now be a saionary series. In R, he job is done wih funcion diff(). Page 9

34 4 Descripive Analysis > diff(swisstraffic) Time Series: Sar = 1991 End = 010 Frequency = 1 [1] [11] Differenced Swiss Traffic Index Change Time Please noe ha he ime series of differences is now 1 insance shorer han he original series. The reason is ha for he firs year, 1990, here is no difference o he previous year available. The differenced series now clearly has a consan mean, i.e. he rend was successfully removed. Log-Transformaion and Differencing On a sidenoe, we consider a series ha was log-ransformed firs, before firsorder differences wih lag 1 were aken. An example is he SMI daa ha were shown in secion The resul is he so-called log reurn, which is an approximaion o he relaive change, i.e. he percen in- or decrease wih respec o he previous insance. In paricular: X X X X X Y log( X) log( X 1) log log 1 X 1 X 1 X The approximaion of he log reurn o he relaive change is very good for small changes, and becomes a lile less precise wih larger values. For example, if we have a 0.00% relaive change, hen Y 0.00%, for 1.00% relaive change we obain Y 0.995% and for 5.00%, Y 4.88%. We conclude wih summarizing ha for any non-saionary series which is also due o a log-ransformaion, he ransformaion is always carried ou firs, and hen followed by he differencing! Page 30

35 4 Descripive Analysis The Backshif Operaor We here inroduce he backshif operaor B because i allows for convenien noaion. When he operaor B is applied o X i reurns he insance a lag 1, i.e. BX ( ). X 1 Less mahemaically, we can also say ha applying B means go back one sep, or incremen he ime series index by -1. The operaion of aking firs-order differences a lag 1 as above can be wrien using he backshif operaor: Y (1 B) X X X 1 However, he main aim of he backshif operaor is o deal wih more complicaed forms of differencing, as will be explained below. Higher-Order Differencing We have seen ha aking firs-order differences is able o remove linear rends from ime series. Wha has differencing o offer for polynomial rends, i.e. quadraic or cubic ones? We here demonsrae ha i is possible o ake higher order differences o remove also hese, for example, in he case of a quadraic rend. X 1 R, R saionary Y (1 B) X ( X X 1) ( X 1X) R R R 1 We see ha he operaor (1 B) means ha afer aking normal differences, he resuling series is again differenced normally. This is a discreized varian of aking he second derivaive, and hus i is no surprising ha i manages o remove a quadraic rend from he daa. As we can see, Y is an addiive combinaion of he saionary R s erms, and hus iself saionary. Again, if R was an independen process, ha would clearly no hold for Y, hus aking higher-order differences (srongly!) alers he dependency srucure. Moreover, he exension o cubic rends and even higher orders d is sraighforward. We jus use he (1 B) d operaor applied o series X. In R, we can employ funcion diff(), bu have o provide argumen differences=d for indicaing he order of he difference d. Removing Seasonal Effecs by Differencing For ime series wih monhly measuremens, seasonal effecs are very common. Using an appropriae form of differencing, i is possible o remove hese, as well as poenial rends. We ake firs-order differences wih lag p : Y (1 p B ) X X X p, Page 31

36 4 Descripive Analysis Here, p is he period of he seasonal effec, or in oher words, he frequency of series, which is he number of measuremens per ime uni. The series Y hen is made up of he changes compared o he previous period s value, e.g. he previous year s value. Also, from he definiion, wih he same argumen as above, i is eviden ha no only he seasonal variaion, bu also a sricly linear rend will be removed. Usually, rends are no exacly linear. We have seen ha aking differences a lag 1 removes slowly evolving (non-linear) rends well due o m m 1. However, here he relevan quaniies are m and m p, and especially if he period p is long, some rend will usually be remaining in he daa. Then, furher acion is required. Example We are illusraing seasonal differencing using he Mauna Loa amospheric CO concenraion daa. This is a ime series wih monhly records from January 1959 o December I exhibis boh a rend and a disinc seasonal paern. We firs load he daa and do a ime series plo: > daa(co) > plo(co, main="mauna Loa CO Concenraions") Seasonal differencing is very convenienly available in R. We use funcion diff(), bu have o se argumen lag=... For he Mauna Loa daa wih monhly measuremens, he correc lag is 1. This resuls in he series shown on he nex page. Because we are comparing every record wih he one from he previous year, he resuling series is 1 observaions shorer han he original one. I is prey obvious ha some rend is remaining and hus, he resul from seasonal differencing canno be considered as saionary. As he seasonal effec is gone, we could ry o add some firs-order differencing a lag 1. Mauna Loa CO Concenraions co Time Page 3

37 4 Descripive Analysis > sd.co <- diff(co, lag=1) > plo(sd.co, main="differenced Mauna Loa Daa (p=1)") Differenced Mauna Loa Daa (p=1) sd.co Time The second differencing sep indeed managems o produce a saionary series, as can be seen below. The equaion for he final series is: Z BY B B X. 1 (1 ) (1 )(1 ) The nex sep would be o analyze he auocorrelaion of he series below and fi an ARMA( p, q ) model. Due o he wo differencing seps, such consrucs are also named SARIMA models. They will be discussed in chaper 6. Twice Differenced Mauna Loa Daa (p=1, p=1) d1.sd.co Time Page 33