Time Series Analysis. Session III: Probability models for time series. Carlos Óscar Sánchez Sorzano, Ph.D. Madrid, July 19th 2006

Transcription

1 Time Series Alysis Sessio III: Proility moels for time series Crlos Óscr Sáche Soro Ph.D. Mri July 9th 6

2 Sessio outlie. Gol. A short itrouctio to system lysis 3. Movig Averge rocesses MA 4. Autoregressive rocesses AR 5. Autoregressive Movig Averge ARMA 6. Autoregressive Itegrte Movig Averge ARIMA FARIMA 7. Sesol Autoregressive Itegrte Movig Averge SARIMA 8. Kow eterl iuts: System ietifictio 9. A fmily of moels. olier moels. Prmeter estimtio. Orer selectio 3. Moel checig

3 . Gol tre erioic rom Elie y sttisticl moels AR MA 3 The gol of the roility moels for time series is to chrcterie the i of romess of the rom comoet. The figure o the right rereset three ifferet stochstic rocesses ll of them re comletely rom. However the i of romess is ifferet from oe to the other. The ifferece comes from their corresoig correltio structure i.e. the utocorreltio fuctio for ech time series is ifferet. The ojective of rmetric moels is to efie cotrolle time series whose correltio structure is similr to the oe of the iut rocess. 3

4 . Gol 4 Although roly oe of us will hve to rogrm y of this time series moels it is ecessry i orer to use the rogrms to ow ectly to wht they re referrig whe they re goig to fit AR moel. For eig le of uerstig wht the rogrm is ectly goig to o is ecessry to review ll the mthemtics tht follow. 4

5 5 5. A short itrouctio to system lysis M y Differece eutio Emle:.5 y y M Y H M Y.5 Y Y.5 Y H Trsfer fuctio C T T y ZT All lier systems c e eresse y ifferece eutio ivolvig the iut series the outut series. If we Z-trsform the ifferece eutio reorgie the eutio we c comute wht is clle the trsfer fuctio of the system. This fuctio comletely efies the ehviour of LTI system. I rticulr it reltes the sectrl cotet of the iut sigl to the sectrl cotet of the outut. Elicitig the efiitio of the Z trsform flls outsie of the scoe of this itrouctory course. The itereste stuet my re Oeheim.

6 . A short itrouctio to system lysis Poles/Zeros is ole of iff H H is ero of H iff H Emle: Stility of LTI systems y 3 H Poles: Zeros: 3 j ± Im{} A cusl system is stle iff ll its oles re isie the uit circle Ivertiility of LTI systems Re{} < The trsfer fuctio of the iverse system of LTI system whose trsfer fuctio is is H. Therefore the eros of oe system re the oles of its iverse vicevers. H 6 Poles eros ly ey role i the lysis of LTI systems sice they efie u to multilyig costt the trsfer fuctio of give system. Furthermore they efie whether cusl LTI system is stle or ot. A iterestig roerty of iverse LTI systems is tht they re the recirocl of the origil system. Thus eros of the origil system ecome oles of the iverse systems vicevers. If we woer out the stility of the iverse system we coul sy tht the iverse of give system is stle iff ll the eros of tht system re isie the uit circle. 6

7 . A short itrouctio to system lysis Dowsmlig M M Usmlig L / L ± L ± L... e δ L resto 7 7

8 3. Movig verge rocesses: MA Defiitio H... B LTI with memory ivertile cusl stle w MA w w... w 3 w tim e tim e tim e ACF w ACF ACF lg lg lg 8 A MA stochstic rocess is oe tht is geerte usig ifferece eutio lie the oe show i the slie. ote tht it oly uses revious smles of the iut sigl. The mi fetures of the ssocite geertig system re tht it is LTI cusl stle. The MA system is FIR therefore ll-ero system. Selectig the right coefficiets my ifferet correltio structures might e crete. The more comle wier is the correltio we wt to rerouce the higher must e the. I the emles show the followig s hve ee use :.9 : The MATLAB coe use for geertig these lots is: fuctio MA_rocess ; wr; geerte_ma_rocessw; geerte_ma_rocessw; sulot3; stem:/w/:; titlew; leltime; sulot3; stem://:; title_; leltime; sulot33; stem://:; title_{}; leltime; K; corr_w corrwcoeff; corr_ corrcoeff; corr_corrcoeff; for :K wr; geerte_ma_rocessw; geerte_ma_rocessw; corr_w corr_w corrwcoeff; corr_ corr_ corrcoeff; corr_corr_corrcoeff; e corr_w corr_w/k; corr_ corr_/k; corr_corr_/k; sulot34; stem-:corr_w-:; titleacf_w_; lellg; is- -..; sulot35; stem-:corr_-:; titleacf_{_}_; lellg; is ; 8

9 Movig verge rocesses: MA δ Γ < Γ < Γ It hs limite suort!! w MA w { } { } Γ w w E w w E E { } w w E δ Sttisticl roerties Proof Oe of the most iterestig thigs of stochstic rocesses is to chrcterie them sttisticlly. ormlly we re oly itereste o their chrcteritio u to seco orer me vrice utocorreltio. If the iut time series is ormlly istriute with ero me give vrice it is esy to show tht the outut time series is lso ormlly istriute with ero me whose vrice is fuctio of the iut vrice the system coefficiets s show i the slie. The comuttio of the utocorreltio fuctio is little it more ivolve. First we show the fil result the we rove it. This roof will oly e erforme for MA rocesses sice for the rest of rocesses comuttios re similrly crrie out.

10 3. Movig verge rocesses: MA < Γ < Γ... δ δ Sttisticl roerties Proof cot.

11 Ivertiility 3. Movig verge rocesses: MA w MA w MA w H Im{} Emle: Re{} H iv H w w w y 3 oes ot hve stle cusl iverse y.9 hs stle cusl iverse Ivertiility of stochstic MA rocess is uite well uerstoo i terms of system lysis. The rocess is ivertile iff ll eros of the MA trsfer fuctio re withi the uit circle. If there re eros o or outsie the uit circle the the system is ot ivertile. Ivertiility is imortt issue i stochstic rocesses ecuse if give MA is ivertile the there eist ijective reltioshi etwee the rocess its corresoig utocorreltio fuctio. Tht is there eist uiue MA rocess tht hs give ACF. The reer itereste i owig more out MA rocesses my re Chtfiel996.

12 3. Movig verge rocesses: geerlitios Moel ot restricte to e cusl w w... w w Cusl comoet w... w Aticusl comoet F F Moel ot restricte to e lier w w w Qurtic comoet w w w w w w Volterr Kerels f w Deeig o the lictio the MA moel ees ot e cusl my use future smles of the time series. This is ossile for istce i recore sigls or the osteriori lysis of time series. A further geerlitio mes use of olier filterig. This c e erforme ths to the followig result: If is strictly sttiory with fiite momets the it c e reresete s the multi-orer covolutio of set of cusl stle o-lier time-ivrit filters. Those itereste i this result my re I.. Serg Esios for Discrete-Time olier Systems Circuits Systems Sigl Processig The stuet itereste i olier time series my re Dwyer3. Aother olier geerlitio of the movig verge is rovie y eressio similr to the lier MA ecet tht ech term w- is ffecte y olier fuctio f. This moel is clle GMA geerlie movig verge.

13 Defiitio 4. Autoregressive rocesses: AR H... A LTI with memory ivertile cusl stle w AR w... 3 w tim e tim e tim e 3 A AR stochstic rocess is oe tht is geerte usig ifferece eutio lie the oe show i the slie. This is uite geerl situtio i which it is resole to thi tht give smle of time series ees lierly o revious smles lus some rom error. 3

14 Defiitio 4. Autoregressive rocesses: AR H... A LTI with memory ivertile cusl stle w AR w... Sttisticl roerties Γ Γ δ Γ Γ A hose solutio is Yule-ler eutios Poles of H Reltioshi to MA rocesses H... Luret series 4 The imulse resose of the ssocite system is IIR its trsfer fuctio is of the i ll-ole. ote tht this time the utocorreltio is ot limite it tes to whe the lg tes to ifiity oly if the moule of ll its oles is strictly smller th. Tht mes tht if this coitio is met the the AR rocess is ergoic. Ay AR rocess c e moelle y MA rocess of ifiite orer. The reltioshi etwee this two i of rocesses is give y the ower series esio of the trsfer fuctio. The reer itereste i owig more out AR rocesses my re Chtfiel996. 4

15 4. Autoregressive rocesses: AR Determitio of the costts A Γ Γ A Γ δ r A r r > Emle: w H Poles: ± 4 i < > < > i R 4 > r A A r A A r A A r r 5 I geerl it is ifficult to solve for the Ai coefficiets i the utocorreltio fuctio sice tht woul imly riori owlege out the iut oise ower vrice of. However it is ossile to ormlie the utocorreltio fuctio to oti the correltio coefficiets. These coefficiets o ot ee o the iut ower re ormlie to. So the first thig to o is to trslte the coitios solutio form of the utocorreltio fuctio ito coitios solutio form for the correltio coefficiets. Oce this is oe it is ossile to fi two eutios to etermie the Ai coefficiets. Oe of the eutios is lwys give y the ormlitio fct r the rest of coitios re give y the - first Yule-ler eutios. Oe imortt thig to hve sesile solutio is tht the oles of the system hve moule smller th. If the two oles re ot rel the the correltio coefficiets follow me siusoil. 5

16 Autoregressive Movig verge: ARMA w A B H H H AR MA AR w Defiitio Sttisticl roerties Γ MA Γ Γ h The ARMA rocess corresos to miture of AR MA y coctetig oth systems the orer of the coctetio is ot imortt sice oth systems re LTI c e iterchge. The rocess is ergoic the utocorreltio fuctio goes to ero s the lg goes to iifity iff the moule of ll system oles is smller th. ote tht the oles of the ARMA system re efie eclusively y the AR rocess therefore the ergoic coitio is eclusively relte to the AR rt of the ARMA moel. The imortce of the ARMA moel is tht it usully escries rom rocesses with fewer rmeters th the AR MA rocesses loe. The reer itereste i owig more out ARMA rocesses my re Chtfiel996.

17 6. Autoregressive Itegrte Movig Averge: ARIMA Defiitio w δ δ *...* δ δ * ARMA l H It H H H ARMA It H ARMA H It ARIMA Poles: Q Multilicity Uit root FARIMA or ARFIMA Emle for : w 7 Oe of the coitios for moellig time series with AR MA or ARMA rocesses is tht the time series is sttiory. Rememer tht oe of the coseueces of sttiority is tht the me vrice t ech time istt is the sme for ll. If the time series hs olyomil tre the it is ossile to etre it y ifferetitio t rorite orer. The resultig ifferetite time series is ow sttiory. ARIMA moels stte tht the ifferetite time series follow ARMA moel. After itegrtig the outut will e resole moel of the time series eig stuie. Usully it is ot ecessry to ifferetite more th oce tht woul e removig lier tre. However it is lwys ifficult to ecie how my times we hve to itegrte. To elucite these uestios there is umer of tests we c erform. All these tests re clle uit root tests. Amog the most fmous re the Dicey-Fuller test the Perro test. The stuet itereste i these tests my re Kwitowsi99. If is frctio the the moel is lso clle FARIMA or ARFIMA Frctiol ARIMA. Frctiol ifferetitio is ot somethig tht is ituitive i the time omi. However frctiol ifferetitio is trivil i Fourier sce. This will e further stuie i the et sessio. The reer itereste i owig more out ARIMA rocesses my re Chtfiel996. 7

18 Sesol ARIMA: SARIMAPDQ s ARIMA w Defiitio H ARIMA s Q D P ARIMA s s s s D Q P ARIMA s H H H H ARIMA s D Q P ARIMA D Q P SARIMA s P s s Q s s s s A s w B s s D The Sesol ARIMA is lso clle the Bo-Jeis moel. This moel tes ito ccout the ossile sesol comoet which might e rom too. The sesol erio is ssume to e ow for istce yerly urterly weely etc. It will e referre to s s. The wy the time series is geerte is y tig white oise cretig sesol ARIMA moel tht woul ccout for the reltioshi mog yers if the sesolity is yerly the cretig ARIMA moel for wht hes isie ech yer. The stuet itereste i owig more out the ARIMA moel my re Chtfiel996c.

19 7. Sesol ARIMA: SARIMAPDQ s Emle: SARIMA w s ARIMA P D Q s s ARIMA P D Q s s Bw Bw s s 3 B w B w 3 B w Bw 9 A emle let s stuy the moel with s. The sesol ARIMA moel yiels s whose ifferece eutio corresos to the s moel s show i the slie. s is the iut to the withi yer ARIMA moel whose ifferece eutio is lso show i the slie. Elimitig s i oth eutios give us the fil moel. ote tht this fil moel is somethig elig whe worig with sesol t it sttes tht the curret smle is wht hee moths go lus some rom iut BwBw- lus somthig tht ccouts for the iffereces etwee the ehviour oe moth go 3 moths go. 9

20 8. Kow eterl iuts: System ietifictio A w A B u w u A U A B AR A C w A B u w c u A C U A B ARMA There re situtios i which the time series irectly ees o ow iut lus some other rom uow effect. For istce the rice of turl gs i itertiol mrets is comute s fctor of the curret rice of oil lus some etr costs ue to trsortig mufcturig etc. A ossile moel for the rice of turl gs might e lier fuctio of the oil rice ow lus lier fuctio of uow costs. This rolem is ow s system ietifictio. Deeig o the secific wy i which these fctors re comie the moel receives me or other. The stuet itereste i owig more out these moels my re Ljug987.

21 9. A fmily of moels Geerl moel w C D A B C U F A D A B F u Polyomils use A C AC ACD AB ABC ABD ABCD BFCD me of the moel AR MA ARMA ARIMA AR ARMA ARAR ARARMA Bo-Jeis Most of the moels stuie so fr re rt of ig fmily of lier moels. The scheme show i the slie reresets the most geerl situtio oe c thi of whe usig lier moels. Deeig o which olyomils we use we hve oe or other moel. The stuet itereste i owig more out these moels my re Ljug987.

22 . olier moels olier AR: Time-vryig AR:... w f w Rom coeff. AR: ε w Bilier moels w M w The lot i the slie shows the mothly verge umer of susots etwee It c e see tht there is cyclic comoet of yers tht the risig sloe is higher th the roig sloe. This i of ssymmetry cot e moelle with sigle siusoil sesol comoet if we try to o so we shoul use o lier moel. Altertively this i of symmetries c e stuie i Fourier sce hrmoic lysis ut we will o this i the et chter. There re umer of olier moels. Here we oly show few of them: -LAR: o lier AR where the fuctio tht reltes the curret smle to st smlesisolier -Time vryig rmeter moels: the comitio coefficiets chge with time -Rom coefficiet moels: the comitio coefficiet is ow u to give egree of certity -Bilier moels: where the time series c lso e elie i terms of the roucts of revious smles with the iut white rocess.

23 3 3. olier moels Smooth TAR STAR: Threshol AR TAR: > t w t w w S Heterocestic moel: w Rom wl ARCH GARCH eurl etwors Chos More olier moels: -Threshol AR: The time series c e elie y two ieeet moels eeig o whether revious smle is igger th give threshol or ot -Smooth Threshol AR: The time series c e cotiuously elie y two moels. S cts s smooth threshol. Usully S/er-/t. There re some other moels to hle for istce situtios i which the vrice is chgig with time heteroscesticity. After removig ll tres sesol comoets the resiul of the time series my hve time vryig vrice. If this is the cse the time series my e elie y ARCH moel Autoregressive Coitiol Heterocestic The stuet itereste i owig more out olier moels my re Chtfiel996.

24 . Prmeter estimtio AR w Assume tht we oserve... Mimum Lielihoo Estimtes MLE w θ Γ δ θ { } E E 3 { } { } 3 { } E θ 4 The rolem ow is to estimte the moel rmeters from set of oserve time series vlues. There re mily two roches: mimum lielihoo lest sures. Mimum lielihoo loo for the moel rmeters tht otimie the lielihoo of oservig the oserve vlues. For oig so we must ssume certi roility istriutio for w. ormlly it is ssume tht it follows Gussi istriutio with ero me costt vrice. It is lso ssume tht w is ucorrelte to itself. For comutig the mimum lielihoo solutio we ee the joit istriutio of ll the oserve vriles. e will tret sertely from the rest. c e rove to follow orml whose rmeters oly ees o the vrice of the iut rocess the comitio rmeter. Oce the rticulr relitio of is oserve we c comute the coitiol roility of oservig rovie tht we hve oserve. It is esy to show tht this coitiol roility is lso orml istriutio with the rmeters show i the slie. 4

25 5 5. Prmeter estimtio Mimum Lielihoo Estimtes MLE θ θ 3 θ... θ f θ f f f f θ θ θ θ f L... log log log... log π π θ θ rg m ˆ ˆ L L L θ θ θ umericl itertive solutio Cofiece itervls It is clerly see tht the roility of oservig 3 give is ieeet from. Therefore the joit roility of ll the oserve vriles is the rouct of oservig times oservig give times oservig 3 give etc. The logrithm of this joit roility is clle the lielihoo fuctio. Mimum lielihoo loos for the moel rmeters tht mimie the oservtio of the rovie smles. This i tur oils ow to solvig ir of olier eutios tht hve to e solve umericlly usig some itertive lgorithm. A imortt issue of the mimum lielihoo roch is tht we c comute cofiece itervls for the moel rmeters. I system ietifictio these cofiece itervls re usully trslte ito cofiece regios for the system oles eros. The reer itereste i the estimtio of the moel rmeters usig the Mimum Lielihoo roch my re Hmilto994.

26 . Prmeter estimtio w ˆ Lest Sures Estimtes LSE w ˆ ˆ w E { w } E { w } Γ Γ Γ Γ Γ Γ Γ Γ Γ Γ 6 The lest sures estimte miimies the ower of the iut rom sigl. The rtiole ehi it is tht the iut vrice will ot e miimum if I coul hve elie usig etter. Miimiig the vrice of the iut sigl with resect to the moel coefficiets yiel set of eutios clle the Yule-ler eutios lrey see i the eltio of the AR moel. Oce ll the moel coefficiets re estimte we c estimte the iut vrice. The reer itereste i the LSE metho to estimte the moel my re Prois988. ote tht the solutio of this roch ee ot e the sme s the oe rovie y MLE lthough it much simler to comute i rctice. I egieerig the solutio of this eutio system is erforme usig the Leviso-Duri lgorithm. The LSE hs the rwc tht its ccurcy relies o the ccurcy of the etermitio of the time series utocorreltio. 6

27 . Orer selectio If I hve to fit moel ARMA wht re the vlues I hve to suly? ACF/PACF lysis Aie Iformtio Criterio Byesi Iformtio Criterio Fil Preictio Error AIC log BIC log FPE log 7 The selectio of the moel orer is tricy toic. Oe is temte to use the moel tht etter fits the t. However there is oit t which icresig the moel orer will mrgilly icrese the fittig. Thus it is lso iterestig to ee the moel orer s low s ossile s log s it fits resoly well the t. To choose such moel we must use some criterio. Oe tht is uite ituitive is to lye the ACF/PACF structure this will e further elie i the et slies. Aother ossiility is to use y of the rsimoy criteri ville. I the slie we show three of them AIC BIC FPE. They re esige to ecrese s log s we re sigifictly fittig the t etter to icrese whe the imrovemets re mrgil. The fit of the t is rovie y the iut vrice which c lso e see s the vrice of the misfit error. is the umer of ville smles i the time series re the ARMA moel orers. 7

28 . Orer selectio Prtil correltio coefficiets PACF First-orer correltios φ φ... w φ r φ r r r r... r r r r r r r... 3 r r r r r r r r r r φ r r φ r r 3 φ 3 r r φ r r φ r Yule-ler eutios 8 The rtil correltio coefficiet is efie s the lst coefficiet of rtil utoregressio moel of orer. It is imortt tht is ero me. The rtil correltio etwee is the correltio etwee these two vriles fter removig ll the lier reltioshis etwee the smles etwee. 8

29 . Orer selectio Thum rule ARMA: ACF: eoetil ecrese; PACF: oe e ARMA: ACF: eoetil ecrese or wves; PACF: two es ARMA: ACF: oe e; PACF: eoetil ecrese ARMA: ACF: two es; PACF: eoetil ecrese or wves ARMA: ACF&PACF: eoetil ecrese 9 For owig more out the thum rules to select the moel orer usig ACF PACF lese follow htt://ooe.free.fr/ui/48_r/5.html 9

30 3. Moel checig Resiul Alysis Emle: ARMA wˆ w w wˆ wˆ Assumtios. Gussiity:. The iut rom sigl w is uivrite orml with ero me. The outut sigl the time series eig stuie is multivrite orml its covrice structure is fully etermie y the moel structure rmeters. Sttiority: is sttiory oce tht the ecessry oertios to rouce sttiory sigl hve ee crrie out. 3. Resiul ieeecy: the iut rom sigl w is ieeet of ll revious smles. 3 Moel checig is s imortt s moel formultio estimtio. After fittig our moel we hve to e sure tht the moel is vli tht oe of the moel hyothesis re violte. This is usully oe through the resiul lysis. This lysis ims t etermiig wht is the oe-smle-he reictio error which is lso iterrete s the system iut w. The mi ssumtios oe i most moels re: gussiity sttiority resiul ieeecy. Be creful tht ARIMA moels c el with osttiory sigls ecuse it elicitly cosiers multile ifferetitio times s tool to rouce sttiory time series. 3

31 3. Moel checig Digostic checig. Comute lot the resiul error. Chec tht its me is roimtely ero 3. Chec for the romess of the resiul i.e. there re o time itervls where the me is sigifictly ifferet from ero itervls where the resiul is systemticlly ositive or egtive. 4. Chec tht the resiul utocorreltio is ot sigifictly ifferet from ero for ll lgs 5. Chec tht the resiul is ormlly istriute. 6. Chec if there re resiul outliers. 7. Chec the ility of the moel to reict future smles 3 Digostic checig is the geerl me use for ll those ifferet checigs tht might oit out tht the moel we hve fitte is ot eute. If this is the cse you woul hve to revise the ssumtios of your moel why they re ot met evetully chge the moel. If sttiority is ot met it c e esily hle y removig tre or sesol comoet or y ifferecig eough times. My times resiul gussiity is ot met however most of the times this is ot severe rolem s log s its istriutio hs ero o momets the moel still rovie useful isight i the t structure. A chec tht is rticulrly useful to test whether the moel is overfittig or ot is to ee the lst M smles for testig reictio ccurcy. Thus if there re smles i totl i time series we use -M for triig the lst M for testig. The resiul lysis o these M smles shoul lso ss ll the igostic checigs. 3

32 Sessio outlie. Gol. A short itrouctio to system lysis 3. Movig Averge rocesses MA 4. Autoregressive rocesses AR 5. Autoregressive Movig Averge ARMA 6. Autoregressive Itegrte Movig Averge ARIMA FARIMA 7. Sesol Autoregressive Itegrte Movig Averge SARIMA 8. Kow eterl iuts: System ietifictio 9. A fmily of moels. olier moels. Prmeter estimtio. Orer selectio 3. Moel checig 3 3

33 Biliogrhy C. Chtfiel. The lysis of time series: itrouctio. Chm & Hll CRC 996. D.S.G. Polloc. A hoo of time-series lysis sigl rocessig ymics. Acemics Press 999. J. D. Hmilto. Time series lysis. Priceto Uiv. Press