Lecture 13 Time Series: Stationarity, AR(p) & MA(q)

Size: px
Start display at page:

Download "Lecture 13 Time Series: Stationarity, AR(p) & MA(q)"

Transcription

1 RS C - ecure 3 ecure 3 Tme Seres: Saoar AR & MAq Tme Seres: Iroduco I he earl 97 s was dscovered ha smle me seres models erformed beer ha he comlcaed mulvarae he oular 96s macro models FRB-MIT-Pe. See Nelso 97. The ools? Smle uvarae ARIMA models oulared b he eboo of Bo & Jes 97. Q: Wha s a me seres? A me seres s a rocess observed sequece over me... T > Y { 3... T } Because of he sequeal aure of Y we eec ha ad - o be deede. The classcal assumos are o vald.

2 RS C - ecure 3 Tme Seres: Iroduco Usuall me seres models are searaed o wo caegores: - uvarae Є R s scalar > rmar model: Auoregressos ARs. - mulvarae Є R m vecor-valued. > rmar model: Veco auoregressos VARs. I me seres { T } are ol RV. We wa o model he codoal eecao: [ F - where F - { } s he as hsor of he seres. Tme Seres: Iroduco Two oular models for [ F - : - A auoregressve AR rocess models [ F - wh lagged deede varables. A movg average MA rocess models [ F - wh lagged errros. Usuall [ F - has bee modeled as a lear rocess. Bu rece mes o-leares have become more commo. I geeral we assume he error erm s ucorrelaed wh mea ad cosa varace σ. We call a rocess le hs a whe ose WN rocess. We deoe as ~ WNσ

3 RS C - ecure 3 CM Revsed: Tme Seres Wh auocorrelaed daa we ge deede observaos. Recall ρ - u The deedece assumo A s volaed. The N ad he CT cao be easl aled hs coe. We eed ew ools ad defos. We wll roduce he coces of saoar ad ergodc. The ergodc heorem wll gve us a couerar o he N. To ge asmoc dsrbuos we also eed a CT for deede varables usg he coce of mg ad saoar. Or we ca rel o he margale CT. Tme Seres - Saoar Cosder he o robabl dsrbuo of he colleco of RVs: F... P The we sa ha a rocess s... s order saoar f F F for a d order saoar f Nh-order saoar f F F for a F... F... for a Defo. A rocess s srogl srcl saoar f s a Nh-order saoar rocess for a N.

4 RS C - ecure 3 [ Cov d f Var d f σ σ ρ σ Tme Seres Momes The momes descrbe a dsrbuo. We calculae he momes as usual. Noe: - s called he auocovarace fuco. s he varace. Saoar requres all hese momes o be deede of me. If he momes are me deede we sa he seres s o-saoar. For srcl saoar rocess: ad σ σ because F F rovded ha F F ρ ρ ρ ρ ρ ρ he ad le cov cov The correlao bewee a wo RVs deeds o he me dfferece. < < The Tme Seres Momes

5 RS C - ecure 3 Tme Seres Wea Saoar A rocess s sad o be N-order weal saoar f all s o momes u o order N es ad are me vara. A Covarace saoar rocess or d order weal saoar has: - cosa mea - cosa varace - covarace fuco deeds o me dfferece bewee R.V. Tha s s covarace saoar f: Var Cov cosa cosa [ f Tme Seres Wea Saoar amles: For all assume ~ WNσ Φ - [ assumg Φ Var[ σ /-Φ assumg Φ < [ - Φ [ - > saoar o me deede - > Σ o - - [ Var[ Σ o - σ σ > o-saoar me deede

6 RS C - ecure 3 Saoar Seres amles: ~ WN % chage % Chages USD/GBP 978:I-:IV Tme No-Saoar Seres amles: ~ WN RW wh drf US CPI US CPI Prces 978:I-:IV

7 RS C - ecure 3 Tme Seres rgodc We wa o allow as much deedece as he N allows us o do. Bu saoar s o eough as he followg eamle shows: amle: e {U } be a sequece of..d. RVs uforml dsrbued o [ ad le be N deede of {U }. Defe Y U. The Y s saoar wh? bu Y Y o Y Y The roblem s ha here s oo much deedece he sequece {Y } because of. I fac he correlao bewee Y ad Y s alwas osve for a value of. Tme Seres rgodc of he Mea We wa o esmae he mea of he rocess { }. Bu we eed o dsgushg bewee esemble average ad me average: - semble Average - Tme Seres Average m m Q: Whch esmaor s he mos arorae? A: semble Average. Bu s mossble o calculae. We ol observe oe. Q: Uder whch crcumsaces we ca use he me average ol oe realao of { }? Is he me average a ubased ad cosse esmaor of he mea? The rgodc Theorem gves us he aswer.

8 RS C - ecure 3 Tme Seres rgodc of he Mea Recall he suffce codos for cossec of a esmaor: he esmaor s asmocall ubased ad s varace asmocall collases o ero.. Q: Is he me average s asmocall ubased? Yes.. Q: Is he varace gog o ero as T grows? I deeds. var s [ ρ ρ ρ cov ρ ρ s s ρ ρ ρ ρ s ρ ρ ρ ρ ρ Tme Seres rgodc of he Mea var lm var lm ρ ρ? ρ If he were ucorrelaed he varace of he me average would be O -. Sce deede radom varables are ecessarl ucorrelaed bu o vce versa we have us recovered a form of he N for deede daa. Q: How ca we mae he remag ar he sum over he uer ragle of he covarace mar go o ero as well? A: We eed o mose codos o ρ. Codos weaer ha "he are all ero;" bu srog eough o eclude he sequece of decal coes.

9 RS C - ecure 3 Tme Seres rgodc of he Mea We use wo equales o u uer bouds o he varace of he me average: ρ Defo: A covarace-saoar rocess s ergodc for he mea f lm Covaraces ca be egave so we uer-boud he sum of he acual covaraces b he sum of her magudes. The we eed he er sum so covers all lags. Ths mgh of course be fe sequece-ofdecal-coes. ρ rgodc Theorem: The a suffce codo for ergodc for he mea s ρ as ρ Tme Seres rgodc of d Momes A suffce codo o esure ergodc for secod momes s: ρ < A rocess whch s ergodc he frs ad secod momes s usuall referred as ergodc he wde sese. rgodc uder Gaussa Dsrbuo If { }s a saoar Gaussa rocess ρ s suffce o esure ergodc for all momes. < Noe: Recall ha ol he frs wo momes are eeded o descrbe he ormal dsrbuo.

10 RS C - ecure 3 Tme Seres rgodc Theorems We sae wo esseal heorems o he aalss of saoar me seres. Dffcul o rove geeral. Theorem I If s srcl saoar ad ergodc ad f s a RV he s srcl saoar ad ergodc. Theorem II rgodc Theorem If s srcl saoar ad ergodc ad [ < ; he as T ; T [ These resuls allow us o cossel esmae arameers usg me-seres momes. Tme Seres - MDS Defo: s a margale dfferece sequece MDS f [ F -. Regresso errors are aurall a MDS. Some me-seres rocesses ma be a MDS as a cosequece of omg behavour. For eamle mos asse rcg models ml ha asse reurs should be he sum of a cosa lus a MDS. Useful roer: s ucorrelaed wh a fuco of he lagged formao F -. The for > > [ -.

11 RS C - ecure 3 Tme Seres MDS CT Theorem MDS CT If u s a srcl saoar ad ergodc MDS ad u u Ω< ; he as T ; d u N Ω T Alcao: e { } a vecor of lagged s. The s a MDS. We ca al he MDS CT Theorem. The d [ ' N Ω Ω T e he dervao of asmoc dsrbuo of OS he above resul s he e o esablsh he asmoc dsrbuo a me seres coe. Auoregressve AR Process We wa o model he codoal eecao of : [ F - where F - { } s he as hsor of he seres. We assume he error erm - - [ F - follows a WNσ. A AR rocess models [ F - wh lagged deede varables. The mos commo models are AR models. A AR model volves a sgle lag whle a AR model volves lags. amle: A lear AR model he mos oular racce: wh [ F -....

12 RS C - ecure 3 AR Process ag Oeraor Defe he oeraor as : I s usuall called ag oeraor. Bu ca roduces lagged or forward varables for egave values of. For eamle: 3 Also oe ha f c s a cosa > c c. Somemes he oao for whe worg as a lag oeraor s B bacshf oeraor ad whe worg as a forward oeraor s F. Imora alcao: Dfferecg d 3 d Auoregressve AR Process e s wor wh he lear AR model s: We ca wre hs rocess as: where... Φ s called he auoregressve olomal of. Noe ha delvers a fe sum o he - s Q: Ca we do hs verso?... : ag oeraor > a MA rocess!

13 RS C - ecure 3 AR Process - Saoar e s comue momes of usg he fe sum assume: [ Var[ [ [ Var[ [ where abusg oao > Usg he fudameal heorem of algebra Φ ca be facored as where he r... r ЄC are he roos of Φ. If he Φ s coeffces are all real he roos are eher real or come comle cougae ars. r r... r /... AR Process - Saoar Theorem: The lear AR rocess s srcl saoar ad ergodc f ad ol f r > for all where r s he modulus of he comle umber r. We usuall sa all roos le ousde he u crcle. Noe: If oe of he r s equals Φ & has a u roo.e. Φ. Ths s a secal case of o-saoar. Recall Φ - roduces a fe sum o he - s. If hs sum does o elode we sa he rocess s sable. If he rocess s sable we ca calculae δ /δ - : how much s affeced oda b a ovao a shoc - erods ago. We call hs he mulse resose fuco IRF.

14 RS C - ecure 3 AR Process amle: AR amle: AR rocess [ Var [ [ Var [ σ * ; sce σ r > < r > Noe: / These fe sums wll o elode sable rocess f Φ < > saoar codo. Uder hs codo we ca calculae he mulse resose fuco: δ /δ - Φ AR Process amle: AR The auocovarace fuco s: Cov Y Y [ Y Y [{ Y } Y [ Y Y [ Y [ Y There s a recursve formula for : Aga whe < he auocovarace do o elode as creases. There s a eoeal deca owards ero.

15 RS C - ecure 3 AR Process amle: AR Noe: - whe < < All auocovaraces are osve. - whe < < The sg of he auocovaraces shows a alerag aer begg a egave value. The AR rocess has he Marov roer: The dsrbuo of Y gve {Y - Y - } s he same as he dsrbuo of Y gve {Y - }. AR Process amle: AR amle: AR rocess We ca ver -Φ -Φ o ge he MA rocess. Saoar Chec - [ /-Φ -Φ * > Φ Φ. - Var[ σ / -Φ -Φ > Φ Φ < Saoar codo: Φ Φ < The aalss ca be smlfed: Rewre he AR mar form as a AR. > ~ Noe: Now we chec [I-A for saoar codos ~ A ~ ~

16 RS C - ecure 3 Noe: Recall Checg ha [I-A s o sgular same as checg ha A does o elode. The sabl of he ssem ca be deermed b he egevalues of A. Tha s ge he λ s ad chec f λ < for all. If λ < for all s sable does o elode ad saoar. The: AR Process - Saoar de λ λ λ λ λ > I A A... F F I F F I A I A ~ [ ~ ~ ~ ~ ~ < < λ λ λ λ λ λ λ λ The auocovarace fuco s gve b: Aga a recursve formula. e s ge he frs auocovaraces: AR Process - Saoar [ [ [ Y Y Y Y Y Y [ [ [ σ Y Y Y

17 RS C - ecure 3 AR Process - Saoar The AR mar AR form s called Vecor AR or VAR. Nce roer: The VAR s Marov -.e. forecass deed ol o oda s daa. I s sraghforward o al he VAR formulao o a AR rocesses. We ca also use he same egevalue codos o chec he saoar of AR rocesses. AR Process - Causal The AR model: where... The > a MA rocess! Bu we eed o mae sure ha we ca ver he olomal Φ. Whe Φ we sa he rocess s causal srcl seag a causal fuco of { }. Defo: A lear rocess { } s causal f here s a ψ ψ ψ wh wh ψ ψ <....

18 RS C - ecure 3 amle: AR rocess: The s causal f ad ol f: Φ < or he roo r of he olomal Φ Φ sasfes r >. Q: How do we calculae he ψ s coeffcees for a AR? Machg coeffces: AR Process Causal where } Ψ < B Y amle: AR - Calculag he ψ s b machg coeffces. AR Process Calculag he ψ s We ca solve hese lear dfferece equaos several was: - Numercall - Guess he form of a soluo ad usg a ducve roof or - Usg he heor of lear dfferece equaos. 3 3 Ψ Ψ Ψ Ψ Ψ Ψ Ψ Ψ Π Π

19 RS C - ecure 3 Defe The he model ca be wre as The OS esmaor s Recall ha u s a MDS. I s also srcl saoar ad ergodc. The vecor s srcl saoar ad ergodc ad b Theorem I so s. The b he rgodc Theorem AR Process smao ad Proeres β 'β X X X b ' ' ˆ β. [ u u T T Q T ' [ ' Cossec Pug ogeher he revous resuls he OS esmaor ca be rewre as: The > he OS esmaor s cosse. AR Process smao ad Proeres β β T T X X X b ' ' ' ' ˆ β β β ' ' Q T T b

20 RS C - ecure 3 AR Process Asmoc Dsrbuo Asmoc Normal We al he MDS CT o. The s sraghforward o derve he asmoc dsrbuo of he esmaor smlar o he OS case: Theorem If he AR rocess s srcl saoar ad ergodc ad [ 4 he as T ; ˆ d T β β N Q ΩQ Ω [ ' Idecal form o he asmoc dsrbuo of OS crossseco regresso > asmoc ferece s he same. The asmoc covarace mar s esmaed us as he crossseco case: The sadwch esmaor. AR Process Boosra So far we cosruced he boosra samle b radoml resamlg from he daa values. Ths creaed a..d boosra samle. Ths s arorae for me-seres. We have deedece. There are wo oular mehods o boosra me seres. Model-Based Paramerc Boosra Bloc Resamlg Boosra

21 RS C - ecure 3 AR Process Boosra Model-Based Paramerc Boosra. smae b ad resduals e:. F a al codo { } 3. Smulae..d. draws e* from he emrcal dsrbuo of he resduals {e e e 3... e T }. 4. Creae he boosra seres b he recursve formula ˆ ˆ * ˆ *... ˆ * * Pros: Smle. Smlar o he usual boosra. * Cos: Ths cosruco moses homosedasc o he errors e* ; whch ma be dffere ha he roeres of he acual e. I also moses he AR as he DGP. AR Process Boosra Bloc Resamlg. Dvde he samle o T/m blocs of legh m.. Resamle comlee blocs. For each smulaed samle draw T/m blocs. 3. Pase he blocs ogeher o creae he boosra me-seres *. Pros: I allows for arbrar saoar seral correlao heerosedasc ad for model mssecfcao. Cos: I ma be sesve o he bloc legh ad he wa ha he daa are aroed o blocs. Ma o wor well small samles.

22 RS C - ecure 3 A MA rocess models [ F - wh lagged error erms. A MAq model volves q lags. We ee he whe ose assumo for. amle: A lear MAq model: Q: Is saoar? Chec he momes. WOG assume. Movg Average Process q q q q q q Q: Is saoar? Chec he momes. WOG assume. I s eas o verf ha he sums are fe > MAq s saoar. Noe ha a MAq rocess ca geerae a AR rocess. We have a fe sum olomal o. Tha s a AR. Movg Average Process - Saoar. oherwse ; [... [ [... var q q q q q σ σ > * π *

23 RS C - ecure 3 MA Process - Iverbl We eed o mae sure ha - s defed. Tha s we requre. Whe hs codo s me we ca wre as a causal fuco of. We sa he MA s verble. For hs o hold we requre: π < Defo: A lear rocess { } s verble srcl seag a verble fuco of { } f here s a π π π wh wh π π <.... MA Process amle: MA amle: MA rocess: - Momes Y Var [ [ σ σ σ > Noe: The auocovarace fuco s ero afer lag. - Iverbl: If < we ca wre - * > * * π

24 RS C - ecure 3 amle: MA rocess: - Momes Noe: he auocovarace fuco s ero afer lag. MA Process amle: MA > Y σ σ σ - Iverbl: The roos of all le sde he u crcle. I ca be show he verbl codo for MA rocess s: MA Process amle: MA λ λ < < < <

25 RS C - ecure 3 MA Process - smao MA are more comlcaed o esmae. I arcular here are oleares. Cosder a MA: - The auo-correlao s ρ /. The MM esmae of sasfes: ˆ ˆ ± 4r r ˆ r A olear soluo ad dffcul o solve. Aleravel f < we ca r aє-; a a a ad loo umercall for he leas-square esmaor ˆ arg a m{ S T a T... a} The Wold Decomoso Theorem - Wold 938. A covarace saoar { } has fe order movg-average rereseao: ψ κ ψ where κ : deermsc ermerfeclforecasab le.sa κ ψ ~ WN σ < s a lear combao of ovaos over me. A saoar rocess ca be rereseed as a MA lus a deermsc red.

26 RS C - ecure 3 amle: e -κ. The chec momes: X s a covarace saoar rocess. The Wold Decomoso ψ ψ σ ψ ψ ψ ψ ψ σ ψ ψ ψ ψ < ψ σ ψ ψ κ [ [. [ [. [ [ [ A combao of AR ad MAq rocesses roduces a ARMAq rocess: Usuall we ss ha Φ ad ha he olomals Φ have o commo facors. Ths mles s o a lower order ARMA model. ARMA Process q q q >

27 RS C - ecure 3 ARMA Process amle: Commo facors. Suose we have he followg ARMA3 model wh Ths model smlfes o: > a MA rocess. Pure AR Rereseao: Pure MA Rereseao: Secal ARMAq cases: Π B a Π B - : MAq - q : AR. Ψ B a Ψ B q B q B B B ARMA: Saoar Causal ad Iverbl Theorem: If Φ ad have o commo facors a uque saoar soluo o f ad ol f.... Ths ARMAq model s causal f ad ol f.... Ths ARMAq model s verble f ad ol f.... Noe: Real daa cao be eacl modeled usg a fe umber of arameers. We choose q o creae a good aromaed model.

28 RS C - ecure 3 ARMA Process SD Rereseao Cosder he ARMAq model: e ad w. The w... > s a -h-order lear sochasc dfferece equao SD. amle: s-order SD AR: Recursve soluo Wold form: ψ where - s a al codo. ARMA Process Damc Muller The damc muller measurers he effec of o subseque values of :. Tha s he frs dervave o he Wold rereseao: δ /δ δ /δ ψ. For a AR rocess: δ /δ δ /δ Φ. Tha s he damc muller for a lear SD deeds ol o he legh of me o o me.

29 RS C - ecure 3 The mulse-resose fuco IRF a sequece of damc mullers as a fuco of me from he oe me chage he ovao. Usuall IRF are rereseed wh a grah ha measures he effec of he ovao o over me: δ /δ δ /δ δ /δ...ψ ψ ψ... Oce we esmae he ARMA coeffces s eas o draw a IRF. ARMA Process Imulse Resose Fuco Q: We add wo ARMA rocess wha order do we ge? Addg MA rocesses - Uder deedece: - The for > Maq q > s ARMAmaq q - Imlcao: MAMAMA ARMA Process Addo u C A u C A [ [ [

30 RS C - ecure 3 ARMA Process Addo Q: We add wo ARMA rocess wha order do we ge? Addg AR rocesses A C - Rewre ssem as: C A A C A C u? - The s ARMA ma C A u C A u u [ C A u