ARIMA models. X t = X t 1 + " t :

Transcription

1 ARIMA models. De nitions.. AR models. An AR(p) (AutoRegressive of order p) model is a discrete time linear equations with noise, of the form X t = X t + ::: + p X t p + " t : Here p is the order, ; :::; p are the parameters or coe cients (real numbers), " t is an error term, usually a white noise with intensity. The model is considered either on integers t Z, thus without initial conditions, or on the non-negative integers t N. In this case, the relation above starts from t = p and some initial condition X 0 ; :::; X p must be speci ed. Example. We have seen above the simplest case of an AR() model X t = X t + " t : With < and V ar [X t ] =, it is a wide sense stationary process (in fact strict sense since it is gaussian, see also below). The autocorrelation coe cient decays exponentially: (n) = n : Even if the general formula is not so simple, one can prove a similar result for any AR model. In order to model more general situations, it may be convenient to introduce models with non-zero average, namely of the form (X t ) = (X t ) + ::: + p (X t p ) + " t : When = 0, if we take an initial condition having zero average (this is needed if we want stationarity), then E [X t ] = 0 for all t. We may escape this restriction by taking 6= 0. The new process Z t = X t has zero average and satis es the usual equation But X t satis es Z t = Z t + ::: + p Z t p + " t : X t = X t + ::: + p X t p + " t + ( ::: p ) = X t + ::: + p X t p + " t + e: i

2 ii ARIMA MODELS.. Time lag operator. Let S be the space of all sequences (x t ) tz of real numbers. Let us de ne an operator L : S S, a map which transform sequences in sequences. It is de ned as Lx t = x t ; for all t Z: We should write (Lx) t = x t, with the meaning that, given a sequence x = (x t ) tz S, we introduce a new sequence Lx S, that at time t is equal to the original sequence at time t, hence the notation (Lx) t = x t. For shortness, we drop the bracket and write Lx t = x t, but it is clear that L operates on the full sequence x, not on the single value x t. The map L is called time lag operator, or backward shift, because the result of L is a shift, a translation, of the sequence, backwards (in the sense that we observe the same sequence but from one position shifted on the left). If we work on the space S + of sequences (x t ) tn de ned only for non-negative times, we cannot de ne this operator since, given (x t ) tn, its rst value is x 0, while the rst value of Lx should be x which does not exist. Nevertheless, if we forget the rst value, the operation of backward shift is meaningful also here. Hence the notation Lx t = x t is used also for sequences (x t ) tn, with the understanding that one cannot take t = 0 in it. Remark. The time lag operator is a linear operator. The powers, positive and negative, of the lag operator are denoted by L k : L k x t = x t k ; for t Z (or, for t max (k; 0), for sequences (x t ) tn ). With this notation, the AR model reads k L k X t = " t :.3. MA models. A MA(q) (Moving Average with orders p and q) model is an explicit formula for X t in terms of noise of the form X t = " t + " t + ::: + q " t q : The process is given by a (weighted) average of the noise, but not an average from time zero to the present time t; instead, an average moving with t is taken, using only the last q + times. Using time lags we can write X t = + k L k " t :.4. ARMA models. An ARMA(p; q) (AutoRegressive Moving Average with orders p and q) model is a discrete time linear equations with noise, of the form k L k X t = + k L k " t

3 . DEFINITIONS iii or explicitly X t = X t + ::: + p X t p + " t + " t + ::: + q " t q : We may incorporate a non-zero average in this model. If we want that X t has average, the natural procedure is to have a zero-average solution Z t of and take X t = Z t +, hence solution of Z t = Z t + ::: + p Z t p + " t + " t + ::: + q " t q X t = X t + ::: + p X t p + " t + " t + ::: + q " t q + e with e = ::: p :.5. Di erence operator. Integration. The rst di erence operator,, is de ned as X t = X t X t = ( L) X t : If we call Y t = ( then we may reconstruct X t from Y t by integration: L) X t X t = Y t + X t = Y t + Y t + X t = ::: = Y t + ::: + Y + X 0 having the initial condition X 0. The second di erence operator,, is de ned as X t = ( L) X t : Assume we have Then Y t = ( L) X t : Y t = ( Z t = ( L) Z t L) X t so we may rst reconstruct Z t from Y t : Z t = Y t + ::: + Y + Z where Z = ( L) X = X X 0 (thus we need X and X 0 ); then we reconstruct X t from Z t : X t = Z t + ::: + Z + X 0 : All this can be generalized to d, for any positive integer d.

4 iv ARIMA MODELS.6. ARIMA models. An ARIMA(p; d; q) (AutoRegressive Integrated Moving Average with orders p, d, q) model is a discrete time linear equations with noise, of the form k L k ( L) d X t = + k L k " t : It is a particular case of ARMA models, but with a special structure. Set Y t := ( is an ARMA(p; q) model k L k Y t = + k L k " t L) d X t. Then Y t and X t is obtained from Y t by d successive integrations. The number d is thus the order of integration. with Example. The random walk is ARIMA(0; ; 0). We may incorporate a non-zero average in the auxiliary process Y t and consider the equation k L k ( L) d X t = + k L k " t + e e = ::: p :. Stationarity, ARMA and ARIMA processes Under suitable conditions on the parameters, there are stationary solutions to ARMA models, called ARMA processes. In the simplest case of AR() models, we have proved the stationarity (with suitable variance of the initial condition) when the parameter satis es <. In general, there are conditions but they are quite technical and we address the interested reader to the specialized literature. In the sequel we shall always use sentences of the form: consider a stationary solution of the following ARMA model, meaning implicitly that it exists, namely that we are in the framework of such conditions. Our statements will therefore hold only in such case, otherwise are ust empty statements. Integration brakes stationarity. Solutions to ARIMA models are always non-stationary if we take Y t stationary (in this case the corresponding X t is called ARIMA process). For instance, the random walk is not stationary. The kind of growth of such processes is not always trivial. But if we include a non-zero average, namely we consider the case k L k ( L) d X t = + k L k " t + e then we have the following: if d =, X t has a linear trend; if d =, a quadratic trend, and so on. Indeed, at a very intuitive level, if Y t is a stationary solution of the associated ARMA model, with mean, then its integration produces a trend: a single step integration gives us X t = Y t + ::: + Y + X 0

5 3. CORRELATION FUNCTION v so the stationary values of Y accumulate linearly; a two step integration produces a quadratic accumulation, and so on. When = 0, the sum Y t +:::+Y has a lot of cancellations, so the trend is sublinear (roughly it behaves as a square root). But the cancellations become statistically not signi cant when 6= 0. If > 0 and d = we observe an average linear growth; if If < 0 and d = we observe an average linear decay. This is also related to the ergodic theorem: since Y t is stationary and its autocorrelation decays at in nity, we may apply the ergodic theorem and have that Y t + ::: + Y E [Y ] = t (in mean square). Hence Y t + ::: + Y t: There are uctuations, roughly of the order of a square root, around this linear trend. 3. Correlation function 3.. First results. Assume we have a stationary, mean zero, ARMA(p; q) process. Set n := R (n) = E [X n X 0 ]. Proposition. For every > q, Proof. Recall that R ( = k k : n) = R (n). Recall also that k L k X t = + Notice that for every n and m we have Then k L k " t E [X t n L m X t ] = E [X t n X t m ] = m n : k k = E [X t X t ] = E = E " " k E [X t k X t ] # X t X t k X t k X t + # k L k " t : In the case > q we have L k " t independent of X t, for k q, hence k k = 0: The proof is complete.

6 vi ARIMA MODELS We get This formula allows us to compute the full autocorrelation function when q = 0 (AR processes). Example 3. Consider the simple case for every > 0, namely where 0 = E X 0. Hence On its own, hence which gives us 0 = We get X t = ax t + " t : = 0 = 0 = ::: = 0 : V ar [X t ] = a V ar [X t ] + V ar [" t ] 0 = a 0 +. This is the same result found in Chapter. a Example 4. Consider the next case, for every > 0, namely The rst equation, in view of =, gives us Hence again we ust need to nd 0. We have hence X t = a X t + a X t + " t : = + = 0 + = + 0 ::: = 0 : V ar [X t ] = a V ar [X t ] + a V ar [X t ] + + Cov (X t ; X t ) 0 = a 0 + a : This is a second relation between 0 and ; together, they will gives us both quantities.

7 3. CORRELATION FUNCTION vii 3.. ARMA as in nite MA. Assume always that we have a stationary, mean zero, ARMA process. Assume it is de ned for all integers (in particular we use the noise for negative integers). From k L k X t = + k L k " t we get, when proper convergence conditions are satis ed, X t = ' L " t where P ' x is the Taylor expansion of the function g (x) = + P q kx k P p kx k : Indeed, assume this function g has the Taylor development g (x) = P ' x in a neighborhood U of the origin. Then, for each x U, X k x k ' x = + k x k : Assume ' < : One can prove that the series X t := P ' L " t converges in mean square; and also P ' L k " t for every k, and thus we have X k L k ' L " t = + k L k " t : Therefore X t := P ' L " t solves the equation which de nes the ARMA process. This is always a solution; if we started with a given solution X t, only under a uniqueness condition in law (that we omit to discuss), we are sure that such solution can be represented in the form X t = P ' L " t. Example 5. Consider the simple case We have hence X t = ax t + " t : g (x) = g (x) = x (x)

8 viii ARIMA MODELS (recall the geometric series). The series converges for x <. We need < to have P ' <. Therefore the ARMA process is X t = L " t : 3.3. Correlation function, part. Assume always that we have a stationary, mean zero, ARMA process and set n := R (n) = E [X n X 0 ]. Under proper conditions, X t = ' i L i " t : i=0 Hence we may compute E X t + P q kl k " t also for q, the case which was left aside above, in the proof of Proposition. Proposition. Under the previous assumptions, for all = 0; :::; q we have Xq k k = ' i i+ : Thus, for every 0 we may write k k = i=0 ' i i+ i+f0;:::;qg : i=0 P Proof. We have (we set 0 = ), from the proof of Proposition and the identity X t = i=0 ' il i " t, " # k k = E X t k L k " t The proof is complete. = = = i=0 k=0 i=0 k=0 k=0 i ' i k E hl i " t L k " t ' i k i+;k ' i i+ i+f0;:::;qg : i=0 This is a general formula. A more direct approach to compute E X t + P q kl k " t even if q consists in the substitution of the equation for X t : " # " # E X t + k L k " t = E k L k X t + + k L k " t + k L k " t :

9 4. POWER SPECTRAL DENSITY ix The products involving L k " t and L k0 " t are easily computed. Then we have products of the form i E hl k X t L k0 " t the worse of which is E L X t L q " t : If q, it is zero, otherwise not, but we may repeat the trick and go backward step by step. In simple examples we may compute all by this strategy. We get Example 6. Consider the simple case for every >, namely X t = X t + " t + " t : = 0 = 3 = ::: but from this relations we miss and 0. About, we have 0 = E [X t ( + L) " t ] = E [X t " t ] = E [(X t + " t + " t ) " t ] = : Therefore is given in terms of 0 (and then iteratively also ; 3 and so on). On its own, hence Moreover, hence This gives us 0. V ar [X t ] = a V ar [X t ] Cov (X t ; " t ) 0 = a Cov (X t ; " t ) : Cov (X t ; " t ) = Cov (X t + " t + " t ; " t ) = 0 = a : 4. Power spectral density We work under the assumptions of the previous sections, in particular that X is a stationary ARMA process and g (x) = +P q kx k P p has the Taylor development g (x) = P kx k ' x in a complex neighborhood U of the origin which includes the closed ball of radius. Moreover, we assume the assumptions of Wiener-Khinchin theorem of Chapter 3.

10 x ARIMA MODELS Theorem. S () = + P q ke ik P p ke ik : Proof. We have (Z T denotes the set of all n Z such that n T=) bx T () = p X e in X n = X p nz T nz T ' e in " n = X h E bxt () X b i T () = E nz T 0 =0 ' ' 0e in e i(n bx T () = p X 4 X n 0 Z T 0 =0 X nz T n 0 Z T 0 =0 +0) = Z T ' 0e in0 " n =0 3 ' ' 0e in e in0 E " n " n ' e i ' 0e i0 = Z T The cardinality Z T of Z T has the property lim T Z T =T =, hence we get S () = ' n e in : n=0 ' n e in : Now, it is su cient to use the the relation +P q kx k P p = P kx k ' x for x = e i. The proof is complete. Remark. Consider the case q = 0 and write the formula with = f n=0 S (f) = P p ke ikf : Assume there is only k = p: S (f) = p e ipf : The maxima are at pf Z, namely for f = p. The function S (f) immediately shows the periodicity in the recursion X t = a p X t p + " t :

11 4. POWER SPECTRAL DENSITY xi 4.. Example. X t = 0:8 X t + " t S (f) = 0:8 e if y x