# Time Series Analysis

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Time Series Analysis Lecture Notes for Ross Ihaka Statistics Department University of Auckland April 14, 2005

2 ii

3 Contents 1 Introduction Time Series Stationarity and Non-Stationarity Some Examples Annual Auckland Rainfall Nile River Flow Yield on British Government Securities Ground Displacement in an Earthquake United States Housing Starts Iowa City Bus Ridership Vector Space Theory Vectors In Two Dimensions Scalar Multiplication and Addition Norms and Inner Products General Vector Spaces Vector Spaces and Inner Products Some Examples Hilbert Spaces Subspaces Projections Hilbert Spaces and Prediction Linear Prediction General Prediction Time Series Theory Time Series Hilbert Spaces and Stationary Time Series The Lag and Differencing Operators Linear Processes Autoregressive Series The AR(1) Series The AR(2) Series Computations Moving Average Series The MA(1) Series Invertibility Computation iii

4 iv Contents 3.7 Autoregressive Moving Average Series The ARMA(1,1) Series The ARMA(p,q) Model Computation Common Factors The Partial Autocorrelation Function Computing the PACF Computation Identifying Time Series Models ACF Estimation PACF Estimation System Identification Model Generalisation Non-Zero Means Deterministic Trends Models With Non-stationary AR Components The Effect of Differencing ARIMA Models Fitting and Forecasting Model Fitting Computations Assessing Quality of Fit Residual Correlations Forecasting Computation Seasonal Models Purely Seasonal Models Models with Short-Term and Seasonal Components A More Complex Example Frequency Domain Analysis Some Background Complex Exponentials, Sines and Cosines Properties of Cosinusoids Frequency and Angular Frequency Invariance and Complex Exponentials Filters and Filtering Filters Transfer Functions Filtering Sines and Cosines Filtering General Series Computing Transfer Functions Sequential Filtering Spectral Theory The Power Spectrum The Cramér Representation Using The Cramér Representation Power Spectrum Examples

5 Contents v 6.4 Statistical Inference Some Distribution Theory The Periodogram and its Distribution An Example Sunspot Numbers Estimating The Power Spectrum Tapering and Prewhitening Cross Spectral Analysis Computation A Simple Spectral Analysis Package for R Power Spectrum Estimation Cross-Spectral Analysis Examples

6 vi Contents

7 Chapter 1 Introduction 1.1 Time Series Time series arise as recordings of processes which vary over time. A recording can either be a continuous trace or a set of discrete observations. We will concentrate on the case where observations are made at discrete equally spaced times. By appropriate choice of origin and scale we can take the observation times to be 1, 2,... T and we can denote the observations by Y 1, Y 2,..., Y T. There are a number of things which are of interest in time series analysis. The most important of these are: Smoothing: The observed Y t are assumed to be the result of noise values ε t additively contaminating a smooth signal η t. Y t = η t + ε t We may wish to recover the values of the underlying η t. Modelling: We may wish to develop a simple mathematical model which explains the observed pattern of Y 1, Y 2,..., Y T. This model may depend on unknown parameters and these will need to be estimated. Forecasting: On the basis of observations Y 1, Y 2,..., Y T, we may wish to predict what the value of Y T +L will be (L 1), and possibly to give an indication of what the uncetainty is in the prediction. Control: We may wish to intervene with the process which is producing the Y t values in such a way that the future values are altered to produce a favourable outcome. 1.2 Stationarity and Non-Stationarity A key idea in time series is that of stationarity. Roughly speaking, a time series is stationary if its behaviour does not change over time. This means, for example, that the values always tend to vary about the same level and that their variability is constant over time. Stationary series have a rich theory and 1

8 2 Chapter 1. Introduction their behaviour is well understood. This means that they play a fundamental role in the study of time series. Obviously, not all time series that we encouter are stationary. Indeed, nonstationary series tend to be the rule rather than the exception. However, many time series are related in simple ways to series which are stationary. Two important examples of this are: Trend models : The series we observe is the sum of a determinstic trend series and a stationary noise series. A simple example is the linear trend model: Y t = β 0 + β 1 t + ε t. Another common trend model assumes that the series is the sum of a periodic seasonal effect and stationary noise. There are many other variations. Integrated models : The time series we observe satisfies Y t+1 Y t = ε t+1 where ε t is a stationary series. A particularly important model of this kind is the random walk. In that case, the ε t values are independent shocks which perturb the current state Y t by an amount ε t+1 to produce a new state Y t Some Examples Annual Auckland Rainfall Figure 1.1 shows the annual amount of rainfall in Auckland for the years from 1949 to The general pattern of rainfall looks similar throughout the record, so this series could be regarded as being stationary. (There is a hint that rainfall amounts are declining over time, but this type of effect can occur over shortish time spans for stationary series.) Nile River Flow Figure 1.2 shows the flow volume of the Nile at Aswan from 1871 to These are yearly values. The general pattern of this data does not change over time so it can be regarded as stationary (at least over this time period) Yield on British Government Securities Figure 1.3 shows the percentage yield on British Government securities, monthly over a 21 year period. There is a steady long-term increase in the yields. Over the period of observation a trend-plus-stationary series model looks like it might be appropriate. An integrated stationary series is another possibility.

9 1.3. Some Examples Ground Displacement in an Earthquake Figure 1.4 shows one component of the horizontal ground motion resulting from an earthquake. The initial motion (a little after 4 seconds) corresponds to the arrival of the p-wave and the large spike just before six seconds corresponds to the arrival of the s-wave. Later features correspond to the arrival of surface waves. This is an example of a transient signal and cannot have techniques appropriate for stationary series applied to it United States Housing Starts Figure 1.5 shows the monthly number of housing starts in the Unites States (in thousands). Housing starts are a leading economic indicator. This means that an increase in the number of housing starts indicates that economic growth is likely to follow and a decline in housing starts indicates that a recession may be on the way Iowa City Bus Ridership Figure 1.6 shows the monthly average weekday bus ridership for Iowa City over the period from September 1971 to December There is clearly a strong seasonal effect suprimposed on top of a general upward trend.

10 4 Chapter 1. Introduction Annual Rainfall (cm) Year Figure 1.1: Annual Auckland rainfall (in cm) from 1949 to 2000 (from Paul Cowpertwait) Flow Year Figure 1.2: Flow volume of the Nile at Aswan from 1871 to 1970 (from Durbin and Koopman).

11 1.3. Some Examples 5 8 Percent Month Figure 1.3: Monthly percentage yield on British Government securities over a 21 year period (from Chatfield) Displacement Seconds Figure 1.4: Horizontal ground displacement during a small Nevada earthquake (from Bill Peppin).

12 6 Chapter 1. Introduction 200 Housing Starts (000s) Time Figure 1.5: Housing starts in the United States (000s) (from S-Plus) Average Weekly Ridership Time Figure 1.6: Average weekday bus ridership, Iowa City (monthly ave) Sep Dec 1982.

13 Chapter 2 Vector Space Theory 2.1 Vectors In Two Dimensions The theory which underlies time series analysis is quite technical in nature. In spite of this, a good deal of intuition can be developed by approaching the subject geometrically. The geometric approach is based on the ideas of vectors and vector spaces Scalar Multiplication and Addition A good deal can be learnt about the theory of vectors by considering the twodimensional case. You can think of two dimensional vectors as being little arrows which have a length and a direction. Individual vectors can be stretched (altering their length but not their direction) and pairs of vectors can be added by placing them head to tail. To get more precise, we ll suppose that a vector v extends for a distance x in the horizontal direction and a distance y in the vertical direction. This gives us a representation of the vector as a pair of numbers and we can write it as v = (x, y). Doubling the length of the vector doubles the x and y values. In a similar way we can scale the length of the vector by any value, and this results in a similar scaling of the x and y values. This gives us a natural way of defining multiplication of a vector by a number. cv = (cx, cy) A negative value for c produces a reversal of direction as well as change of length of magnitude c. Adding two vectors amounts to placing them head to tail and taking the sum to be the arrow which extends from the base of the first to the head of the second. This corresponds to adding the x and y values corresponding to the vectors. For two vectors v 1 = (x 1, y 1 ) and v 2 = (x 2, y 2 ) the result is (x 1 + x 2, y 1 + y 2 ). This corresponds to a natural definition of addition for vectors. v 1 + v 2 = (x 1 + x 2, y 1 + y 2 ) 7

14 8 Chapter 2. Vector Space Theory Norms and Inner Products While coordinates give a complete description of vectors, it is often more useful to describe them in terms of the lengths of individual vectors and the angles between pairs of vectors. If we denote the length of a vector by u, then by Pythagoras theorem we know u = x 2 + y 2. Vector length has a number of simple properties: Positivity: For every vector u, we must have u 0, with equality if and only if u = 0. Linearity: For every scalar c and vector u we have cu = c u. The Triangle Inequality: If u and v are vectors, then u + v u + v. The first and second of these properties are obvious, and the third is simply a statement that the shortest distance between two points is a straight line. The technical mathematical name for the length of a vector is the norm of the vector. Although the length of an individual vector gives some information about it, it is also important to consider the angles between pairs of vectors. Suppose that we have vectors v 1 = (x 1, y 1 ) and v 2 = (x 2, y 2 ), and that we want to know the angle between them. If v 1 subtends an angle θ 1 with the x axis and v 2 subtends an angle θ 2 with the x axis, simple geometry tells us that: cos θ i = sin θ i = x i v i, y i v i. The angle we are interested in is θ 1 θ 2 and the cosine of this can be computed using the formulae: cos(α β) = cos α cos β + sin α sin β so that sin(α β) = sin α cos β cos α sin β, cos(θ 1 θ 2 ) = x 1x 2 + y 1 y 2 v 1 v 2 sin(θ 1 θ 2 ) = x 2y 1 x 1 y 2 v 1 v 2. Knowledge of the sine and cosine values makes it possible to compute the angle between the vectors. There are actually two angles between a pair of vectors; one measured clockwise and one measured counter-clockwise. Often we are interested in the smaller of these two angles. This can be determined from the cosine value (because cos θ = cos θ). The cosine is determined by the lengths of the two vectors and the quantity x 1 x 2 + y 1 y 2. This quantity is called the inner product of the vectors v 1 and v 2 and is denoted by v 1, v 2. Inner products have the following basic properties.

15 2.2. General Vector Spaces 9 Positivity: For every vector v, we must have v, v 0, with v, v = 0 only when v = 0. Linearity: For all vectors u, v, and w; and scalars α and β; we must have αu + βv, w = α u, w + β v, w. Symmetry: For all vectors u and v, we must have u, v = v, u (or in the case of complex vector spaces u, v = v, u, where the bar indicates a complex conjugate). It is clear that the norm can be defined in terms of the inner product by v 2 = v, v. It is easy to show that the properties of the norm v follow from those of the inner product. Two vectors u and v are said to be orthogonal if u, v = 0. This means that the vectors are at right angles to each other. 2.2 General Vector Spaces The theory and intuition obtained from studying two-dimensional vectors carries over into more general situations. It is not the particular representation of vectors as pairs of coordinates which is important but rather the ideas of addition, scalar multiplication and inner-product which are important Vector Spaces and Inner Products The following concepts provide an abstract generalisation of vectors in two dimensions. 1. A vector space is a set of objects, called vectors, which is closed under addition and scalar multiplication. This means that when vectors are scaled or added, the result is a vector. 2. An inner product on a vector space is a function which takes two vectors u and v and returns a scalar denoted by u,v which has the conditions of positivity, linearity and symmetry described in section A vector space with an associated inner product is called an inner product space. 3. The norm associated with an inner product space is defined in terms of the inner product as u = u, u. These ideas can be applied to quite general vector spaces. Although it may seem strange to apply ideas of direction and length to some of these spaces, thinking in terms of two and three dimensional pictures can be quite useful. Example The concept of two-dimensional vectors can be generalised by looking at the set of n-tuples of the form u = (u 1, u 2,..., u n ), where each u i is a real number. With addition and scalar multiplication defined element-wise,

16 10 Chapter 2. Vector Space Theory the set of all such n-tuples forms a vector space, usually denoted by R n. An inner product can be defined on the space by and this produces the norm u, v = u = n u i v i, i=1 ( n i=1 u 2 i ) 1 2. This generalisation of vector ideas to n dimensions provides the basis for a good deal of statistical theory. This is especially true for linear models and multivariate analysis. We will need a number of fundamental results on vector spaces. These follow directly from the definition of vector space, inner-product and norm, and do not rely on any special assumptions about the kind of vectors being considered. The Cauchy-Schwarz Inequality. For any two vectors u and v u, v u v. (2.1) Proof : For any two vectors u and v and scalars α and β, 0 αu βv 2 = αu βv, αu βv = α 2 u 2 2αβ u, v + β 2 v 2 Setting α = v and β = u, v / v yields 0 u 2 v 2 2 u, v 2 + u, v 2 = u 2 v 2 u, v 2. This can be rewritten as u, v u v. The Triangle Inequality. For any two vectors u and v, u + v u + v. Proof : For any u and v u + v 2 = u + v, u + v = u, u + u, v + v, u + v, v u u v + v 2 = ( u + v ) 2 The Angle Between Vectors. 1 (Cauchy-Schwarz) The Cauchy-Schwarz inequality means that x, y x y 1. This means that just as in the 2-d case we can define the angle between vectors u and v by 1 x, y θ = cos x y.

17 2.2. General Vector Spaces Some Examples The ideas in abstract vector space theory are derived from the concrete ideas of vectors in 2 and 3 dimensions. The real power of vector space theory is that it applies to much more general spaces. To show off the generality of the theory we ll now look at a couple of examples of less concrete spaces. Example The set F of all (real-valued) random variables with EX = 0 and E X 2 < is a vector space, because for any choice of X 1 and X 2 from F and scalars β 1 and β 2 E[β 1 X 1 + β 2 X 2 ] = 0 and E β 1 X 1 + β 2 X 2 2 <. It is possible to define an inner product on this vector space by This in turn produces the norm X, Y = EXY. X = ( E X 2) 1 2, which is just the standard deviation of X. The cosine of the angle between random variables is defined as EXY E X 2 E Y 2, which is recognisable as the correlation between the X and Y. (There is a technical problem with uniqueness in this case. Any random variable which has probability 1 of being zero will have X, X = 0, which violates the requirement that this only happen for X = 0. The workaround is to regard random variables as identical if they are the same with probability one.) Example The set of all continuous functions from [ 1, 1] to R is a vector space because there is a natural definition of addition and scalar multiplication for such functions. (f + g)(x) = f(x) + g(x) (βf)(x) = βf(x) A natural inner-product can be defined on this space by f, g = 1 1 f(x)g(x) dx. Vector space theory immediately gets us results such as 1 1 ( 1 f(x)g(x) dx 1 ) 1/2 ( 1 1/2 f(x) 2 dx g(x) dx) 2, 1 which is just a restatement of the Cauchy-Schwarz inequality.

18 12 Chapter 2. Vector Space Theory There is also a technical uniqueness problem in this case. This is handled by regarding functions f and g as equal if 1 1 f(x) g(x) 2 dx = 0. This happens, for example, if f and g differ only on a finite or countably infinite set of points. Integration theory gives a characterisation in terms of sets with measure zero. 2.3 Hilbert Spaces The vector spaces we ve looked at so far work perfectly well for performing finite operations such as forming linear combinations. However, there can be problems when considering limits. Consider the case of example Each of the functions f n defined by 0, x < 1 n, f n (x) = nx + 1, 1 2 n x 1 n, 1, x > 1 n, is continuous and the sequence obviously converges to the function 0, x < 0, f(x) = 1/2, x = 0, 1, x > 0. This limit function is not in the space under consideration because it is not continuous. Hilbert spaces add a requirement of completeness to those for an inner product space. In Hilbert spaces, sequences that look like they are converging will actually converge to an element of the space. To make this precise we need to define the meaning of convergence. Convergence. Suppose that {u n } is a sequence of vectors and u is a vector such that u n u 0, then u n is said to converge to u and this is denoted by u n u. If {u n } is a sequence of vectors such that the u n u and v is any vector then, by the Cauchy-Schwarz inequality u n, v u, v. In a similar way u n u. These properties are referred to as continuity of the inner product and norm. A sequence for which lim m,n u m u n 0 is called a Cauchy sequence. It is easy to see that every convergent sequence is a Cauchy sequence. If conversely every Cauchy sequence converges to an element of the space, the space is said to be complete. A complete inner product space is called a Hilbert space. Hilbert spaces preserve many of the important properties of R n. In particular the notions of length and direction retain their intuitive meanings. This makes it possible to carry out mathematical arguments geometrically and even to use pictures to understand what is happening in quite complex cases.

19 2.3. Hilbert Spaces Subspaces A subset M of a Hilbert space H is called a linear manifold if whenever u and v are elements of M then so is αu + βv. A linear manifold which contains the limit of every Cauchy sequence of its elements is called a linear subspace of H. A linear subspace of a Hilbert space is itself a Hilbert space. A vector v is orthogonal to a subspace M if v, u = 0 for every u M. The set all vectors which are orthogonal to M is itself a subspace denoted by M and called the orthogonal complement of M. A set of vectors {v λ : λ Λ} is said to generate a linear subspace M if M is the smallest subspace containing those vectors. It is relatively straightforward to show that the subspace consists of all linear combinations of the v λ s together with the limits of all Cauchy sequences formed from these linear combinations. We will use the notation sp{v λ : λ Λ} to indicate the subspace generated by the random variables {v λ : λ Λ} Projections If M is a subspace of a Hilbert space H and v is a vector not in M then the distance from M to v is defined to be min u M v u. A crucial result in Hilbert space theory tells us about how this minimum is attained. The Projection Theorem. There is a unique vector P M v M such that v P M v = min v u. u M The vector P M v satisfies P M v M and v P M v M. It is called the orthogonal projection of v on M. When M is generated by a set of elements {u λ : λ Λ}, the condition v P M v M is equivalent to the condition (v P M v) u λ for every λ Λ. In other words, v P M v, u λ = 0 for every λ Λ. This produces the equations P M v, u λ = v, u λ, λ Λ, which together with the requirement P M v M, completely determines P M v. When a subspace of a Hilbert space is generated by a countable set of orthogonal vectors {ξ n }, the orthogonal projection has the form P M v = n v, ξ n ξ n 2 ξ n In the case of a finite set {ξ 1,..., ξ n } this is easy to see. v, ξ 1 ξ 1 2 ξ v, ξ n ξ n 2 ξ n In addition, the orthogonality of {ξ 1,..., ξ n } means v, ξ1 ξ 1 2 ξ v, ξ n ξ n 2 ξ n, ξ i = v, ξ i ξ i 2 ξ i, ξ i = v, ξ i

20 14 Chapter 2. Vector Space Theory so the conditions above are verified. The general (countable) case follows from the continuity of the norm and inner product. 2.4 Hilbert Spaces and Prediction Consider the Hilbert space H consisting of all finite-variance random variables on some probability space, with inner product defined by X, Y = EXY. We will now look at the problem of predicting a variable Y using zero, one or more predicting random variables X 1,..., X n Linear Prediction The problem can be stated as follows. Given a response random variable Y and predictor random variables X 1,..., X n, what is the best way of predicting Y using a linear function of the Xs. This amounts to finding the coefficients which minimise the mean squared error or, in a Hilbert space setting, E Y β 0 β 1 X 1 β n X n 2, Y β 0 β 1 X 1 β n X n 2. The variables {1, X 1,..., X n } generate a subspace M of H, and the minimisation problem above amounts to finding the projection P M Y. We ll approach this problem in steps, beginning with n = 0. The subspace C generated by the constant random variable 1 consists of all constant random variables. Using the result above, the projection of a random variable Y with mean µ Y and variance σy 2 onto this subspace is P C Y = Y, = EY = µ Y. This tells us immediately that the value of c which minimises E[Y c] 2 is µ Y. Now consider the subspace L generated by 1 and a single random variable X with mean µ X and variance σx 2. This is clearly the same as the subspace generated by 1 and X EX. Since 1 and X EX are orthogonal, we can use the projection result of the previous section to compute the projection of Y onto L. P L Y = Y, 1 Y, X EX 1 + (X EX) 1 2 X EX 2 = EY + = EY + Y EY, X EX X EX 2 (X EX) + Y EY, X EX X EX 2 (X EX) EY, X EX X EX 2 (X EX)

21 2.4. Hilbert Spaces and Prediction 15 because EY, X EX = 0. Now Y EY, X EX = cov(y, X) which is in turn equal to ρ Y X σ Y σ X, where ρ YX is the correlation between Y and X. This means that P L Y = µ Y + ρ YX σ Y σ X (X µ X ). P L Y is the best possible linear prediction of Y based on X, because among all predictions of the form β 0 + β 1 X, it is the one which minimises E(Y β 0 β 1 X) 2. The general case of n predictors proceeds in exactly the same way, but is more complicated because we must use progressive orthogonalisation of the set of variables {1, X 1,..., X n }. The final result is that the best predictor of Y is P L Y = µ Y + Σ YX Σ 1 XX (X µ X), where X represents the variables X 1,..., X n assembled into a vector, µ X is the vector made up of the means of the Xs, Σ YX is the vector of covariances between Y and each of the Xs, and Σ XX is the variance-covariance matrix of the Xs General Prediction It is clear that linear prediction theory can be developed using Hilbert space theory. What is a little less clear is that Hilbert space theory also yields a general non-linear prediction theory. Linear prediction theory uses only linear information about the predictors and there is much more information available. In the one variable case, the additional information can be obtained by considering all possible (Borel) functions φ(x). These are still just random variables and so generate a subspace M of H. The projection onto this subspace gives the best possible predictor of Y based on X 1,..., X n. With some technical mathematics it is possible to show that this projection is in fact just the conditional expectation E[Y X]. More generally, it is possible to show that the best predictor of Y based on X 1,..., X n is E[Y X 1,..., X n ]. Although this is theoretically simple, it requires very strong assumptions about the distribution of Y and X 1,..., X n to actually compute an explicit value for the prediction. The simplest assumption to make is that Y and X 1,..., X n have a joint normal distribution. In this case, the general predictor and the linear one are identical. Because of this it is almost always the case that linear prediction is preferred to general prediction.

22 16 Chapter 2. Vector Space Theory

23 Chapter 3 Time Series Theory 3.1 Time Series We will assume that the time series values we observe are the realisations of random variables Y 1,..., Y T, which are in turn part of a larger stochastic process {Y t : t Z}. It is this underlying process that will be the focus for our theoretical development. Although it is best to distinguish the observed time series from the underlying stochastic process, the distinction is usually blurred and the term time series is used to refer to both the observations and the underlying process which generates them. The mean and the variance of random variables have a special place in the theory of statistics. In time series analysis, the analogs of these are the mean function and the autocovariance function. Definition (Mean and Autocovariance Functions): The mean function of a time series is defined to be µ(t) = EY t and the autocovariance function is defined to be γ(s, t) = cov(y s, Y t ). The mean and the autocovariance functions are fundamental parameters and it would be useful to obtain sample estimates of them. For general time series there are 2T + T (T 1)/2 parameters associated with Y 1,..., Y T and it is not possible to estimate all these parameters from T data values. To make any progress at all we must impose constraints on the time series we are investigating. The most common constraint is that of stationarity. There are two common definitions of stationarity. Definition (Strict Stationarity): A time series {Y t : t Z} is said to be strictly stationary if for any k > 0 and any t 1,..., t k Z, the distribution of (Y t1,..., Y tk ) is the same as that for for every value of u. (Y t1+u,..., Y tk +u) 17

24 18 Chapter 3. Time Series Theory This definition says that the stochastic behaviour of the process does not change through time. If Y t is stationary then and µ(t) = µ(0) γ(s, t) = γ(s t, 0). So for stationary series, the mean function is constant and the autocovariance function depends only on the time-lag between the two values for which the covariance is being computed. These two restrictions on the mean and covariance functions are enough for a reasonable amount of theory to be developed. Because of this a less restrictive definition of stationarity is often used in place of strict stationarity. Definition (Weak Stationarity): A time series is said to be weakly, widesense or covariance stationary if E Y t 2 <, µ(t) = µ and γ(t + u, t) = γ(u, 0) for all t and u. In the case of Gaussian time series, the two definitions of stationarity are equivalent. This is because the finite dimensional distributions of the time series are completely characterised by the mean and covariance functions. When time series are stationary it is possible to simplify the parameterisation of the mean and autocovariance functions. In this case we can define the mean of the series to be µ = E(Y t ) and the autocovariance function to be γ(u) = cov(y t+u, Y t ). We will also have occasion to examine the autocorrelation function ρ(u) = γ(u) γ(0) = cor(y t+u, Y t ). Example (White Noise) If the random variables which make up {Y t } are uncorrelated, have means 0 and variance σ 2, then {Y t } is stationary with autocovariance function { σ 2 u = 0, γ(u) = 0 otherwise. This type of series is referred to as white noise. 3.2 Hilbert Spaces and Stationary Time Series Suppose that {Y t : t Z} is a stationary zero-mean time series. We can consider the Hilbert space H generated by the random variables {Y t : t Z} with inner product X, Y = E(XY ), and norm X 2 = E X 2. At a given time t, we can consider the subspace M generated by the random variables {Y u : u t}. This subspace represents the past and present of the process. Future values of the series can be predicted by projecting onto the

25 3.3. The Lag and Differencing Operators 19 subspace M. For example, Y t+1 can be predicted by P M Y t+1, Y t+1 by P M Y t+2 and so on. Computing these predictions requires a knowledge of the autocovariance function of the time series and typically this is not known. We will spend a good deal of this chapter studying simple parametric models for time series. By fitting such models we will be able to determine the covariance structure of the time series and so be able to obtain predictions or forecasts of future values. 3.3 The Lag and Differencing Operators The lag operator L is defined for a time series {Y t } by LY t = Y t 1. The operator can be defined for linear combinations by L(c 1 Y t1 + c 2 Y t2 ) = c 1 Y t1 1 + c 2 Y t2 1 and can be extended to all of H by a suitable definition for limits. In addition to being linear, the lag operator preserves inner products. LY s, LY t = cov(y s 1, Y t 1 ) = cov(y s, Y t ) = Y s, Y t (An operator of this type is called a unitary operator.) There is a natural calculus of operators on H. For example we can define powers of L naturally by and linear combinations by L 2 Y t = LLY t = LY t 1 = Y t 2 L 3 Y t = LL 2 Y t = Y t 3. L k Y t = Y t k (αl k + βl l )Y t = αy t k + βy t l. Other operators can be defined in terms in terms of L. operator defined by Y t = (1 L)Y t = Y t Y t 1 The differencing is of fundamental importance when dealing with models for non-stationary time series. Again, we can define powers of this operator 2 Y t = ( Y t ) = (Y t Y t 1 ) = (Y t Y t 1 ) (Y t 1 Y t 2 ) = Y t 2Y t 1 + Y t 2. We will not dwell on the rich Hilbert space theory associated with time series, but it is important to know that many of the operator manipulations which we will carry out can be placed on a rigorous footing.

26 20 Chapter 3. Time Series Theory 3.4 Linear Processes We will now turn to an examination of a large class of useful time series models. These are almost all defined in terms of the lag operator L. As the simplest example, consider the autoregressive model defined by: Y t = φy t 1 + ε t, (3.1) where φ is a constant with φ < 1 and ε t is a sequence of uncorrelated random variables, each with with mean 0 and variance σ 2. From a statistical point of view this model makes perfect sense, but is not clear that any Y t which satisfies this equation exists. One way to see that there is a solution is to re-arrange equation 3.1 and write it in its operator form. (1 φl)y t = ε t. Formally inverting the operator (1 φl) leads to Y t = (1 φl) 1 ε t = φ u L u ε t = u=0 φ u ε t u. u=0 The series on the right is defined as the limit as n of n φ u ε t u. u=0 Loosely speaking, this limit exists if Since u=n+1 u=n+1 φ u ε t u 2 0. φ u ε t u 2 = var( = u=n+1 u=n+1 φ 2u σ 2 φ u ε t u ) and φ < 1, there is indeed a well-defined solution of 3.1. Further, this solution can be written as an (infinite) moving average of current and earlier ε t values. This type of infinite moving average plays a special role in the theory of time series. Definition (Linear Processes) The time series Y t defined by Y t = u= ψ u ε t u

27 3.5. Autoregressive Series 21 where ε t is a white-noise series and is called a linear process. u= ψ u 2 < The general linear process depends on both past and future values of ε t. A linear process which depends only on the past and present values of ε t is said to be causal. Causal processes are preferred for forecasting because they reflect the way in which we believe the real world works. Many time series can be represented as linear processes. This provides a unifying underpinning for time series theory, but may be of limited practical interest because of the potentially infinite number of parameters required. 3.5 Autoregressive Series Definition (Autoregressive Series) If Y t satisfies Y t = φ 1 Y t φ p Y t p + ε t where ε t is white-noise and the φ u are constants, then Y t is called an autoregressive series of order p, denoted by AR(p). Autoregressive series are important because: 1. They have a natural interpretation the next value observed is a slight perturbation of a simple function of the most recent observations. 2. It is easy to estimate their parameters. It can be done with standard regression software. 3. They are easy to forecast. Again standard regression software will do the job The AR(1) Series The AR(1) series is defined by Y t = φy t 1 + ε t. (3.2) Because Y t 1 and ε t are uncorrelated, the variance of this series is var(y t ) = φ 2 var(y t 1 ) + σ 2 ε. If {Y t } is stationary then var(y t ) = var(y t 1 ) = σ 2 Y and so σ 2 Y = φ 2 σ 2 Y + σ 2 ε. (3.3) This implies that and hence σ 2 Y > φ 2 σ 2 Y 1 > φ 2.

28 22 Chapter 3. Time Series Theory There is an alternative view of this, using the operator formulation of equation 3.2, namely (1 φl)y t = ε t It is possible to formally invert the autoregressive operator to obtain (1 φl) 1 = φ u L u. u=0 Applying this to the series {ε t } produces the representation Y t = φ u ε t u. (3.4) u=0 If φ < 1 this series converges in mean square because φ 2u <. The limit series is stationary and satisfies equation 3.2. Thus, if φ < 1 then there is a stationary solution to 3.2. An equivalent condition is that the root of the equation 1 φz = 0 (namely 1/φ) lies outside the unit circle in the complex plane. If φ > 1 the series defined by 3.4 does not converge. We can, however, rearrange the defining equation 3.2 in the form Y t = 1 φ Y t+1 1 φ ε t+1, and a similar argument to that above produces the representation Y t = φ u ε t+u. u=0 This series does converge, so there is a stationary series Y t which satisfies 3.2. The resulting series is generally not regarded as satisfactory for modelling and forecasting because it is not causal, i.e. it depends on future values of ε t. If φ = 1 there is no stationary solution to 3.2. This means that for the purposes of modelling and forecasting stationary time series, we must restrict our attention to series for which φ < 1 or, equivalently, to series for which the root of the polynomial 1 φz lies outside the unit circle in the complex plane. If we multiply both sides of equation 3.2 by Y t u and take expectations we obtain E(Y t Y t u ) = φe(y t 1 Y t u ) + E(ε t Y t u ). The term on the right is zero because, from the linear process representation, ε t is independent of earlier Y t values. This means that the autocovariances must satisfy the recursion γ(u) = φγ(u 1), u = 1, 2, 3,.... This is a first-order linear difference equation with solution γ(u) = φ u γ(0), u = 0, 1, 2,...

29 3.5. Autoregressive Series 23 φ 1 = 0.7 φ 1 = ACF Figure 3.1: Autocorrelation functions for two of AR(1) models. By rearranging equation 3.3 we find γ(0) = σ 2 ε/(1 φ 2 ), and hence that γ(u) = φu σε 2, u = 0, 1, 2,... 1 φ2 This in turn means that the autocorrelation function is given by ρ(u) = φ u, u = 0, 1, 2,... The autocorrelation functions for the the AR(1) series with φ 1 =.7 and φ 1 =.7 are shown in figure 3.1. Both functions show exponential decay The AR(2) Series The AR(2) model is defined by or, in operator form Y t = φ 1 Y t 1 + φ 2 Y t 2 + ε t (3.5) (1 φ 1 L φ 2 L 2 )Y t = ε t. As in the AR(1) case we can consider inverting the AR operator. To see whether this is possible, we can consider factorising the operator 1 φ 1 L φ 2 L 2 and inverting each factor separately. Suppose that 1 φ 1 L φ 2 L 2 = (1 c 1 L)(1 c 2 L),

30 24 Chapter 3. Time Series Theory then it is clear that we can invert the operator if we can invert each factor separately. This is possible if c 1 < 1 and c 2 < 1, or equivalently, if the roots of the polynomial 1 φ 1 z φ 2 z 2 lie outside the unit circle. A little algebraic manipulation shows that this is equivalent to the conditions: φ 1 + φ 2 < 1, φ 1 + φ 2 < 1, φ 2 > 1. These constraints define a triangular region in the φ 1, φ 2 plane. The region is shown as the shaded triangle in figure 3.3. The autocovariance function for the AR(2) series can be investigated by multiplying both sides of equation 3.5 by Y t u and taking expectations. E(Y t Y t u ) = φ 1 E(Y t 1 Y t u ) + φ 2 E(Y t 2 Y t u ) + E(ε t Y t u ). This in turns leads to the recurrence with initial conditions or, using the fact that γ( u) = γ(u), γ(u) = φ 1 γ(u 1) + φ 2 γ(u 2) γ(0) = φ 1 γ( 1) + φ 2 γ( 2) + σ 2 ε γ(1) = φ 1 γ(0) + φ 2 γ( 1). γ(0) = φ 1 γ(1) + φ 2 γ(2) + σ 2 ε γ(1) = φ 1 γ(0) + φ 2 γ(1). The solution to these equations has the form γ(u) = A 1 G u 1 + A 2 G u 2 where G 1 1 and G 1 2 are the roots of the polynomial 1 φ 1 z φ 2 z 2 (3.6) and A 1 and A 2 are constants that can be determined from the initial conditions. In the case that the roots are equal, the solution has the general form γ(u) = (A 1 + A 2 u)g u These equations indicate that the autocovariance function for the AR(2) series will exhibit (exponential) decay as u. If G k corresponds to a complex root, then and hence G k = G k e iθ k G u k = G k u e iθ ku = G k u (cos θ k u + i sin θ k u) Complex roots will thus introduce a pattern of decaying sinusoidal variation into the covariance function (or autocorrelation function). The region of the φ 1, φ 2 plane corresponding to complex roots is indicated by the cross-hatched region in figure 3.3.

31 3.5. Autoregressive Series 25 φ 1 = 0.5, φ 2 = 0.3 φ 1 = 1, φ 2 = ACF φ 1 = 0.5, Lagφ 2 = φ 1 = 0.5, Lagφ 2 = ACF Figure 3.2: Autocorrelation functions for a variety of AR(2) models.

32 26 Chapter 3. Time Series Theory 1 φ φ 1 Figure 3.3: The regions of φ 1 /φ 2 space where the series produced by the AR(2) scheme is stationary (indicated in grey) and has complex roots (indicated by cross-hatching). The AR(p) Series The AR(p) series is defined by If the roots of Y t = φ 1 Y t φ p Y t p + ε t (3.7) 1 φ 1 z φ p z p (3.8) lie outside the unit circle, there is a stationary causal solution to 3.7. The autocovariance function can be investigated by multiplying equation 3.7 by Y t u and taking expectations. This yields γ(u) = φ 1 γ(u 1) + + φ p γ(u p). This is a linear homogeneous difference equation and has the general solution γ(u) = A 1 G u A p G u p (this is for distinct roots), where G 1,...,G p are the reciprocals of the roots of equation 3.8. Note that the stationarity condition means that γ(u) 0, exhibiting exponential decay. As in the AR(2) case, complex roots will introduce oscillatory behaviour into the autocovariance function Computations R contains a good deal of time series functionality. On older versions of R (those prior to 1.7.0) you will need to type the command

33 3.5. Autoregressive Series 27 > library(ts) to ensure that this functionality is loaded. The function polyroot can be used to find the roots of polynomials and so determine whether a proposed model is stationary. Consider the model or, its equivalent operator form Y t = 2.5Y t 1 Y t 2 + ε t (1 2.5L + L 2 )Y t = ε t. We can compute magnitudes of the roots of the polynomial 1 2.5z + z 2 with polyroot. > Mod(polyroot(c(1,-2.5,1))) [1] The roots have magnitudes.5 and 2. Because the first of these is less than 1 in magnitude the model is thus not stationary and causal. For the model Y t = 1.5Y t 1.75Y t 2 + ε t or its operator equivalent (1 1.5L +.75L 2 )Y t = ε t we can check stationarity by examining the magnitudes of the roots of 1 1.5z +.75z 2. > Mod(polyroot(c(1,-1.5,.75))) [1] Both roots are bigger than 1 in magnitude, so the series is stationary. We can obtain the roots themselves as follows. > polyroot(c(1,-1.5,.75)) [1] i i Because the roots are complex we can expect to see a cosine-like ripple in the autocovariance function and the autocorrelation function. The autocorrelation function for a given model can be computed using the ARMAacf function. The acf for the model above can be computed and plotted as follows. > plot(0:14, ARMAacf(ar=c(1.5,-.75), lag=14), type="h", xlab = "Lag", ylab = "ACF") > abline(h = 0) The result is shown in figure 3.4. Finally, it may be useful to simulate a time series of a given form. We can create a time series from the model Y t = 1.5Y t 1.75Y t 2 + ε t and plot it with the following statements. > x = arima.sim(model = list(ar=c(1.5,-.75)), n = 100) > plot(x) The result is shown in figure 3.5. Note that there is evidence that the series contains a quasi-periodic component with period about 12, as suggested by the autocorrelation function.

34 28 Chapter 3. Time Series Theory ACF Lag Figure 3.4: The acf for the model Y t = 1.5Y t 1.75Y t 2 + ε t x Time Figure 3.5: Simulation of the model Y t = 1.5Y t 1.75Y t 2 + ε t.

35 3.6. Moving Average Series Moving Average Series A time series {Y t } which satisfies Y t = ε t + θ 1 ε t θ q ε t q (3.9) (with {ε t } white noise) is said to be a moving average process of order q or MA(q) process. No additional conditions are required to ensure stationarity. The autocovariance function for the MA(q) process is (1 + θ θq)σ 2 2 u = 0 γ(u) = (θ u + θ 1 θ u θ q u θ q )σ 2 u = 1,..., q 0 otherwise. which says there is only a finite span of dependence on the series. Note that it is easy to distinguish MA and AR series by the behaviour of their autocorrelation functions. The acf for MA series cuts off sharply while that for an AR series decays exponentially (with a possible sinusoidal ripple superimposed) The MA(1) Series The MA(1) series is defined by Y t = ε t + θε t 1. (3.10) It has autocovariance function (1 + θ 2 )σ 2 u = 0 γ(u) = θσ 2 u = 1 0 otherwise. and autocorrelation function θ ρ(u) = 1 + θ 2 for u = 1, 0 otherwise. (3.11) Invertibility If we replace θ by 1/θ and σ 2 by θσ 2 the autocorrelation function given by 3.11 is unchanged. There are thus two sets of parameter values which can explain the structure of the series. For the general process defined by equation 3.9, there is a similar identifiability problem. The problem can be resolved by requiring that the operator 1 + θ 1 L + + θ q L q be invertible i.e. that all roots of the polynomial lie outside the unit circle. 1 + θ 1 z + + θ q z q

36 30 Chapter 3. Time Series Theory ACF Lag Figure 3.6: The acf for the model Y t = ε t +.9ε t Computation The function polyroot can be used to check invertibility for MA models. Remember that the invertibility requirement is only so that each MA model is only defined by one set of parameters. The function ARMAacf can be used to compute the acf for MA series. For example, the acf of the model Y t = ε t + 0.9ε t 1 can be computed and plotted as follows. > plot(0:14, ARMAacf(ma=.9, lag=14), type="h", xlab = "Lag", ylab = "ACF") > abline(h = 0) The result is shown in figure 3.6. A simulation of the series can be computed and plotted as follows. > x = arima.sim(model = list(ma=.9), n = 100) > plot(x) The result of the simulation is shown in figure Autoregressive Moving Average Series Definition If a series satisfies Y t = φ 1 Y t φ p Y t p + ε t + θ 1 ε t θ q ε t q (3.12)

37 3.7. Autoregressive Moving Average Series x Time Figure 3.7: Simulation of the model Y t = ε t +.9ε t 1. (with {ε t } white noise), it is called an autoregressive-moving average series of order (p, q), or an ARMA(p, q) series. An ARMA(p, q) series is stationary if the roots of the polynomial lie outside the unit circle. 1 φ 1 z φ p z p The ARMA(1,1) Series The ARMA(1,1) series is defined by Y t = φy t 1 + ε t + θε t 1. (3.13) To derive the autocovariance function for Y t, note that and E(ε t Y t ) = E[ε t (φy t 1 + ε t + θε t 1 )] = σ 2 ε E(ε t 1 Y t ) = E[ε t 1 (φy t 1 + ε t + θε t 1 )] = φσ 2 ε + θσ 2 ε = (φ + θ)σ 2 ε.

38 32 Chapter 3. Time Series Theory Multiplying equation 3.13 by Y t u and taking expectation yields: φγ(1) + (1 + θ(φ + θ))σε 2 u = 0 γ(u) = φγ(0) + θσε 2 u = 1 φγ(u 1) u 2 Solving the first two equations produces γ(0) = (1 + 2θφ + θ2 ) 1 φ 2 σ 2 ε = (1 + 2θφ + θ2 ) 1 φ 2 σ 2 ε and using the last recursively shows γ(u) = (1 + θφ)(φ + θ) 1 φ 2 φ u 1 σ 2 ε for u 1. The autocorrelation function can then be computed as ρ(u) = (1 + θφ)(φ + θ) (1 + 2θφ + θ 2 ) φu 1 for u 1. The pattern here is similar to that for AR(1), except for the first term The ARMA(p,q) Model It is possible to make general statements about the behaviour of general ARMA(p,q) series. When values are more than q time units apart, the memory of the movingaverage part of the series is lost. The functions γ(u) and ρ(u) will then behave very similarly to those for the AR(p) series Y t = φ 1 Y t φ p Y t p + ε t for large u, but the first few terms will exhibit additional structure Computation Stationarity can be checked by examining the roots of the characteristic polynomial of the AR operator and model parameterisation can be checked by examining the roots of the characteristic polynomial of the MA operator. Both checks can be carried out with polyroot. The autocorrelation function for an ARMA series can be computed with ARMAacf. For the model this can be done as follows Y t =.5Y t 1 + ε t +.3ε t 1 > plot(0:14, ARMAacf(ar=-.5, ma=.3, lag=14), type="h", xlab = "Lag", ylab = "ACF") > abline(h = 0) and produces the result shown in figure 3.8. Simulation can be carried out using arma.sim > x = arima.sim(model = list(ar=-.5,ma=.3), n = 100) > plot(x) producing the result in figure 3.9.

39 3.7. Autoregressive Moving Average Series ACF Lag Figure 3.8: The acf for the model Y t =.5Y t 1 + ε t +.3ε t x Time Figure 3.9: Simulation of the model Y t =.5Y t 1 + ε t +.3ε t 1.

40 34 Chapter 3. Time Series Theory Common Factors If the AR and MA operators in an ARMA(p, q) model possess common factors, then the model is over-parameterised. By dividing through by the common factors we can obtain a simpler model giving and identical description of the series. It is important to recognise common factors in an ARMA model because they will produce numerical problems in model fitting. 3.8 The Partial Autocorrelation Function The autocorrelation function of an MA series exhibits different behaviour from that of AR and general ARMA series. The acf of an MA series cuts of sharply whereas those for AR and ARMA series exhibit exponential decay (with possible sinusoidal behaviour superimposed). This makes it possible to identify an ARMA series as being a purely MA one just by plotting its autocorrelation function. The partial autocorrelation function provides a similar way of identifying a series as a purely AR one. Given a stretch of time series values..., Y t u, Y t u+1,..., Y t 1, Y t,... the partial correlation of Y t and Y t u is the correlation between these random variables which is not conveyed through the intervening values. If the Y values are normally distributed, the partial autocorrelation between Y t and Y t u can be defined as φ(u) = cor(y t, Y t u Y t 1,..., Y t u+1 ). A more general approach is based on regression theory. Consider predicting Y t based on Y t 1,..., Y t u+1. The prediction is with the βs chosen to minimise Ŷ t = β 1 Y t 1 + β 2 Y t 2, β u 1 Y t u+1 E(Y t Ŷt) 2. It is also possible to think backwards in time and consider predicting Y t u with the same set of predictors. The best predictor will be Ŷ i u = β 1 Y t u+1 + β 2 Y t u+2, β u 1 Y t 1. (The coefficients are the same because the correlation structure is the same whether the series is run forwards or backwards in time. The partial correlation function at lag u is the correlation between the prediction errors. By convention we take φ(1) = ρ(1). φ(u) = cor(y t Ŷt, Y t u Ŷt u)

### Section 1.1. Introduction to R n

The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to

### 3. INNER PRODUCT SPACES

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

### A Multiplicative Seasonal Box-Jenkins Model to Nigerian Stock Prices

A Multiplicative Seasonal Box-Jenkins Model to Nigerian Stock Prices Ette Harrison Etuk Department of Mathematics/Computer Science, Rivers State University of Science and Technology, Nigeria Email: ettetuk@yahoo.com

### 1 Short Introduction to Time Series

ECONOMICS 7344, Spring 202 Bent E. Sørensen January 24, 202 Short Introduction to Time Series A time series is a collection of stochastic variables x,.., x t,.., x T indexed by an integer value t. The

### Time Series Analysis

JUNE 2012 Time Series Analysis CONTENT A time series is a chronological sequence of observations on a particular variable. Usually the observations are taken at regular intervals (days, months, years),

### Introduction to ARMA Models

Statistics 910, #8 1 Overview Introduction to ARMA Models 1. Modeling paradigm 2. Review stationary linear processes 3. ARMA processes 4. Stationarity of ARMA processes 5. Identifiability of ARMA processes

### Univariate and Multivariate Methods PEARSON. Addison Wesley

Time Series Analysis Univariate and Multivariate Methods SECOND EDITION William W. S. Wei Department of Statistics The Fox School of Business and Management Temple University PEARSON Addison Wesley Boston

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

### Analysis and Computation for Finance Time Series - An Introduction

ECMM703 Analysis and Computation for Finance Time Series - An Introduction Alejandra González Harrison 161 Email: mag208@exeter.ac.uk Time Series - An Introduction A time series is a sequence of observations

### Sales forecasting # 2

Sales forecasting # 2 Arthur Charpentier arthur.charpentier@univ-rennes1.fr 1 Agenda Qualitative and quantitative methods, a very general introduction Series decomposition Short versus long term forecasting

### State Space Time Series Analysis

State Space Time Series Analysis p. 1 State Space Time Series Analysis Siem Jan Koopman http://staff.feweb.vu.nl/koopman Department of Econometrics VU University Amsterdam Tinbergen Institute 2011 State

### Time Series Analysis

Time Series 1 April 9, 2013 Time Series Analysis This chapter presents an introduction to the branch of statistics known as time series analysis. Often the data we collect in environmental studies is collected

### TIME SERIES ANALYSIS

TIME SERIES ANALYSIS L.M. BHAR AND V.K.SHARMA Indian Agricultural Statistics Research Institute Library Avenue, New Delhi-0 02 lmb@iasri.res.in. Introduction Time series (TS) data refers to observations

### Solving Simultaneous Equations and Matrices

Solving Simultaneous Equations and Matrices The following represents a systematic investigation for the steps used to solve two simultaneous linear equations in two unknowns. The motivation for considering

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### Time Series Analysis

Time Series Analysis Autoregressive, MA and ARMA processes Andrés M. Alonso Carolina García-Martos Universidad Carlos III de Madrid Universidad Politécnica de Madrid June July, 212 Alonso and García-Martos

### Time Series - ARIMA Models. Instructor: G. William Schwert

APS 425 Fall 25 Time Series : ARIMA Models Instructor: G. William Schwert 585-275-247 schwert@schwert.ssb.rochester.edu Topics Typical time series plot Pattern recognition in auto and partial autocorrelations

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### Time Series Analysis

Time Series Analysis Identifying possible ARIMA models Andrés M. Alonso Carolina García-Martos Universidad Carlos III de Madrid Universidad Politécnica de Madrid June July, 2012 Alonso and García-Martos

### Lecture 4: Seasonal Time Series, Trend Analysis & Component Model Bus 41910, Time Series Analysis, Mr. R. Tsay

Lecture 4: Seasonal Time Series, Trend Analysis & Component Model Bus 41910, Time Series Analysis, Mr. R. Tsay Business cycle plays an important role in economics. In time series analysis, business cycle

### South Carolina College- and Career-Ready (SCCCR) Pre-Calculus

South Carolina College- and Career-Ready (SCCCR) Pre-Calculus Key Concepts Arithmetic with Polynomials and Rational Expressions PC.AAPR.2 PC.AAPR.3 PC.AAPR.4 PC.AAPR.5 PC.AAPR.6 PC.AAPR.7 Standards Know

### Univariate Time Series Analysis; ARIMA Models

Econometrics 2 Spring 25 Univariate Time Series Analysis; ARIMA Models Heino Bohn Nielsen of4 Outline of the Lecture () Introduction to univariate time series analysis. (2) Stationarity. (3) Characterizing

### Some useful concepts in univariate time series analysis

Some useful concepts in univariate time series analysis Autoregressive moving average models Autocorrelation functions Model Estimation Diagnostic measure Model selection Forecasting Assumptions: 1. Non-seasonal

### Econometrics Simple Linear Regression

Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### ITSM-R Reference Manual

ITSM-R Reference Manual George Weigt June 5, 2015 1 Contents 1 Introduction 3 1.1 Time series analysis in a nutshell............................... 3 1.2 White Noise Variance.....................................

### Chapter 4: Vector Autoregressive Models

Chapter 4: Vector Autoregressive Models 1 Contents: Lehrstuhl für Department Empirische of Wirtschaftsforschung Empirical Research and und Econometrics Ökonometrie IV.1 Vector Autoregressive Models (VAR)...

### ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems

Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed

### 4.3 Moving Average Process MA(q)

66 CHAPTER 4. STATIONARY TS MODELS 4.3 Moving Average Process MA(q) Definition 4.5. {X t } is a moving-average process of order q if X t = Z t + θ 1 Z t 1 +... + θ q Z t q, (4.9) where and θ 1,...,θ q

### Time Series Analysis

Time Series Analysis Forecasting with ARIMA models Andrés M. Alonso Carolina García-Martos Universidad Carlos III de Madrid Universidad Politécnica de Madrid June July, 2012 Alonso and García-Martos (UC3M-UPM)

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### Discrete Time Series Analysis with ARMA Models

Discrete Time Series Analysis with ARMA Models Veronica Sitsofe Ahiati (veronica@aims.ac.za) African Institute for Mathematical Sciences (AIMS) Supervised by Tina Marquardt Munich University of Technology,

### Introduction to Time Series Analysis. Lecture 6.

Introduction to Time Series Analysis. Lecture 6. Peter Bartlett www.stat.berkeley.edu/ bartlett/courses/153-fall2010 Last lecture: 1. Causality 2. Invertibility 3. AR(p) models 4. ARMA(p,q) models 1 Introduction

### 4. Simple regression. QBUS6840 Predictive Analytics. https://www.otexts.org/fpp/4

4. Simple regression QBUS6840 Predictive Analytics https://www.otexts.org/fpp/4 Outline The simple linear model Least squares estimation Forecasting with regression Non-linear functional forms Regression

### Biggar High School Mathematics Department. National 5 Learning Intentions & Success Criteria: Assessing My Progress

Biggar High School Mathematics Department National 5 Learning Intentions & Success Criteria: Assessing My Progress Expressions & Formulae Topic Learning Intention Success Criteria I understand this Approximation

### Inner Product Spaces

Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

### SIGNAL PROCESSING & SIMULATION NEWSLETTER

1 of 10 1/25/2008 3:38 AM SIGNAL PROCESSING & SIMULATION NEWSLETTER Note: This is not a particularly interesting topic for anyone other than those who ar e involved in simulation. So if you have difficulty

### NEW YORK STATE TEACHER CERTIFICATION EXAMINATIONS

NEW YORK STATE TEACHER CERTIFICATION EXAMINATIONS TEST DESIGN AND FRAMEWORK September 2014 Authorized for Distribution by the New York State Education Department This test design and framework document

### CORRELATED TO THE SOUTH CAROLINA COLLEGE AND CAREER-READY FOUNDATIONS IN ALGEBRA

We Can Early Learning Curriculum PreK Grades 8 12 INSIDE ALGEBRA, GRADES 8 12 CORRELATED TO THE SOUTH CAROLINA COLLEGE AND CAREER-READY FOUNDATIONS IN ALGEBRA April 2016 www.voyagersopris.com Mathematical

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### Purpose of Time Series Analysis. Autocovariance Function. Autocorrelation Function. Part 3: Time Series I

Part 3: Time Series I Purpose of Time Series Analysis (Figure from Panofsky and Brier 1968) Autocorrelation Function Harmonic Analysis Spectrum Analysis Data Window Significance Tests Some major purposes

### Nonlinear Iterative Partial Least Squares Method

Numerical Methods for Determining Principal Component Analysis Abstract Factors Béchu, S., Richard-Plouet, M., Fernandez, V., Walton, J., and Fairley, N. (2016) Developments in numerical treatments for

### Figure 1.1 Vector A and Vector F

CHAPTER I VECTOR QUANTITIES Quantities are anything which can be measured, and stated with number. Quantities in physics are divided into two types; scalar and vector quantities. Scalar quantities have

### MATHEMATICS (CLASSES XI XII)

MATHEMATICS (CLASSES XI XII) General Guidelines (i) All concepts/identities must be illustrated by situational examples. (ii) The language of word problems must be clear, simple and unambiguous. (iii)

### Chapter 9: Univariate Time Series Analysis

Chapter 9: Univariate Time Series Analysis In the last chapter we discussed models with only lags of explanatory variables. These can be misleading if: 1. The dependent variable Y t depends on lags of

### Luciano Rispoli Department of Economics, Mathematics and Statistics Birkbeck College (University of London)

Luciano Rispoli Department of Economics, Mathematics and Statistics Birkbeck College (University of London) 1 Forecasting: definition Forecasting is the process of making statements about events whose

### Univariate Time Series Analysis; ARIMA Models

Econometrics 2 Fall 25 Univariate Time Series Analysis; ARIMA Models Heino Bohn Nielsen of4 Univariate Time Series Analysis We consider a single time series, y,y 2,..., y T. We want to construct simple

### Lecture 14: Section 3.3

Lecture 14: Section 3.3 Shuanglin Shao October 23, 2013 Definition. Two nonzero vectors u and v in R n are said to be orthogonal (or perpendicular) if u v = 0. We will also agree that the zero vector in

### Chapter 1. Vector autoregressions. 1.1 VARs and the identi cation problem

Chapter Vector autoregressions We begin by taking a look at the data of macroeconomics. A way to summarize the dynamics of macroeconomic data is to make use of vector autoregressions. VAR models have become

### Time Series Analysis and Forecasting

Time Series Analysis and Forecasting Math 667 Al Nosedal Department of Mathematics Indiana University of Pennsylvania Time Series Analysis and Forecasting p. 1/11 Introduction Many decision-making applications

### ELEMENTS OF VECTOR ALGEBRA

ELEMENTS OF VECTOR ALGEBRA A.1. VECTORS AND SCALAR QUANTITIES We have now proposed sets of basic dimensions and secondary dimensions to describe certain aspects of nature, but more than just dimensions

### Thnkwell s Homeschool Precalculus Course Lesson Plan: 36 weeks

Thnkwell s Homeschool Precalculus Course Lesson Plan: 36 weeks Welcome to Thinkwell s Homeschool Precalculus! We re thrilled that you ve decided to make us part of your homeschool curriculum. This lesson

### MATH 304 Linear Algebra Lecture 24: Scalar product.

MATH 304 Linear Algebra Lecture 24: Scalar product. Vectors: geometric approach B A B A A vector is represented by a directed segment. Directed segment is drawn as an arrow. Different arrows represent

### MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.

MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α

### α = u v. In other words, Orthogonal Projection

Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v

### We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.

Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to

### PHYSICS 151 Notes for Online Lecture #6

PHYSICS 151 Notes for Online Lecture #6 Vectors - A vector is basically an arrow. The length of the arrow represents the magnitude (value) and the arrow points in the direction. Many different quantities

### Time Series Analysis 1. Lecture 8: Time Series Analysis. Time Series Analysis MIT 18.S096. Dr. Kempthorne. Fall 2013 MIT 18.S096

Lecture 8: Time Series Analysis MIT 18.S096 Dr. Kempthorne Fall 2013 MIT 18.S096 Time Series Analysis 1 Outline Time Series Analysis 1 Time Series Analysis MIT 18.S096 Time Series Analysis 2 A stochastic

### Software Review: ITSM 2000 Professional Version 6.0.

Lee, J. & Strazicich, M.C. (2002). Software Review: ITSM 2000 Professional Version 6.0. International Journal of Forecasting, 18(3): 455-459 (June 2002). Published by Elsevier (ISSN: 0169-2070). http://0-

### Lecture 2: ARMA(p,q) models (part 3)

Lecture 2: ARMA(p,q) models (part 3) Florian Pelgrin University of Lausanne, École des HEC Department of mathematics (IMEA-Nice) Sept. 2011 - Jan. 2012 Florian Pelgrin (HEC) Univariate time series Sept.

### COMP6053 lecture: Time series analysis, autocorrelation. jn2@ecs.soton.ac.uk

COMP6053 lecture: Time series analysis, autocorrelation jn2@ecs.soton.ac.uk Time series analysis The basic idea of time series analysis is simple: given an observed sequence, how can we build a model that

### Adding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors

1 Chapter 13. VECTORS IN THREE DIMENSIONAL SPACE Let s begin with some names and notation for things: R is the set (collection) of real numbers. We write x R to mean that x is a real number. A real number

### discuss how to describe points, lines and planes in 3 space.

Chapter 2 3 Space: lines and planes In this chapter we discuss how to describe points, lines and planes in 3 space. introduce the language of vectors. discuss various matters concerning the relative position

### Applications to Data Smoothing and Image Processing I

Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is

### x1 x 2 x 3 y 1 y 2 y 3 x 1 y 2 x 2 y 1 0.

Cross product 1 Chapter 7 Cross product We are getting ready to study integration in several variables. Until now we have been doing only differential calculus. One outcome of this study will be our ability

### 6 EXTENDING ALGEBRA. 6.0 Introduction. 6.1 The cubic equation. Objectives

6 EXTENDING ALGEBRA Chapter 6 Extending Algebra Objectives After studying this chapter you should understand techniques whereby equations of cubic degree and higher can be solved; be able to factorise

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

### TIME SERIES ANALYSIS

TIME SERIES ANALYSIS Ramasubramanian V. I.A.S.R.I., Library Avenue, New Delhi- 110 012 ram_stat@yahoo.co.in 1. Introduction A Time Series (TS) is a sequence of observations ordered in time. Mostly these

### Understanding Basic Calculus

Understanding Basic Calculus S.K. Chung Dedicated to all the people who have helped me in my life. i Preface This book is a revised and expanded version of the lecture notes for Basic Calculus and other

### Overview of Math Standards

Algebra 2 Welcome to math curriculum design maps for Manhattan- Ogden USD 383, striving to produce learners who are: Effective Communicators who clearly express ideas and effectively communicate with diverse

Chapter 7 Chapter Table of Contents OVERVIEW...193 GETTING STARTED...194 TheThreeStagesofARIMAModeling...194 IdentificationStage...194 Estimation and Diagnostic Checking Stage...... 200 Forecasting Stage...205

### 3.1 Stationary Processes and Mean Reversion

3. Univariate Time Series Models 3.1 Stationary Processes and Mean Reversion Definition 3.1: A time series y t, t = 1,..., T is called (covariance) stationary if (1) E[y t ] = µ, for all t Cov[y t, y t

### 1.5. Factorisation. Introduction. Prerequisites. Learning Outcomes. Learning Style

Factorisation 1.5 Introduction In Block 4 we showed the way in which brackets were removed from algebraic expressions. Factorisation, which can be considered as the reverse of this process, is dealt with

### Time series Forecasting using Holt-Winters Exponential Smoothing

Time series Forecasting using Holt-Winters Exponential Smoothing Prajakta S. Kalekar(04329008) Kanwal Rekhi School of Information Technology Under the guidance of Prof. Bernard December 6, 2004 Abstract

### Chapter 10 Introduction to Time Series Analysis

Chapter 1 Introduction to Time Series Analysis A time series is a collection of observations made sequentially in time. Examples are daily mortality counts, particulate air pollution measurements, and

### Probability and Statistics

CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2b - 0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be

### Chapter 2 Portfolio Management and the Capital Asset Pricing Model

Chapter 2 Portfolio Management and the Capital Asset Pricing Model In this chapter, we explore the issue of risk management in a portfolio of assets. The main issue is how to balance a portfolio, that

### 9th Russian Summer School in Information Retrieval Big Data Analytics with R

9th Russian Summer School in Information Retrieval Big Data Analytics with R Introduction to Time Series with R A. Karakitsiou A. Migdalas Industrial Logistics, ETS Institute Luleå University of Technology

### THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear

### Section 1.4. Lines, Planes, and Hyperplanes. The Calculus of Functions of Several Variables

The Calculus of Functions of Several Variables Section 1.4 Lines, Planes, Hyperplanes In this section we will add to our basic geometric understing of R n by studying lines planes. If we do this carefully,

### Common Core Unit Summary Grades 6 to 8

Common Core Unit Summary Grades 6 to 8 Grade 8: Unit 1: Congruence and Similarity- 8G1-8G5 rotations reflections and translations,( RRT=congruence) understand congruence of 2 d figures after RRT Dilations

### FURTHER VECTORS (MEI)

Mathematics Revision Guides Further Vectors (MEI) (column notation) Page of MK HOME TUITION Mathematics Revision Guides Level: AS / A Level - MEI OCR MEI: C FURTHER VECTORS (MEI) Version : Date: -9-7 Mathematics

### THREE DIMENSIONAL GEOMETRY

Chapter 8 THREE DIMENSIONAL GEOMETRY 8.1 Introduction In this chapter we present a vector algebra approach to three dimensional geometry. The aim is to present standard properties of lines and planes,

### NOV - 30211/II. 1. Let f(z) = sin z, z C. Then f(z) : 3. Let the sequence {a n } be given. (A) is bounded in the complex plane

Mathematical Sciences Paper II Time Allowed : 75 Minutes] [Maximum Marks : 100 Note : This Paper contains Fifty (50) multiple choice questions. Each question carries Two () marks. Attempt All questions.

### LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008 This chapter is available free to all individuals, on understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

### Introduction to time series analysis

Introduction to time series analysis Margherita Gerolimetto November 3, 2010 1 What is a time series? A time series is a collection of observations ordered following a parameter that for us is time. Examples

Academic Content Standards Grade Eight and Grade Nine Ohio Algebra 1 2008 Grade Eight STANDARDS Number, Number Sense and Operations Standard Number and Number Systems 1. Use scientific notation to express

### Geometry of Vectors. 1 Cartesian Coordinates. Carlo Tomasi

Geometry of Vectors Carlo Tomasi This note explores the geometric meaning of norm, inner product, orthogonality, and projection for vectors. For vectors in three-dimensional space, we also examine the

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Lecture 1: Univariate Time Series B41910: Autumn Quarter, 2008, by Mr. Ruey S. Tsay

Lecture 1: Univariate Time Series B41910: Autumn Quarter, 2008, by Mr. Ruey S. Tsay 1 Some Basic Concepts 1. Time Series: A sequence of random variables measuring certain quantity of interest over time.

### Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.

This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra

### Math 241, Exam 1 Information.

Math 241, Exam 1 Information. 9/24/12, LC 310, 11:15-12:05. Exam 1 will be based on: Sections 12.1-12.5, 14.1-14.3. The corresponding assigned homework problems (see http://www.math.sc.edu/ boylan/sccourses/241fa12/241.html)

### PITFALLS IN TIME SERIES ANALYSIS. Cliff Hurvich Stern School, NYU

PITFALLS IN TIME SERIES ANALYSIS Cliff Hurvich Stern School, NYU The t -Test If x 1,..., x n are independent and identically distributed with mean 0, and n is not too small, then t = x 0 s n has a standard

### Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

### National 5 Mathematics Course Assessment Specification (C747 75)

National 5 Mathematics Course Assessment Specification (C747 75) Valid from August 013 First edition: April 01 Revised: June 013, version 1.1 This specification may be reproduced in whole or in part for