FORECASTING AND TIME SERIES ANALYSIS USING THE SCA STATISTICAL SYSTEM

Transcription

1 FORECASTING AND TIME SERIES ANALYSIS USING THE SCA STATISTICAL SYSTEM VOLUME 2 Expert System Capabilities for Time Series Modeling Simultaneous Transfer Function Modeling Vector Modeling by Lon-Mu Liu in collaboration with George E. P. Box George C. Tiao This manual is published by Scientific Computing Associates Corp. 913 West Van Buren Street, Suite 3H Chicago, Illinois U.S.A. Copyright Scientific Computing Associates Corp.,

2

3 TABLE OF CONTENTS CHAPTER 1 CHAPTER 2 INTRODUCTION MODELING AND FORECASTING TIME SERIES USING SCA-EXPERT CAPABILITIES 2.1 Modeling and Forecasting a Univariate Time Series The univariate ARIMA Model Modeling and forecasting multi-variable time series Transfer Function Models LTF Method Differencings An Illustrative Example for Reduced Form Transfer Function Modeling An Illustrative Example for Structural Form Transfer Function Modeling Identification of Rational Transfer Function Models Appendix A Summary of the SCA Paragraphs in Chapter IARIMA, IESTIM Appendix B Identification of seasonal ARIMA models using a filtering method B.1 Introduction B.2 Basic Concepts of the Method and Its Rationale B.3 A Summary of the Method B.4 Example B.5 Discussion References in Chapter CHAPTER 3 MULTIVARIATE TIME SERIES ANALYSIS AND FORECASTING USING SIMULTANEOUS TRANSFER FUNCTION MODELS 3.1 Model Building Strategy for STF Models Identification of STF models Specification of an STF model Estimation of an STF model Diagnostic checking Forecasting future observations Further analysis of the example

4 3.2 Identification of STF Models Transfer function models The LTF method An illustrated example Multivariate Time Series Analysis with Interventions Econometric Modeling Using the STF Models Specification of endogenous variables Specification of definitional equations Examples for estimation of STF model (A) Kmenta's demand-supply model (B) Klein s U.S. economy Model I Model Simulation Summary of the SCA Paragraphs in Chapter CCM, STEPAR, MIDEN, ECCM, SCAN, MTSMODEL, MESTIM, IMESTIM, MFORECAST, and CANONICAL Paragraphs References in Chapter CHAPTER 4 MULTIVARIATE TIME SERIES ANALYSIS AND FORECASTING USING VECTOR ARMA MODELS 4.1 Implications of the Vector ARMA Model The vector MA(1) model The vector AR(1) model The vector ARMA(1,1) model A nonstationary vector model Relationship of vector ARMA models to transfer function models Relationship of vector models to econometric models Model building strategy for vector ARMA models A Simulated Example Sample cross correlation matrices Specification of a vector ARMA model Model estimation for the simulated MA(1) data Diagnostic checks of the fitted model Forecasting an estimated model Modeling an Autoregressive Process: Lydia Pinkham Data Preliminary identification: sample cross correlation matrices Preliminary identification, continued: stepwise Autoregressive fitting

5 4.3.3 Initial model specification and estimation for Lydia Pinkham data Estimation with constraints Interpreting the estimation results Analysis of a Mixed Vector ARMA Model Preliminary model identification, CCM and STEPAR Identification methods for a mixed model Model specification and estimation for the U.K. financial data Diagnostic checking and implication of the fitted model Modeling Seasonal Data: Census Housing Data Preliminary model identification Model specification and estimation Diagnostic checks of the estimated model Automatic Vector ARMA Estimation The simulated example in Section The Lydia Pinkham example in Section The U.K. financial data example in Section The Census housing data example in Section Summary of the SCA Paragraphs in Chapter CCM, STEPAR, MIDEN, ECCM, SCAN, MTSMODEL, MESTIM, IMESTIM, MFORECAST, and CANONICAL Paragraphs References in Chapter

6

7 CHAPTER 1 INTRODUCTION The Forecasting and Modeling Package of the SCA Statistical System is comprised of five products. These products are: UTS: Univariate time series analysis and forecasting using Box-Jenkins ARIMA, intervention and transfer function models. This product also includes forecasting capabilities using general exponential smoothing methods. Extended UTS: Univariate time series analysis and forecasting with automatic outlier detection and adjustment, as well as analysis and forecasting of time series containing missing data EXPERT: Automatic time series modeling using Box-Jenkins ARIMA, intervention, transfer function, and vector ARMA models. The automatic vector ARMA modeling component requires the SCA-MTS product. ECON/M: Econometric modeling, multivariate time series analysis, and forecasting using simultaneous transfer function (STF) models. This module also provides the seasonal adjustment procedures X-11, X-11-ARIMA, and a model-based canonical decomposition method. MTS: Multivariate time series analysis and forecasting using vector ARMA models The manual, Forecasting and Time Series Analysis Using the SCA Statistical System, describes the capabilities in the above products. This manual has two volumes. Volume 1 describes the capabilities of the SCA-UTS and Extended UTS products, and Volume 2 describes additional forecasting and time series analysis capabilities of the SCA Statistical System as of document s print date. Any new capabilities or new SCA products for time series analysis and forecasting will be documented as an addendum to these manuals or as a stand-alone monograph. Capabilities described in this volume include: Expert modeling capabilities: (Chapter 2) Simultaneous transfer function modeling: (Chapter 3) Vector ARMA modeling (Chapter 4) Automatic time series modeling using Box-Jenkins ARIMA, intervention, and transfer function models. Multivariate time series analysis and forecasting using STF models. It also discusses the use of STF models in econometric analysis. Multivariate time series analysis and forecasting using vector ARMA models.

8 This volume should be used in conjunction with companion manuals, Forecasting and Time Series Analysis Using the SCA Statistical System, Volume 1, and The SCA Statistical System: Reference Manual for Fundamental Capabilities. Within these companion manuals, information is provided on univariate time series analysis, and basic functionality of the SCA System. Whenever possible, material in this manual is presented in a data analysis form. That is, SCA System capabilities, commands, and output are usually presented within the context of a data analysis. Examples have been chosen to both demonstrate the use of the SCA System and to provide some broad guidelines for forecasting and time series analysis. This volume is an extension of the manual, Forecasting and Time Series Analysis Using the SCA Statistical System, Volume 1. It is highly recommended that both volumes be used as reference material when working with the time series analysis and forecasting capabilities of the SCA System. One key reference and source of examples in this manual is the text Time Series Analysis: Forecasting and Control by Box and Jenkins (1970). This text contains many important concepts and properties of forecasting and time series analysis. REFERENCES Box, G.E.P. and Jenkins, G.M. (1970). Time Series Analysis: Forecasting and Control. San Francisco: Holden-Day. (Revised edition published 1976).

9

10 CHAPTER 2 MODELING AND FORECASTING TIME SERIES USING SCA-EXPERT CAPABILITIES In forecasting and analysis of time series data, it is well demonstrated that autoregressive-integrated moving average (ARIMA), intervention, and transfer function models are very effective in handling practical applications. Vast advancements in both theory and methods in this area of research have been accomplished over the last two decades. Unfortunately these methods are not as widely used as they should, given the great advantage they offer. It seems that the complexity and often time consuming nature of the model building process imposed a barrier between the methodology and its use in main stream business and industrial applications. SCA has removed this barrier with the development of SCA-EXPERT. SCA-EXPERT employs new expert system technology to facilitate automatic ARIMA, intervention, and transfer function modeling. The SCA-EXPERT product is very easy to use, and is well suited to most forecasting and time series analysis applications. It is an asset to both novices and experts alike. The automatic nature of the modeling capability provides a mechanism for time series models to be adopted in business and industrial applications. It is a quick and effective solution to handle repetitive, or large scale modeling and forecasting problems. SCA- EXPERT can identify and estimate an appropriate time series model within seconds. In education, the SCA-EXPERT product allows the student to concentrate efforts on the interpretation and application of results, and less time on the complexity of model identification techniques. The SCA-EXPERT product is designed to be a self-contained product. It provides all SCA fundamental capabilities (please refer to the SCA document The SCA Statistical System: Reference Manual for Fundamental Capabilities for detailed information). In addition, it contains the following paragraphs for modeling and forecasting time series: IARIMA, IESTIM, TSMODEL, ESTIM, FORECAST, SFORECAST, OUTLIER, ACF, CCF, CORNER, REGRESSION, DAYS, EASTER, AGGREGATE, PERCENT, and PATCH. Except for IARIMA and IESTIM, the functionality of the above SCA paragraphs can be found in the SCA document Forecasting and Time Series Analysis Using the SCA Statistical System: Volume 1. The capabilities of the IARIMA and IESTIM paragraphs will be described in this document. Since outliers (abnormal or extreme observations) commonly occur in reallife time series, it is desirable to employ the SCA Extended UTS capabilities OESTIM and OFORECAST in conjunction with the SCA-EXPERT capabilities. Several examples of this interplay will be provided in this document. This document should be used in conjunction with other SCA reference manuals. For general functionality of the SCA System, the information can be found in The SCA Statistical System: Reference Manual for Fundamental Capabilities (Liu and Hudak 1991). For detailed information on modeling and forecasting using ARIMA, intervention, and transfer function models, please refer to Forecasting and Time Series Analysis Using the SCA Statistical

11 2.2 EXPERT MODELING AND FORECASTING TIME SERIES System: Volume 1 (Liu et. al. 1992). The latter text also contains information regarding outlier detection and adjustment in time series modeling. This document begins by first describing how to use SCA-EXPERT for univariate ARIMA modeling. We then describe the use of SCA-EXPERT in transfer function modeling. 2.1 Modeling And Forecasting A Univariate Time Series ARIMA models (Box and Jenkins 1970) are useful in many aspects of time series analysis. Such models can be used (1) to understand the nature of a time series, (2) to forecast future observations, and (3) to capture serial correlation in an intervention or a transfer function model. ARIMA models are simple in their model structure, yet they are quite effective in capturing the patterns of serial correlations and in forecasting the future observations of a time series. When forecasts are derived using a more complicated model (such as a multi-variable or non-linear time series model), they are often compared with those generated by an ARIMA model. If the forecasts generated under a more complicated time series model are less accurate than those under an ARIMA model, it often signifies misspecification in the more complicated model, or the existence of outliers in the series. The effects of outliers on forecasting performance can be found in Hillmer (1984), Ledolter (1989), and Chen and Liu (1993). It is highly recommended that the first step in any statistical modeling is to plot the data. In time series modeling, we can use the TSPLOT or TPLOT paragraph, or the time plot capability of SCAGRAF (see The SCA Graphics Package User's Guide) for this purpose. By viewing the plot, we can easily spot important characteristics of a time series, for example, nonstationarity of the series, presence of trend or seasonality, and the existence of major outliers or extreme values. Such information is not only useful for modeling, but also important for forecasting and other applications of the time series model. After examining the time series plot, the IARIMA paragraph can be employed to automatically identify and estimate an ARIMA model for the series. Since the IARIMA paragraph typically employs the conditional likelihood algorithm for model estimation, the SCA estimation paragraph ESTIM (or OESTIM in the Extended UTS) can be used to obtain more efficient parameter estimates if needed. With an appropriately estimated model, the forecasts of the series can be generated using the FORECAST (or OFORECAST in the Extended UTS) paragraph. In this process, if a user's primary interest is forecasting, it is possible to ignore model information and simply just generate the forecasts based on the automatically identified and estimated model. Generally speaking, however, it is advisable to have some basic knowledge in terms of ARIMA models in order to have a better understanding of the generated forecasts.

12 EXPERT MODELING AND FORECASTING TIME SERIES 2.3 Some useful descriptions of ARIMA models are presented in Section 2, where we also present some key ideas used in the identification techniques employed in SCA-EXPERT. In the remainder of this section, we shall illustrate the modeling and forecasting procedure outlined above by using the logged variety stores sales data discussed in Hillmer, Bell and Tiao (1983). Figure 1. Log Retail Sales of Variety Stores: 1/1967 9/1979 The time series plot of the monthly logged variety stores sales data between January 1967 and September 1979 is displayed in Figure 1. In this plot, we observe that the series has strong seasonality and an upward trend. A downward level shift beginning in mid 1976 (between t=108 and t=120) is also visible. After reading the time series data into the SCA workspace and storing the log transformed data as LVSALES, we can obtain an ARIMA model for the series by entering the statement -->IARIMA LVSALES. SEASONALITY IS 12.

13 2.4 EXPERT MODELING AND FORECASTING TIME SERIES THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 153 THE CRITICAL VALUE FOR SIGNIFICANCE TESTS OF ACF AND ESTIMATES IS SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- UTSMODEL VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 12 1 LVSALES RANDOM ORIGINAL (1-B ) (1-B ) PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 LVSALES MA 1 12 NONE LVSALES D-AR 1 1 NONE LVSALES D-AR 1 2 NONE TOTAL NUMBER OF OBSERVATIONS EFFECTIVE NUMBER OF OBSERVATIONS RESIDUAL STANDARD ERROR E In the above IARIMA paragraph, LVSALES is the name of the series, and the SEASONALITY sentence is used to specify the potential seasonality in the series (which is 12 for monthly data). The identified model, which is the same as that presented in Hillmer, Bell, and Tiao (1983), can be expressed as or Θ1B t 2 1 1B 2B (1 B ) (1 B)VSALES =, (1) φ φ 12 1 Θ1B 12 t 2 t 1 1B 2B VSALES = a. (2) φ φ In the latter expression, we use 12 to represent (1 B ) and to represent (1 B). It is important to note that we place the AR(2) operator in the denominator of the above ARIMA model (hence they are referred as D-AR in the model display of the IARIMA output). Such an expression is somewhat different from the conventional form employed in Box and Jenkins (1970) and others. The rationale is presented in the next section. The model information generated by the IARIMA paragraph is stored under the model name UTSMODEL by default. To obtain the exact maximum likelihood estimates for the above model, we enter -->ESTIM UTSMODEL. METHOD IS EXACT. 12

14 EXPERT MODELING AND FORECASTING TIME SERIES 2.5 THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 153 NONLINEAR ESTIMATION TERMINATED DUE TO: MAXIMUM NUMBER OF ITERATIONS 10 REACHED SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- UTSMODEL VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 12 1 LVSALES RANDOM ORIGINAL (1-B ) (1-B ) PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 LVSALES MA 1 12 NONE LVSALES D-AR 1 1 NONE LVSALES D-AR 1 2 NONE EFFECTIVE NUMBER OF OBSERVATIONS R-SQUARE RESIDUAL STANDARD ERROR E We notice that the estimates under the EXACT method are somewhat different from those under the IARIMA paragraph (where a conditional likelihood method is employed). The difference is more pronounced for the seasonal MA parameters. The exact estimation is recommended if an ARIMA model contains MA parameters, particularly seasonal MA parameters. If a model only contains AR parameters, then there is no need to perform exact maximum likelihood estimation. To obtain forecasts for the series, we enter -->FORECAST UTSMODEL FORECASTS, BEGINNING AT TIME FORECAST STD. ERROR ACTUAL IF KNOWN

15 2.6 EXPERT MODELING AND FORECASTING TIME SERIES The above forecasts seem to follow the pattern of the original series very well. As shown in Figure 1, the variety stores series contains major outliers. To account for the effects of outliers, we may use the OESTIM paragraph in the Extended UTS product to perform joint estimation of model parameters and outlier effects. Here, we enter -->OESTIM UTSMODEL. METHOD IS EXACT. NEW-SERIES ARE ADJR, ADJY. THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 153 SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- UTSMODEL VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 12 1 LVSALES RANDOM ORIGINAL (1-B ) (1-B ) PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 LVSALES MA 1 12 NONE LVSALES D-AR 1 1 NONE LVSALES D-AR 1 2 NONE SUMMARY OF OUTLIER DETECTION AND ADJUSTMENT TIME ESTIMATE T-VALUE TYPE TC AO LS TOTAL NUMBER OF OBSERVATIONS EFFECTIVE NUMBER OF OBSERVATIONS RESIDUAL STANDARD ERROR (WITHOUT OUTLIER ADJUSTMENT) E-01 RESIDUAL STANDARD ERROR (WITH OUTLIER ADJUSTMENT) E In this example, three outliers are identified. At time period 45, a temporary change occurs. At time period 96, an additive outlier is found. At time period 112, a negative level shift occurs in the series. The last outlier, which occurred in April 1976, corresponds to the closing of a major variety store chain, W.T. Grant. As a result, a significant proportion of retail sales previously made at variety stores were shifted to department stores. The parameter estimates obtained by the OESTIM paragraph are different from those obtained under the ESTIM paragraph, since the model parameters are jointly estimated with the outlier effects in the

16 EXPERT MODELING AND FORECASTING TIME SERIES 2.7 OESTIM paragraph. A display of outliers, adjusted series, and the original series is shown in Figure 2. Figure 2. Outliers and Adjusted Series of Log Retail Sales Data To obtain forecasts with outlier effects accounted, we enter -->OFORECAST UTSMODEL. RESIDUAL STANDARD ERROR (USES DATA UP TO THE FIRST FORECAST ORIGIN)= E-01 TIME ESTIMATE T-VALUE TYPE TC AO LS FORECASTS, BEGINNING AT TIME FORECAST STD. ERROR ACTUAL IF KNOWN

17 2.8 EXPERT MODELING AND FORECASTING TIME SERIES The forecasts generated by the OFORECAST paragraph are not much different from those generated under the FORECAST paragraph. This is not surprising since the forecast origin is far away from the last outlier. More detailed discussions regarding the effects of outliers on forecasts can be found in Chen and Liu (1993). 2.2 The Univariate ARIMA Model Following Box and Jenkins (1970), the characteristics of a univariate time series typically can be well represented by a relatively simple ARIMA model. Using the backshift operator B (where BZt = Zt 1), a non-seasonal ARIMA model traditionally is expressed as d φ(b)(1 B) Z t = C 0 +θ (B)a t, t=1,2,...,n (3) where { Z t } is a time series with n observations, { a t } is a sequence of random errors that are 2 independently and identically distributed with a normal distribution N(0, σ a ), C 0 is a constant term, and d is the number of differencings. The φ (B) and θ (B) operators are polynomials in B where 2 p 1 2 p φ (B) = (1 φ B φ B φ B ), and 2 q 1 2 q θ (B) = (1 θ B θ B θ B ). The value p denotes the order of the autoregressive (AR) operator φ (B), and q denotes the order of the moving average (MA) operator θ (B). In most practical situations, p and q are small values no greater than 3, and d is 0, 1, or at most 2. Double differencing (i.e., d=2) seldom occurs in real-life time series applications. The model in (3) can also be expressed as θ(b) (1 B) Z C a (B) d t = + φ t (4) with C = C 0/(1 φ1 φ2... φ p). The latter representation is more preferable since the term C can be easily interpreted. When d=0 (i.e., the series requires no differencing), the constant term C represents the mean of the series. An ARIMA model in such a case is also referred to as an autoregressive-moving average (ARMA) model. When d=1 (i.e., the series requires a first-order differencing), the constant term represents the trend of the series (i.e., the increase or decrease between two successive observations). When d=2 (i.e., the series requires double differencing), the constant term represents the second order trend (i.e., the

18 EXPERT MODELING AND FORECASTING TIME SERIES 2.9 trend of the trend), which seldom occurs in real-life application. Unlike the representation in (4), the term C0 in (3) does not have an easy-to-understand interpretation. Due to these reasons, all ARIMA models identified by SCA-EXPERT are expressed in the form of (4). A special case of (4) is the mixed ARMA(1,1) model, which can be expressed as 1 θ1b Zt = C+ a 1 φ B 1 t (5) SCA-EXPERT uses the parameter estimates of the above ARMA(1,1) model, and the sample ACF and PACF of the series to automatically identify the number of differencings required, and the initial orders for the AR and MA polynomials. Seasonal ARIMA models The models in (3) and (4) can be extended to represent a seasonal time series. The general form of a multiplicative seasonal ARIMA model can be expressed as or s d s D s t φ(b) Φ(B )(1 B) (1 B ) Z = C +θ(b) Θ (B )a, t = 1,2,...,n (6) s d s D θ(b) Θ(B ) (1 B) (1 B ) Z t = C + a s t, t=1,2,...,n (7) φ (B) Φ (B ) t where s s 2s Ps 1 2 p Φ (B ) = (1 Φ B Φ B Φ B ), and s s 2s Qs 1 2 p Θ (B ) = (1 Θ B Θ B Θ B ). In most practical applications, the values of D, P, and Q are either 0 or 1. A special case of model (7) is the mixed ARMA(1,1)xARMA(1,1) s model, which can be expressed as 1 1 s (1 θ1b) (1 Θ1B ) Zt = C+ a s t. (8) (1 φ B) (1 Φ B ) SCA-EXPERT uses the parameter estimates of the above model, and the sample ACF and PACF for the filtered series of model (8) to automatically identify the differencings required for the final model, and the initial orders for the seasonal and non-seasonal AR and MA polynomials. More details regarding the theory and techniques for the identification of seasonal ARIMA models can be found in Liu (1989). A revised version of Liu (1989) is included in this document as Appendix B.

19 2.10 EXPERT MODELING AND FORECASTING TIME SERIES The determination of appropriate differencing order(s) for a seasonal time series can be difficult in some situations. We found that a number of published models in fact are overdifferenced. SCA-EXPERT employs an effective technique to determine appropriate differencing orders and avoid over-differencing. 2.3 Modeling and Forecasting Multi-Variable Time Series An effective means for modeling and forecasting multi-variable time series is to employ transfer function models. Transfer function models (Box and Jenkins 1970) can be regarded as extensions of classical regression and econometric models, and are useful in many applications. In this document, we describe two application areas: (1) for forecasting, and (2) for understanding and interpretation of inter-relationships among variables in a system. The latter application is also known as structural analysis. Traditionally structural form models (i.e., models that allow for contemporaneous relationships between the input variables and the output variable) are used for both forecasting and structural analysis. A number of difficulties in the identification (specification) and estimation of structural form models have been extensively discussed in econometric literature. Liu (1991) suggested that if the primary interest of transfer function modeling is forecasting, a reduced form model may be more preferable than a structural form model. A reduced form model does not allow for contemporaneous relationships in a model unless an input variable is certain to be exogenous. Due to this restriction, the use of a reduced form model avoids a number of difficulties that commonly occur in structural form modeling. Both reduced form and structural form models may consist of a system of equations. Using the approach to be discussed in this document, we can deal with the identification and estimation of the model in the system equation by equation. This flexibility allows us to focus on a partial system (e.g., just one or a few equations) depending upon our interest and application. Joint model estimation for a system of transfer function equations is discussed in Wall (1976), Liu and Hudak (1985), and Liu et. al. (1986) Transfer Function Models A transfer function model can be a single-equation model or a multi-equation model. The latter is also referred to as a simultaneous transfer function (STF) model (Wall 1976, Liu et. al. 1986, Liu 1987, and Liu 1991). For notational simplicity, we will consider a transfer function model with two variables, Y t and X t, where Y t and X t may be inter-related and both can be endogenous variables. Assuming both Y t and X t are stationary, the general form of a transfer function model can be expressed as or Yt = C +ω (B)Xt + N t, t=1,2,...,n (9) ω(b) Yt = C + Xt + N t, t=1,2,...,n δ(b) (10)

20 EXPERT MODELING AND FORECASTING TIME SERIES 2.11 where C is a constant term, and N t is the disturbance term which may follow a stationary ARMA process described in model (4) or (7). The polynomial ω (B) and δ (B) can be generally expressed as 2 g g ω (B) =ω +ω B +ω B ω B, and (11) 1 r r δ (B) = 1 δ B... δ B. (12) We refer to the model in (9) as a linear transfer function (LTF) model (which implies δ (B) = 1), and the model in (10) as a rational transfer function model (which implies δ(b) 1). It is also important to note that the parameter ω 0 in (11) is constrained to be zero (i.e., cannot be present in the ω (B) polynomial) if the model is in reduced form. Similar to (9) and (10), we may consider a transfer function model for Y, which may be expressed as or t X t dependent on Xt = C' +ω '(B)Yt + N ' t, t=1,2,...,n (13) ω'(b) Xt = C' + Yt + N ' t, t=1,2,...,n. δ'(b) (14) Joint estimation for a system of transfer function equations using the maximum likelihood method was first addressed in Wall (1976) and implemented in the SCA System (Liu et. al. 1986). Identification of transfer function equations will be discussed next LTF Method An effective way to identify a transfer function equation is to employ the linear transfer function (LTF) method. The LTF method follows an approach proposed by Liu and Hanssens (1982) and is detailed in Liu and Hudak (1985), Liu (1986, 1987), Pankratz (1991), and particularly in Liu et. al. (1992). This method is effective for both non-seasonal and seasonal time series, and for both reduced form and structural form models. Furthermore, it is easy to use, flexible, and easy to understand. More information regarding the LTF method can be found in Chapter 9, Forecasting and Time Series Analysis Using the SCA Statistical System: Volume 1 (Liu et. al. 1992). Consider the transfer function equations described in (9) through (12). The LTF method employs the following linear transfer function model for the identification of a structural form equation: 2 k t k t t Y = C + (v + v B + v B v B )X + N (15) where k is a lag order for X t chosen by the user based on the subject matter, and N t is the disturbance term (to be discussed later). For the identification of a reduced form equation, the following model structure is employed:

21 2.12 EXPERT MODELING AND FORECASTING TIME SERIES 2 k t 1 2 k t t Y = C + (v B + v B v B )X + N (16) The key difference between (15) and (16) is that (16) imposes the exclusion of a potential contemporaneous relationship between X t and Y t, and (15) allows for a potential contemporaneous relationship between X t and Y. t If X t is an exogeneous variable, the model structure used in (15) or (16) typically leads to the same model if X t and Y t does not have a contemporaneous relationship. This is particularly true if the exogenous variable is a pre-determined non-stochastic time series. However, if X t is also an endogenous variable, the transfer function weight estimates based on (15) can be seriously biased, and render rather misleading results even if X t and Y t are not contemporaneously related. Therefore, it is preferable to employ a reduced form model unless exogeneity of an input variable is definite. The disturbance term in (15) and (16) can be effectively approximated by simple autoregressive models for the purpose of model identification. If the output variable is nonseasonal, the disturbance term may be approximated by 1 Nt = a t. 1 φ B 1 (17) In the case of a seasonal output variable (with seasonality s), an initial approximation for the disturbance term may be N t 1 = a s (1 φ B)(1 Φ B ) 1 1 t The combined use of a linear transfer function with an autoregressive disturbance term provides some unique advantages. These include: (18) (1) Obtaining efficient estimates of transfer function weights. Based on these weights, we can determine if a linear or a rational transfer function is needed for the model equation. (2) Obtaining an estimated disturbance series ˆN t. The model for identified by the IARIMA paragraph. ˆN t can then be easily (3) Providing information on differencing. If either φ 1 or Φ 1 in (17) and (18) is close to 1, then it is definitely necessary to perform appropriate differencing(s) on all variables in the model. This topic will be further discussed in the next subsection Differencings The models we discussed assume that both Y t and X t are stationary. In most real-life applications, Y t and X t may not follow this assumption. Similar to ARIMA modeling, the determination of differencing orders is a key aspect in transfer function modeling. We may examine the value of φ 1 or Φ 1 in the autoregressive term (to see if they are close to 1) to s determine if a regular (i.e., (1 B) ) and/or a seasonal (i.e., (1 B ) ) differencing is necessary. However it is important to note that the estimates φ 1 or Φ 1 can be seriously biased if a non-

22 EXPERT MODELING AND FORECASTING TIME SERIES 2.13 seasonal or seasonal MA parameter is required in the disturbance term. This is particularly serious for the seasonal AR(1) estimate. An easy way to overcome this difficulty is to employ the IARIMA paragraph to automatically identify an ARIMA model for the estimated disturbance series. If the identified model for the disturbance series requires differencing or if the model contains a non-seasonal or seasonal AR(1) polynomial with the parameter estimate(s) close to 1, then appropriate differencing(s) is necessary. To illustrate this method of differencing determination, we employ a set of data consisting of monthly shipments and new orders of durable goods in the United States between January 1958 and December 1974 (Hiller 1976, and Liu 1987). These series are displayed in Figure 3. Our primary interest in the analysis is to develop a reduced form model that employs information in both series for forecasting. Following the analysis in Liu (1987), we shall only employ the data between January 1958 and December 1972 (i.e., the first 180 observations) for model building, the last 24 observations can be used for post-sample forecasting comparision (see Liu 1987). Both series are log transformed, and stored in the SCA workspace as SHIPMENT and NEWORDER respectively. Figure 3. U.S. Durable Goods Shipments and New Orders (a) Durable goods shipments

23 2.14 EXPERT MODELING AND FORECASTING TIME SERIES (b) Durable goods new orders Our first step is to develop a transfer function equation for the variable SHIPMENT (with NEWORDER as an input variable). Following the models presented in (16) and (18), we consider a linear transfer function model 1 Y = C + (v B + v B v B )X + a. (19) φ Φ 2 6 t t 12 t (1 1B)(1 1B ) for the determination of differencing orders (and subsequently for the identification of the model equation). The above LTF model can be specified and estimated using the following TSMODEL and ESTIM paragraphs. -->TSMODEL NAME IS EQ1. NO --> MODEL IS SHIPMENT=C1+(1 TO 6)NEWORDER+1/(1)(12)NOISE. -->ESTIM EQ1. HOLD DISTURBANCE(NS). OUTPUT LEVEL(BRIEF). THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 180 NONLINEAR ESTIMATION TERMINATED DUE TO: RELATIVE CHANGE IN (OBJECTIVE FUNCTION)**0.5 LESS THAN D-03 SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- EQ VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED SHIPMENT RANDOM ORIGINAL NONE NEWORDER RANDOM ORIGINAL NONE

24 EXPERT MODELING AND FORECASTING TIME SERIES 2.15 PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 C1 CNST 1 0 NONE NEWORDER NUM. 1 1 NONE NEWORDER NUM. 1 2 NONE NEWORDER NUM. 1 3 NONE NEWORDER NUM. 1 4 NONE NEWORDER NUM. 1 5 NONE NEWORDER NUM. 1 6 NONE SHIPMENT D-AR 1 1 NONE SHIPMENT D-AR 2 12 NONE EFFECTIVE NUMBER OF OBSERVATIONS R-SQUARE RESIDUAL STANDARD ERROR E Reviewing the above estimation results, we find that the seasonal AR(1) parameter estimate is close to 1 ( Φ 1 =.9121), suggesting that a seasonal differencing is required for the model. Since we stored the estimated disturbance series (N ˆ t ) in the variable NS, we can identify a model for the disturbance series using the following IARIMA paragraph: -->IARIMA NS. SEASONALITY IS 12. THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 180 THE CRITICAL VALUE FOR SIGNIFICANCE TESTS OF ACF AND ESTIMATES IS SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- UTSMODEL VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 12 NS RANDOM ORIGINAL (1-B ) PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 CNST 1 0 NONE NS MA 1 12 NONE NS D-AR 1 1 NONE TOTAL NUMBER OF OBSERVATIONS EFFECTIVE NUMBER OF OBSERVATIONS RESIDUAL STANDARD ERROR E Based on the above results, it confirms that the transfer function equation for SHIPMENT requires a seasonal differencing, but a regular differencing does not seem to be necessary. Using a model similar to (19), we can determine the differencing order(s) required for the model equation for NEWORDER. The LTF model specification and its subsequent estimation results are listed below:

25 2.16 EXPERT MODELING AND FORECASTING TIME SERIES -->TSMODEL NAME IS EQ2. NO --> MODEL IS NEWORDER=C2+(1 TO 6)SHIPMENT+1/(1)(12)NOISE. -->ESTIM EQ2. HOLD DISTURBANCE(NS). OUTPUT LEVEL(BRIEF). THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 180 NONLINEAR ESTIMATION TERMINATED DUE TO: RELATIVE CHANGE IN (OBJECTIVE FUNCTION)**0.5 LESS THAN D-03 SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- EQ VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED NEWORDER RANDOM ORIGINAL NONE SHIPMENT RANDOM ORIGINAL NONE PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 C2 CNST 1 0 NONE SHIPMENT NUM. 1 1 NONE SHIPMENT NUM. 1 2 NONE SHIPMENT NUM. 1 3 NONE SHIPMENT NUM. 1 4 NONE SHIPMENT NUM. 1 5 NONE SHIPMENT NUM. 1 6 NONE NEWORDER D-AR 1 1 NONE NEWORDER D-AR 2 12 NONE EFFECTIVE NUMBER OF OBSERVATIONS R-SQUARE RESIDUAL STANDARD ERROR E Since the non-seasonal AR(1) estimate is close to 1 ( φ =.9888), it suggests that regular differencing is necessary. However, it is not clear whether seasonal differencing is required since the seasonal AR(1) estimate is only The IARIMA paragraph (as shown below) provides valuable information in determining differencing orders. -->IARIMA NS. SEASONALITY IS 12. THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 180 THE CRITICAL VALUE FOR SIGNIFICANCE TESTS OF ACF AND ESTIMATES IS SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- UTSMODEL VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 1 NS RANDOM ORIGINAL (1-B )

26 EXPERT MODELING AND FORECASTING TIME SERIES 2.17 PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 NS MA 1 1 NONE NS MA 2 12 NONE NS D-AR 1 12 NONE TOTAL NUMBER OF OBSERVATIONS EFFECTIVE NUMBER OF OBSERVATIONS RESIDUAL STANDARD ERROR E Based on the above results, it is advisable to consider seasonal differencing since Φ 1 = is rather close to 1. In some situations, it is possible that both Y t and X t are nonstationary (or seasonal), but φ1(or Φ 1) in (17) or (18) is not close to 1. In such a circumstance, if we believe that the nonstationarity (or seasonality) of Y t is solely caused by X t, then we should not include differencing in the model. Otherwise, appropriate differencing(s) should be imposed. Generally speaking, if both Y t and X t are seasonal, it is safe to impose a seasonal differencing in a transfer function model. This is due to the fact that seasonality in most applications (e.g., business, economic or environmental studies) is caused by some common factors or common environments, rather than just by the particular input variable(s) in the model. This is also generally true for nonstationarity. However, in some situations it is possible that the nonstationarity of the output variable is caused by the input variable(s). In such a situation, the first-order differencing should not be imposed. The results using differencing and no differencing can be quite different.

27 2.18 EXPERT MODELING AND FORECASTING TIME SERIES An Illustrative Example for Reduced Form Transfer Function Modeling In this section, we continue using the example presented in Section 3.3 to illustrate key aspects in transfer function modeling using SCA-EXPERT. Based on the analysis presented in Section 3.3, we find that a seasonal differencing is necessary for the SHIPMENT model. The remaining SCA paragraphs we use to automatically identify a transfer function model for SHIPMENT are listed below. TSMODEL NAME IS EQ1. NO MODEL IS SHIPMENT(12)=C1+(1 TO 6)NEWORDER(12)+1/(1)(12)NOISE. IESTIM EQ1. PRESERVE ARMA. HOLD DISTURBANCE(NS). IARIMA NS. SEASONALITY IS 12. REPLACE EQ1. ESTIM EQ1. METHOD IS EXACT. HOLD OUTPUT LEVEL(BRIEF). In the above SCA statements, the TSMODEL paragraph specifies the differencing, the multiplicative AR(1) disturbance, and most importantly the initial lags for the linear transfer function. The IESTIM paragraph estimates the parameters in the specified model, and automatically deletes insignificant transfer function weight estimates from the model following a prudent algorithm. This algorithm first deletes insignificant weight estimates from both ends. Insignificant weight estimates in the middle are not deleted in the first pass, but will be deleted in the later iterations. The insignificant constant term is always deleted in the final iteration. The sentence PRESERVE ARMA ' requests that parameter estimates in the ARMA component not to be deleted, even if they are insignificant. The estimated disturbance series is stored in the variable NS. The IARIMA paragraph is then used to identify an ARMA model for the estimated disturbance series. By specifying the sentence REPLACE EQ1", the identified ARMA model automatically replaces the intermediate AR disturbance term in the transfer function model EQ1. If the transfer function model is in linear form (i.e., with δ (B) = 1), then we have obtained a final model after the execution of the above SCA statements. The model can be more accurately estimated using the ESTIM paragraph. For seasonal time series, it is recommended that EXACT maximum likelihood estimation is used. In this example, we shall use the default CONDITIONAL method. The residual series is stored in the variable RES in this example, and can be used for diagnostic checking. The output for the above SCA paragraphs are listed below: -->TSMODEL NAME IS EQ1. NO --> MODEL IS SHIPMENT(12)=C1+(1 TO 6)NEWORDER(12)+1/(1)(12)NOISE. -->IESTIM EQ1. PRESERVE ARMA. HOLD DISTURBANCE(NS). THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 180 SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- EQ VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 12 SHIPMENT RANDOM ORIGINAL (1-B ) 12 NEWORDER RANDOM ORIGINAL (1-B )

28 EXPERT MODELING AND FORECASTING TIME SERIES 2.19 PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 C1 CNST 1 0 NONE NEWORDER NUM. 1 1 NONE NEWORDER NUM. 1 2 NONE NEWORDER NUM. 1 3 NONE NEWORDER NUM. 1 4 NONE NEWORDER NUM. 1 5 NONE NEWORDER NUM. 1 6 NONE SHIPMENT D-AR 1 1 NONE SHIPMENT D-AR 2 12 NONE TOTAL NUMBER OF OBSERVATIONS EFFECTIVE NUMBER OF OBSERVATIONS RESIDUAL STANDARD ERROR (WITHOUT OUTLIER ADJUSTMENT) E-01 ================================================= RESULTS OF THE REVISED MODEL: REVISION NUMBER 1 ================================================= SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- EQ VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 12 SHIPMENT RANDOM ORIGINAL (1-B ) 12 NEWORDER RANDOM ORIGINAL (1-B ) PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 C1 CNST 1 0 NONE NEWORDER NUM. 1 1 NONE NEWORDER NUM. 1 2 NONE NEWORDER NUM. 1 3 NONE SHIPMENT D-AR 1 1 NONE SHIPMENT D-AR 2 12 NONE TOTAL NUMBER OF OBSERVATIONS EFFECTIVE NUMBER OF OBSERVATIONS RESIDUAL STANDARD ERROR (WITHOUT OUTLIER ADJUSTMENT) E >IARIMA NS. SEASONALITY IS 12. REPLACE EQ1. THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 180 THE CRITICAL VALUE FOR SIGNIFICANCE TESTS OF ACF AND ESTIMATES IS SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- UTSMODEL VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED NS RANDOM ORIGINAL NONE PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 NS MA 1 12 NONE NS D-AR 1 1 NONE

29 2.20 EXPERT MODELING AND FORECASTING TIME SERIES TOTAL NUMBER OF OBSERVATIONS EFFECTIVE NUMBER OF OBSERVATIONS RESIDUAL STANDARD ERROR E >ESTIM EQ1. HOLD RESIDUALS(RES). OUTPUT LEVEL(BRIEF). THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 180 NONLINEAR ESTIMATION TERMINATED DUE TO: RELATIVE CHANGE IN (OBJECTIVE FUNCTION)**0.5 LESS THAN.1000D-02 SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- EQ VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 12 SHIPMENT RANDOM ORIGINAL (1-B ) 12 NEWORDER RANDOM ORIGINAL (1-B ) PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 C1 CNST 1 0 NONE NEWORDER NUM. 1 1 NONE NEWORDER NUM. 1 2 NONE NEWORDER NUM. 1 3 NONE SHIPMENT MA 1 12 NONE SHIPMENT D-AR 1 1 NONE EFFECTIVE NUMBER OF OBSERVATIONS R-SQUARE RESIDUAL STANDARD ERROR E Thus the final transfer function model for SHIPMENT is Θ1B t t 1 φ1b SHIPMENT = C + ( ω B +ω B +ωb ) NEWORDER + a. Similarly, following the analysis in Section 3.3, we find that regular and seasonal 12 differencing (1 B)(1 B ) is necessary for the NEWORDER model. The SCA paragraphs for automatic identification of a transfer function model for NEWORDER is listed below: TSMODEL NAME IS EQ2. NO MODEL IS NEWORDER(1,12)=C2+(1 TO 6)SHIPMENT(1,12)+1/(1)(12)NOISE. IESTIM EQ2. PRESERVE ARMA. HOLD DISTURBANCE(NS). IARIMA NS. SEASONALITY IS 12. REPLACE EQ2. ESTIM EQ2. METHOD IS EXACT. HOLD OUTPUT LEVEL(BRIEF). The output for the above SCA paragraphs is listed below: (20)

30 EXPERT MODELING AND FORECASTING TIME SERIES >TSMODEL NAME IS EQ2. NO --> MODEL IS NEWORDER(1,12)=C2+(1 TO 6)SHIPMENT(1,12)+1/(1)(12)NOISE. -->IESTIM EQ2. PRESERVE ARMA. HOLD DISTURBANCE(NS). THE FOLLOWING ANALYSIS IS BASED ON TIME SPAN 1 THRU 180 SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- EQ VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 1 12 NEWORDER RANDOM ORIGINAL (1-B ) (1-B ) 1 12 SHIPMENT RANDOM ORIGINAL (1-B ) (1-B ) PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 C2 CNST 1 0 NONE SHIPMENT NUM. 1 1 NONE SHIPMENT NUM. 1 2 NONE SHIPMENT NUM. 1 3 NONE SHIPMENT NUM. 1 4 NONE SHIPMENT NUM. 1 5 NONE SHIPMENT NUM. 1 6 NONE NEWORDER D-AR 1 1 NONE NEWORDER D-AR 2 12 NONE TOTAL NUMBER OF OBSERVATIONS EFFECTIVE NUMBER OF OBSERVATIONS RESIDUAL STANDARD ERROR (WITHOUT OUTLIER ADJUSTMENT) E-01 ================================================= RESULTS OF THE REVISED MODEL: REVISION NUMBER 1 ================================================= SUMMARY FOR UNIVARIATE TIME SERIES MODEL -- EQ VARIABLE TYPE OF ORIGINAL DIFFERENCING VARIABLE OR CENTERED 1 12 NEWORDER RANDOM ORIGINAL (1-B ) (1-B ) 1 12 SHIPMENT RANDOM ORIGINAL (1-B ) (1-B ) PARAMETER VARIABLE NUM./ FACTOR ORDER CONS- VALUE STD T LABEL NAME DENOM. TRAINT ERROR VALUE 1 C2 CNST 1 0 NONE SHIPMENT NUM. 1 2 NONE NEWORDER D-AR 1 1 NONE NEWORDER D-AR 2 12 NONE TOTAL NUMBER OF OBSERVATIONS EFFECTIVE NUMBER OF OBSERVATIONS RESIDUAL STANDARD ERROR (WITHOUT OUTLIER ADJUSTMENT) E-01 ================================================= RESULTS OF THE REVISED MODEL: REVISION NUMBER 2 =================================================