A Comparative Study of Parametric and Nonparametric Regressions

Transcription

1 Iranian Econoic Review, Vol.16, No.3, Fall 11 A Coparative Study of Paraetric and Nonparaetric Regressions Shahra Fattahi Received: 1/8/8 Accepted: 11/4/4 Abstract his paper evaluates inflation forecasts ade by paraetric and Tnonparaetric odels. The results revealed that the neural network odel yields better estiates of inflation rate than do paraetric autoregressive integrated oving average (ARIMA) and linear odels. Furtherore, the neural network odel outperfored nonparaetric odels (except MARS). Keywords: ARIMA, AM, MARS, PPR, NN, Inflation Forecast. 1- Introduction The paraetric regression approach is based on the prior knowledge of the functional for relationship. If the knowledge is correct, the paraetric ethod can odel ost data sets well. However, if the wrong functional for is chosen a priori, this will result in larger bias as copared to copetitive odels (Fan and Yao, 3). Paraetric linear odels, as a type of paraetric regression, are frequently used to describe the association between the dependent variable and explanatory variables. They require the estiation of a finite nuber of paraeters. Furtherore, paraetric linear dynaic odels which are based on a theoretical or data-driven approach will be eployed. M.A. Student in Departent of Econoics, Razi University.

2 / A Coparative Study of Paraetric and Nonparaetric Regressions Why is nonparaetric regression iportant? Over the last decade, increasing attention has been devoted to nonparaetric regression as a new technique for estiation and forecasting in different sciences including econoics. Nonparaetric regression analysis relaxes the assuption of the linearity in the regression analysis and enables one to explore the data ore flexibly. However, in high diensions variance of the estiates rapidly increases, called as curse of diensionality, due to the sparseness of data. To overcoe this proble, soe nonparaetric ethods have been proposed such as additive odel (AM), ultiple adaptive regression splines (MARS), proection-pursuit regression (PPR) and neural networks (NN). This study copares paraetric and nonparaetric ethods. The agents are assued to use an optial paraetric autoregressive oving average (ARMA) odel or nonparaetric odels including Additive odel (AD), Multiple Adaptive Regression Splines (MARS), Proection-Pursuit Regression (PPR), and Neural Networks (NN) for forecasting. In fact, outof-saple estiates of inflation generated by the paraetric and nonparaetric odels will be copared. The paper is divided into four sections. Following introduction in section 1 we discuss paraetric and nonparaetric ethods in section. Section 3 reports the epirical results of this study. Finally, section 4 concludes. - Statistical predictors -1- Paraetric prediction odels A brief review of ARIMA odeling is presented (Chatfield, ). Autoregressive integrated oving average (ARIMA) or (Box-Jenkins) odels are the basis of any fundaental ideas in tie-series analysis. In order to analyze a tie series, it ust be assued that the structure of the stochastic process which generates the observations is essentially invariant through tie. The iportant assuption is that of stationarity, which requires the process to be in a particular state of statistical equilibriu (Box and Jenkins, 1976). An autoregressive oving average process: ARMA (p,q) is obtained by cobining p autoregressive ters and q oving average ters and can be written as

3 Fattahi, SH. /1 φ ( L ) X = α + θ ( L ) ε t t p φ ( L ) = 1 φ1l φ L... φ p L with AR polynoial and MA polynoial q θ ( L ) = 1 + θ 1L + θ L θ q L. An ARMA odel is stationary provided that the roots of φ ( L ) = lie outside the unite circle. This process is invertible if the roots of θ ( L) = lie outside the unite circle. Low order ARMA odels are of uch interest since any real data sets are well approxiated by the rather than by a pure AR or pure MA odel. In general, ARMA odels need fewer paraeters to describe the process. In ost cases econoic tie series are non-stationary and therefore we cannot apply ARMA odels directly. One possible way to reove the proble is to take difference so as to ake the stationary. Non-stationary series often becoe stationary after taking first difference ( X t X t 1 = (1 L) X ). If the original tie series is differenced d t ties, then the odel is said to be an ARIMA (p, d, q) where I stands for integrated and d denotes the nuber of differences taken. Such a odel is described by φ ( L)(1 L) d X = α + θ ( L) ε t The cobined AR operator is now φ ( L)(1 L) d. The polynoials φ ( z ) and θ ( z ) have all their roots outside the unit circle. The odel is called integrated of order d and the process is said to have d unit roots. t -- Nonparaetric prediction odels --1- Nonparaetric Soothers The general nonparaetric regression odel (Fox,, 5) is as follows: ' y = f ( X ) + ε i i i = f ( x, x,..., x ) + ε i1 i ik i ε σ i ~ NID(, ) The regression function is f(.) unspecified in advance and is estiated directly. In fact, there is no paraeter to estiate. It is iplicitly assued that f (.) is a sooth, continuous function. If there is only one

4 / A Coparative Study of Paraetric and Nonparaetric Regressions predictor yi = f ( xi) + ε then it is called scatter plot soothing because i it traces a sooth curve through a scatter plot of y against x. There are several soothers such as local averaging, kernel soother, Weighted Scatterplot Soothing (Lowess) and spline Soother that fit a linear or polynoial regression to the data points in the vicinity of x and then use the soothed value as the predicted value at x Local Averaging In local averaging procedures, we ove a window continuously over the data, averaging the observations that fall in the window. The estiated values f ˆ ( x ) at a nuber of focal values of x are calculated and connected. It is possible to use a window of fixed width or to adust the width of window to include a constant nuber of observations. Local averages are usually subect to boundary bias, roughness and distortion (when outliers fall in the window) Kernel Soother A Kernel soother is an extension of local averaging and usually produces a soother result. At the focal value x, it is of the for n fˆ( x ) = i= 1 xi x yk i b n i= 1 xi x K b where b is a bandwidth paraeter, and K a kernel function. The Gaussian kernel ( K N ( z )) and the tricube kernel ( K T ( z )) are popular choices of kernel functions. 1 z K N ( z) = e π { 3 3 K T ( z) = (1- z ) for z <1 for z 1 For the Gaussian kernel the bandwidth b is the standard deviation of a noral distribution and for the tricube kernel b is the half-width of a window

5 Fattahi, SH. /3 enclosing the observations for the local regression. Although the kernel soother has a better perforance as copared to the local average regression, it is still subect to boundary bias. It is iplicitly assued that the bandwidth b is fixed, but it is possible for kernel soothers to be adapted to nearest-neighbor bandwidths. We can adust b (x) so that a constant nuber of observations are included in the window. The fraction /n is called the span of the kernel soother and is chosen based on a cross-validation approach. The kernel estiator can produce soother results using larger bandwidths. In fact, there is a direct relationship between the span and soothing degree: the larger the span, the soother the result Lowess Soother As entioned above, the kernel estiation has soe probles. Local polynoial regression tries to overcoe these difficulties and provides a generally adequate ethod of nonparaetric regression which extends to additive regression (Fox, 5). An ipleentation of local polynoial regression is lowess (Cleveland, 1979). The algorith used by lowess soothing applies robust locally linear fits. It is siilar to local averaging but the data points that lie in the window are weighted so that nearby points get the ost weight and a robust weighted regression is used. We can exaine local polynoial regression in two cases: siple regression and ultiple regression. Siple Regression: suppose we want to estiate the siple regression yi = f ( xi) + ε i at a particular x-value, for exaple x. Local polynoial regression extends kernel estiation to a polynoial fit at x, using local kernel weights, wi = K[( xi x ) / b]. We ipleent a pth - order weighted-least-squares polynoial regression of y on x, y = α + β ( x x ) + β ( x x ) β ( x x ) p + e i 1 i i p i i to iniize the weighted residual su of squares, we. This i = 1 i i procedure is repeated for representative values of x. As in kernel regression, the bandwidth b can either be fixed or variable, b(x), and the span of the local-regression soother is selected based on a cross-validation approach. n

6 4/ A Coparative Study of Paraetric and Nonparaetric Regressions ' Multiple Regression: in this case, y i = f ( X i) + ε, we need to i define a ' a ultivariate neighborhood around a focal point x = ( x1, x,..., xk ). Furtherore, Euclidean distance is eployed in the lowess function as: k D( x, x ) = ( z z ) where the z i are the standardized predictors, i i = 1 zi = xi x s, x is the ean of the th predictor and s is its standard deviation. Calculating weights are based on the scaled distances: D( xi, x ) wi = W b Where w (.) is a weight function. In soe cases, b needs to be adusted to define a neighborhood including the [ns] nearest neighbors of x (where the square brackets denote rounding to the nearest integer). As a siple exaple, a local linear fit takes the for: y = α + β ( x x ) + β ( x x ) β ( x x ) + e i 1 i1 1 i k ik k i The cobinations of predictor values are used repeatedly to create the regression surface Spline Soother Suppose we have n pairs( x, y ) considered as n x ax '' ( ) [ i ( i) ] ( ) x in i = 1 i i. A soothing spline equation is s s h = y f x + h f x dx The equation consists of two ters. The first ter is the residual su of squares and the second ter is a roughness penalty. The obect is to find the function f ˆ ( x ) with two continuous derivatives that iniized the penalized su of squares. Here h is a soothing paraeter. For h=, f ˆ ( x ) will interpolate the data if the xi are distinct; this is siilar to a localregression estiate with span=1/n. If h is very large, then ˆf will be selected

7 Fattahi, SH. /5 so that ˆ " f ( x ) is everywhere, which iplies globally linear least-squares fit to the data. This is again siilar to local regression with infinite neighborhoods. The Spline Soother is ore attractive than local regression because there is an explicit obective-function to optiize. But it is not easy to generalize splines to ultiple regression. Generally, the soothing paraeter h is selected indirectly by setting the equivalent nuber of paraeters for the soother.both soothing-spline and local-regression fits with the sae degree of freedo are usually very siilar. --- Nonparaetric Models Additive Model (AM) Nonparaetric regression based on kernel and soothing spline estiates in high diensions faces two probles, that is, the curse of diensionality and interpretability. Stone (1985) proposed the additive odel to overcoe these probles. In this odel, since each of the individual additive ters is estiated using a univariate soother, the curse of diensionality is avoided. Furtherore, while the nonparaetric for akes the odel ore flexible, the additivity allows us to interpret the estiates of the individual ters. Hastie and Tibshirani (199) proposed generalized additive odels for a wide range of distribution failies. These odels allow the response variable distribution to be any eber of the exponential faily of distributions. We can apply additive odels to Gaussian response data, logistic regression odels for binary data, and loglinear or log-additive odels for Poisson count data. A generalized additive odel has the for Y = α + f ( X ) + f ( X ) f ( X ) + ε 1 1 where f (.) are unspecified sooth (partial-regression)functions. We fit each function using a scatterplot soother and provide an algorith for siultaneously estiating all functions. Here an additive odel is applied to a logistic regression odel as a generalized additive odel. Consider a logistic regression odel for binary data. The ean of the binary response μ ( X ) = Pr( Y = 1 X ) is related to the explanatory variables via a linear regression odel and the logit link functions: p p

8 6/ A Coparative Study of Paraetric and Nonparaetric Regressions μ ( X ) log = α + β 1X β X 1 μ ( X ) The additive logistic odel replaces each linear ter by a ore general functional for ( ) log μ X = α + f1( X 1) f ( X ) 1 μ ( X ) In general, the conditional ean μ ( X ) of a response Y is related to an additive function of the explanatory variables via a link function g: g[ μ ( X )] = α + f1( X 1) f ( X ) The functions f are estiated in a flexible way using the backfitting algorith. This algorith fits an additive odel using regression-type fitting echaniss. Consider the th set of partial residuals ε = Y ( α + f k ( X k )) k Then E ( ε X ) = f ( X ). This observation provides a way for estiating each f (.) given estiates { f ˆ (.), } i for all the others. The iterative process is called the backfitting algorith (Friedan and Stuetzle, 1981) Multiple Adaptive Regression Splines (MARS) This approach (Friedan (1991), Hastie et al (1)) fits a weighted su of ultivariate spline basis functions and is well suited for high-diensional probles, where the curse of diensionality would likely create probles for other ethods. The MARS uses the basis functions ( x t ) and + ( t x ) + in the following way ( x t ) + ={ ={ ( t x ) + x-t if x>t otherwise t-x if x<t otherwise

9 Fattahi, SH. /7 The + denotes positive part. Each function is piecewise linear or linear spline, with a knot at value t. These functions are called a reflected pair for each input X with knots at each observed value x i of that input, and then the set of basis functions is defined as C = { ( X t) +,( t X ) + } The strategy for odel-building is a forward stepwise linear regression using functions fro the set C and their products. Thus the MARS odel has the for M = β + = 1 f ( X ) β h ( X ) where the coefficients β are estiated by iniizing the residual suof-squares and each h ( X ) is a function in C. By setting h ( X ) = 1 (constant function), the other ultivariate splines are products of univariate spline basis functions: k h ( X ) = h( x t ) 1 k i( s, ) s, s= 1 where the subscript i( s, ) eans a particular explanatory variable, and the basis spline in that variable has a knot at t s,. k is the level of interactions between i( s, ) variables and the values of, k1, k,..., k, are the knot sets. Explanatory variables in the odel can be linearly or nonlinearly and are chosen for inclusion adaptively fro the data. The odel will be additive if the order of interactions equals one ( k = 1 ). A backward deletion procedure is used in the MARS odel to prevent overfitting. The basis functions which have little contributions to the accuracy of fit are deleted fro the odel at each stage, producing an estiated best odel f ˆ ( λ ) of each size λ. We can apply a generalized cross-validation criterion to estiate the optial value of λ in the following way N ˆ ( y ( )) 1 i f x i= λ i GCV ( λ) = (1 M( λ) / N) The value of M (λ) includes the nuber of basis functions and the nuber of paraeters used in selecting the optial positions of the knots.

10 8/ A Coparative Study of Paraetric and Nonparaetric Regressions Proection-Pursuit Regression (PPR) If the explanatory vector X is of high diension, the additive odel does not cover the effect of interactions between the independent variables. Proection-Pursuit Regression (Friedan and Stuetzle, 1981) applies an additive odel to proected variables, proecting predictor variables X in M, as follows M T ( ) E ( ε ) =,var( ε ) = σ = 1 Y = g w X + ε where w are unit p-vectors of unknown paraeters. The functions g are unspecified and estiated along with the direction w using soe flexible soothing ethod. The PPR odel eploys the backfitting algorith and Gauss-Newton search to fit Y. T The functions g ( wx ) are called the ridge functions because they are constant in all but one direction. They vary only in the direction defined T by the vector w. The scalar variable V = ( wx ) is the proection X onto the unit vector w. The ai is to find w to yield the best fit to the data. If M is chosen large enough then the PPR odel can approxiate arbitrary continuous function of X (Diaconis and Shahshahani, 1984). However, in this case there is a proble of interpretation of the fitted odel since each input enters into the odel in a coplex and ultifaceted way (Hastie et al, 1). As a result, the PPR odel is a good option only for forecasting. To fit a PPR odel, we need to iniize the error function N M T i i i= 1 = 1 E = [ y g ( w x )] over functions g and direction vectors w. The g and w are estiated by iteration. Iposing coplexity constraints on the g is needed to avoid overfitting. There are two stages to estiate g and w. First, to obtain an estiate of g, suppose there is one ter (M=1). We can for the derived T variables vi = w x for any value of w. This iplies a one-diensional i soothing proble and any scatterplot soother such as soothing spline can be used to estiate g. Second, we iniize E over w for any value of g. These two steps are iterated until convergence. If there is ore than one ter

11 Fattahi, SH. /9 in the PPR odel then the odel is built in a forward stage-wise anner that at each stage a pair ( w, g ) is added Neural Networks Many recent ethods to developing data-driven odels have been inspired by the learning abilities of biological systes. For instance, ost adults drive a car without knowledge of the underlying laws of physics and huans as well as anials can recognize patterns for the tasks such as face, voice or sell recognition. They learn the only through data-driven interaction with the environent. The field of pattern recognition considers such abilities and tries to build artificial pattern recognition systes that can iitate huan brain. The interest to such systes led to extensive studies about neural networks in the id-198s (Cherkassky and Mulier, 7). Why use Neural Networks? Neural network odeling has seen an explosion of interest as a new technique for estiation and forecasting in econoics over the last decades. They are able to learn fro experience in order to iprove their perforance and to adapt theselves to changes in the environent. In fact, they can derive trends and detect patterns fro coplicated or iprecise data, and then odel coplex relationships between explanatory variables (inputs) and dependent variables (outputs). They are resistance to noisy data due to a assively parallel distributed processing. Learning in neural network odel Stochastic approxiation (or gradient descent) is one of the basic nonlinear optiization strategies coonly used in statistical and neural network ethods (Cherkassky and Mulier, 7). The gradient-descent ethods are based on the first order Taylor expansion of a risk functional R ( w) = L( y, f ( x, w)) p( x, y) dxdy (1) where R( w) is the risk functional, (, (, )) (, ) L y f x w the loss function and p x y the oint probability density function. For regression, a coon loss function is the squared error

12 3/ A Coparative Study of Paraetric and Nonparaetric Regressions L ( y, f ( x, w)) ( y f ( x, w)) = () Learning is then defined as the process of estiating the function f ( x, w ) that iniizes the risk functional R( w) = ( y f ( xw, )) pxy (, ) dxdy using only the training data. Although the gradient-descent ethods are coputationally rather slow, their siplicity has ade the popular in neural networks. We will exaine two cases to describe such ethods: linear paraeter estiation and nonlinear paraeter estiation. Linear Paraeter Estiation Consider a linear (in paraeters) approxiating function and the loss function specified above. For the task of regression, it can be shown that the epirical risk is as follows 1 1 R w L x y w y f x w n n ep ( ) = ( i, i, ) = ( i ( i, )) (3) n i= 1 n i= 1 This function is to be iniized with respect to the vector of paraeters w. Here the approxiating function is a linear cobination of fixed basis functions yˆ = f ( x, w) = w g ( x) (4) = 1 For soe (fixed). The updating equation for iniizing R ( ) ep w with respect to w is (5) w( k + 1) = w( k ) γ k L( x( k ), y( k ), w) w where x ( k ) and y( k) are the sequences of input and output data saples presented at iteration step k. The gradient above can be written as

13 Fattahi, SH. /31 L yˆ L( x, y, w) = = ( yˆ y) g ( x) w yˆ w (6) Now the local iniu of the epirical risk can be coputed using the gradient (6). Let us start with soe initial values w (). The stochastic approxiation ethod for paraeter updating during each presentation of kth training saple is as: Step 1: Forward pass coputations. z ( k) = g ( x( k)), 1,.., (7) yˆ ( k) = w ( k) z ( k). (8) = 1 Step : Backward pass coputations. δ ( k) = yˆ ( k) y( k) (9) w ( k + 1) = w ( k) γ δ ( k) z ( k) (1) k γ k is a sall positive nuber decreasing with k. where the learning rate In the forward pass, the output of the approxiating function is coputed whereas in the backward pass, the error ter (9), which is called delta in neural network literature, for the presented saple is calculated and utilized to odify the paraeters. The paraeter updating equation (1), known as delta rule, updates paraeters with every training saple. Nonlinear Paraeter Estiation The standard ethod used in the neural network literature is the back propagation algorith which is an exaple of stochastic approxiation strategy for nonlinear approxiating functions. As it was considered already, the apping fro inputs to output given by a single layer of hidden units is as follows

14 3/ A Coparative Study of Paraetric and Nonparaetric Regressions n f ( x, w, V ) = w + w g( v + x v ) i i = 1 i= 1 d (11) In contrast to (4), the set of functions is nonlinear in the paraeters V. We seek values for the unknown paraeters (weights) V and w that ake the odel fit the training data well. To do so, the su of squared errors as a easure of fit ust be iniized: n ep = ( ( i,, ) i ) (1) i= 1 R f x w V y The stochastic approxiation procedure for iniizing respect to the paraeters V and w is R ep with V ( k + 1) = V ( k) γ L( x( k), y( k), V ( k), w( k)), (13) k V wk ( + 1) = wk ( ) γ Lxk ( ( ), yk ( ), V( k), wk ( )), k= 1,..., n, k w (14) where x( k ) and y( k) are the kth training saples, presented at iteration step k. The loss function L is 1 L ( x( k), y( k), V ( k), w( k)) ( f ( x, w, V ) y) = (15) where the factor ½ is included only for siplifying gradient calculations in the learning algorith. We need to decopose the approxiation function (11) for coputations of the gradient of loss function (15) as follows a z d = xv (16) i i i= = g( a ), (17) z =, 1 = yˆ = w z (18)

15 Fattahi, SH. /33 For siplicity, we drop the iteration step k, consider calculation/ paraeter update for one saple at a tie and incorporate the ters w and v into the suations ( x 1 ). The relevant gradients, based on the chain rule of derivatives, are R R yˆ a = v yˆ a v i i R R yˆ = w yˆ w, (19). () In order to calculate each of the partial derivatives, we need equations (15) to (18). Therefore, R = yˆ y yˆ yˆ ' = g ( a ) w a a v i yˆ w = x i = z (1) () (3) (4) If we plug the partial derivatives (1)-(4) into (19) and (), the gradient equations are R ' = ( yˆ y) g ( a ) w x i (5) v i R w = ( yˆ y) z (6)

16 34/ A Coparative Study of Paraetric and Nonparaetric Regressions Using these gradients and the updating equations, we can construct a coputational ethod to iniize the epirical risk. Starting with soe initial values w () and V (), the stochastic approxiation ethod updates weights upon presentation of a saple (x (k), y (k)) at iteration step k with learning rate γ as k Step 1: Forward pass coputations. Hidden layer d a ( k) = x ( k) v ( k), (7) i i i= z ( k) = g( a ( k)), (8) Output layer z ( k ) = 1 yˆ ( k) = w ( k) z ( k). (9) = Step : Backward pass coputations. Output layer δ ˆ ( k ) = y( k ) y( k ) (3) w ( k + 1) = w ( k) γ δ ( k) z ( k) (31) k Hidden layer ' δ ( k ) = δ ( k ) g ( a ( k )) w ( k + 1) (3) 1 v ( k + 1) = v ( k) γ δ ( k) x ( k) (33) i i k 1 i In the forward pass, the output of the approxiating function is coputed whereas in the backward pass, the error ter for the presented saple is calculated and utilized to odify the paraeters in the output layer. Since it is possible to propagate the error at the output back to an error at each of the internal nodes a through the chain rule of derivatives, the procedure is called error backpropagation. In fact it is a propagation of the error signals fro the output layer to the input layer.

17 Fattahi, SH. /35 The updating steps for output layer are siilar to those for the linear case. Besides, the updating rule for the hidden layer is the sae as the linear one but for the delta ter (3). For this reason, backpropagation update rules (3) and (33) are usually called the generalized delta rule. The paraeter updating algorith holds if the saple size is large (infinite). However, if the nuber of training saples is finite, the asyptotic conditions of stochastic approxiation are (approxiately) satisfied by the repeated presentation of the finite training saple to the training algorith. This is called recycling and the nuber of such repeated training saples is called the nuber of cycles (or epochs). It is possible to use the backpropagation algorith for networks with several output layers and networks with several hidden layers. For instance, if additional layers are added to the approxiation function, then errors are propagated fro layer to layer by repeated application of generalized delta rule. It should be noted that a neural network odel can be identified as a pursuit proection regression (PPR) odel (Hastie et al, 1). In fact, the neural network with one hidden layer has the exactly the sae for as the PPR odel. The only difference is that the PPR odel uses nonparaetric functions ( g ( v )) while the neural network eploys a sipler function which is based on a sigoid transfer function.

18 36/ A Coparative Study of Paraetric and Nonparaetric Regressions 3- Epirical Results 3-1- Background The Iranian econoy is oil-reliant so that any change in oil price can directly affect all econoic sectors. It should be noted that Iran ranks second in the world in natural gas reserves and third in oil reserves. It is also OPEC s second largest oil exporter. The econoic sectors include the services sector, industry (oil, ining and anufacturing) and the agricultural sector. During the recent decades, the services sector has contributed the largest percentage of the GNP, followed by industry and agricultural sectors. The share of the services sector was 51 percent of GNP in 3 while those of the industry and agricultural sectors were 35.1 and 13.9 percent of GNP respectively. The Iranian econoy has experienced a relatively high inflation averaging about 15 percent per year. The inflation rate has even been ore than 1 percent on average after the 1973 oil crisis. Furtherore, there is a general agreeent over the underestiating of the easured inflation due to price controls and governent subsidies. The epirical evidence iplies that inflation is persistent in Iran. In other words, the effects of a shock to inflation result in a changed level of inflation for an extended period. To see this, the inflation rate is regressed on its own lags. rgnpi =.44rgnpi +.49rgnpi t t 1 t (t-value) (3.15) (3.4) As the su of coefficients on lagged inflation (.93) is close to one, shocks to inflation have long-lasting effects on inflation. 3-- Results This study copares the perforance of paraetric and nonparaetric regressions in the Iranian econoy over the period Different odels will be applied to the inflation series and then the predicted values will be evaluated. In fact, out-of-saple estiates of inflation generated by the paraetric and nonparaetric odels will be copared.

19 Fattahi, SH. /37 The epirical results of the Augented Dicky-Fuller test indicated that all the variables eployed are stationary, and thus this issue helps us to avoid the proble of the spurious relationships (see Appendix for the data source and definitions). It is assued agents use the lagged values of inflation and real GNP growth to forecast inflation. Figure 1.a and Figure 1.b deonstrate a local linear regression fit of inflation rate (rgnpi), defined as the rate of change of GNP deflator, on the lagged inflation rate (rgnpilag1) and lagged real GNP growth rate (rgnplag1) using the Lowess function for a variety of spans. If the fitted regression looks too rough, then we try to increase the span but if it looks sooth, then we will exaine whether the span can be decreased without aking the fit too rough (Fox, 5). The obective is to find the sallest value of span (s) that provides a sooth fit. A trial and error procedure suggests that the span s=.5 is suitable and it sees to provide a reasonable coproise between soothness and fidelity to the data. s =.1 s =.3 s =.5 rgnpi rgnpi rgnpi rgnpilag rgnpilag rgnpilag1 s =.7 s =.9 rgnpi rgnpi rgnpilag rgnpilag1 Figure 1-a: Local Linear Regression fit of Inflation Rate (Rgnpi) on the Lagged Inflation Rate (Rgnpilag1) Using Lowess Function for a Variety of Spans

20 38/ A Coparative Study of Paraetric and Nonparaetric Regressions s =.1 s =.3 s =.5 rgnpi rgnpi rgnpi rgnplag1-4 rgnplag1-4 rgnplag1 s =.7 s =.9 rgnpi rgnpi rgnplag1-4 rgnplag1 Figure 1-b: Local linear regression fit of inflation rate (rgnpi) on the lagged real GNP growth rate (rgnplag1) using Lowess function for a variety of spans A test of nonlinearity is perfored by contrasting the nonparaetric regression odel with the linear siple-regression odel. We regress inflation on rgnpilag1 (Case 1) and rgnplag1 (Case) separately. As a linear odel is a special case of a nonlinear odel, two odels are nested. An F- test is forulated by coparing alternative nested odels. The results is as follows Linear odel vs Nonparaetric regression (Case1): F=8.78(pvalue=.8) Linear odel vs Nonparaetric regression (Case): F=6.48(pvalue=.4) It is obvious that the relationship between the dependent variable and explanatory variables are significantly nonlinear. It should be noted that the variable rgnplags1 will not be significant if a linear regression is considered. Since nonparaetric regression based on soothing functions faces the curse of diensionality, the additive odel has been proposed (Stone, 1985). The result of fitting an additive odel using Lowess soother can be written as

21 Fattahi, SH. /39 rgnpi = S (rgnpilag1) + S (rgnplag1) F (4.13) (4.43) p-value (.1) (.3) where S denotes the Lowess soother function. It is obvious that both soothers are significantly eaningful. Furtherore, the linear odel is nested by the additive odel with p-value being equal to.1. Figure illustrates plots of the estiated partial-regression functions for the additive regression odel. The points in each graph are partial residuals for the corresponding explanatory variable, reoving the effect of the other explanatory variable. The broken lines deonstrate pointwise 95-percent confidence envelopes for the partial fits. rgnpi rgnpi rgnplag rgnpilag1 Figure : Plots of the Estiated Partial-Regression Functions for the Additive Regression of the Inflation Rate (Rgnpi) on the Lagged Real GNP growth Rate (rgnplag1) and the Lagged Inflation Rate (Rgnpilag1) We use MARS odel to fit a piecewise linear odel with additive ters to the data. The results indicate that pairwise interaction ters (by degree= and degree=3) ake little difference to the effectiveness of explanatory variables. The additive odel sees to be too flexible and it is not able to cover the effect of interactions between explanatory variables. To reove this proble, the Proection- Pursuit Regression odel has been proposed. The

22 4/ A Coparative Study of Paraetric and Nonparaetric Regressions PPR odel applies an additive odel to proected variables (Friedan and Stuetzle, 1981). Figure 3 shows plots of the ridge functions for the three twoter proection pursuit regressions fitted to the data ter ter ter ter ter ter Figure 3: Plots of the Ridge regression for three two-ter Proection Pursuit Regressions Fitted to the Data. Although MARS odel is an accurate ethod, it is sensitive to concurvity. Neural networks do not share this proble and are better able to predict in this situation. In fact, as neural networks are nonlinear proection ethods and tend to overparaeterize, they are not subect to concurvity (Hastie et al, 1). We exained several neural network odels and the results indicate that a -3-1 network has a better perforance. The Wilcoxon test has been used to copare the squared error of a neural network odel and a rival odel. In fact, out of saple forecasting has been eployed to evaluate different odels. The perforance of PPR and additive odels appears to differ fro the neural network odel, iplying that the NN odel can significantly outperfor the PPR odel and it has a better perforance than the additive odel, but not by uch. Furtherore, the NN odel is significantly better than the linear odel (LM). However,

23 Fattahi, SH. /41 there is no possibility that the NN odel can outperfor the MARS odel (see Table 1). Table 1: Out of Saple Forecasting based on Wilcoxon test p-value PPR vs. NN.1 LM vs. NN. MARS vs. NN 1 AM vs. NN.38 Now we copare the NN odel to the paraetric autoregressive oving average (ARMA) odel for inflation. A collection of ARMA (p, q) odels, for different orders of p and q, have been estiated and then the best odel was selected according to the Akaike inforation criterion (AIC) and the Schwarz inforation criterion (SIC). Exaining the ARMA odels for the inflation series indicated that ARMA (1, 1) is the best-fitting odel. Diagnostic checking, the correlogra (autocorrelations) of inflation fro the regression tests was exained and confired the results. The last 5 observations are used for coparing the ex post forecasts generated by the two odels. Furtherore, the Root Mean Square Error (RMSE) is used to evaluate ex post forecasts. We apply the feed-forward backpropagation as learning algorith where only lagged inflation is used as input. The results iply that the forecasting perforance of the NN odel (RMSE=.5) is significantly better than that of the ARMA odel (RMSE=11.73). It should be noted that the results fro the inflation lags exceeding one and ore nuber of hidden layers are alost the sae. Therefore, the NN odel outperfors the paraetric ARMA odel. 3- Conclusions This study exained the forecast perforance of paraetric and nonparaetric odels. The agents were assued to use a paraetric autoregressive oving average (ARMA) odel or nonparaetric odels to forecast inflation. The results revealed that the neural network odel yields better estiates of inflation rate than do paraetric autoregressive oving average (ARMA) and linear odels. Coparing to the nonparaetric

24 4/ A Coparative Study of Paraetric and Nonparaetric Regressions alternatives, the results of Wilcoxon tests deonstrated that the forecasting perforance of Proection-Pursuit Regression and Additive odels appeared to differ fro the Neural Network odel, iplying that the Neural Network odel can significantly outperfor Proection-Pursuit Regression and Additive odels. However, there was no possibility that the Neural Network odel can outperfor the Multiple Adaptive Regression Splines odel. References 1- Barucci E and Landi L(1998), Nonlinear Versus Linear Devices: A Procedural Perspective, Coputational Econoics, Vol. 1, pp Box, G.E.P and Jenkins, G (1976) Tie Series Analysis: Forecasting and Control. San Francisco: Holden Day. 3- Chatfield C (), Tie-Series Forecasting, Chapan & Hall/CRC 4- Cherkassky, V and Mulier F (7), Learning fro Data: Concepts, Theory, and Methods, John Wiley & Sons, Second Edition 5- Cleveland, W.S. (1979), Robust Locally Weighted Regression and Soothing Scatterplots, Journal of the Aerican Statistical Association 74 (368): Diaconis P and Shahshahani M (1984), On nonlinear functions of linear cobinations, SIAM Journal of Scientific and Statistical Coputing, Vol. 5, No. 1, pp Fan, J, Yao, Q (3), Nonlinear Tie Series: Nonparaetric and Paraetric Methods, Springer-Verlag: Berlin Heidelberg and New York. 8- Fox, J (), Multivariate Generalized Nonparaetric Regression, Sage Publications. 9- Fox, J (5), Introduction to Nonparaetric Regression, McMaster University, Canada. 1- Friedan, JH, Stuetzle, W (1981), Proection Pursuit Regression, Journal of the Aerican Statistical Association, Vol. 76, No. 376, pp Friedan, HF (1991), Multivariate Adaptive Regression Splines, The Annals of Statistics, Vol. 19, No. 1, pp Hastie, TJ, Tibshirani, RJ (199), Generalized Additive Models, Chapan & Hall. 13- Hastie, TJ, Tibshirani, R, Friedan, J (1), The Eleents of Statistical Learning: Data Mining, Inference, and Prediction, Springer-Verlag: Berlin Heidelberg and New York. 14- Stone, C.J. (1985), Additive Regression and Other Nonparaetric Models, Annals of Statistics, 13,

25 Fattahi, SH. /43 Appendix: Data source and definitions The data are annually for the period and are collected fro the Central Bank of Iran. gnp = real GNP (at the constant 1997 prices) gnpi = GNP deflator (1997=1)