Improvement in Forecasting Accuracy Using the Hybrid Model of ARFIMA and Feed Forward Neural Network

Transcription

1 American Journal of Inelligen Sysems 2012, 2(2): DOI: /j.ajis Improvemen in Forecasing Accuracy Using he Hybrid Model of ARFIMA and Feed Forward Neural Nework Cagdas Hakan Aladag 1, Erol Egrioglu 2,*, Cem Kadilar 1 1 Deparmen of Saisics, Haceepe Universiy, Ankara, Turkey 2 Deparmen of Saisics, Ondokuz Mayis Universiy, Samsun, Turkey Absrac In recen years, auoregressive fracionally inegraed moving average (ARFIMA) models have been used for forecasing of long memory ime series in he lieraure. Major limiaion of ARFIMA models is he pre-assumed linear form of he model. Since many ime series in real-world have non-linear srucure, ARFIMA models are no always saisfacory. Boh heoreical and empirical findings in lieraure show ha combining linear and non-linear models such as ARIMA and arificial neural neworks (ANN) can be an effecive and efficien way o improve forecass. However, o model long memory ime series, any hybrid approach has no been proposed in he lieraure. In his sudy, a new hybrid approach combining ARFIMA and feedforward neural neworks (FNN) is proposed o analyze long memory ime series. The proposed hybrid mehod is applied o ourism daa of Turkey whose srucure shows dominanly he characerisic of long erm. Then, his hybrid mehod is compared wih oher mehods and i is found ha he proposed hybrid approach has he bes forecasing accuracy. Keywords Feed forward neural neworks, ARFIMA, Hybrid approach, Long memory 1. Inroducion A popular class of models for ime series wih long memory behaviour is he ARFIMA ( pdq,, ) model ([27]). This kind of models exended classical ARIMA ( pdq,, ) models by assuming he differencing parameer d as a real value. I is well known ha ARFIMA models are linear ime series model. Since some long memory ime series have boh linear and non-linear srucures, ARFIMA models can be inadequae for his ype of series. Therefore, hese ime series are modeled by a hybrid of linear and non-linear models, and by his way forecas accuracy is improved. ANN have been widely used o model ime series in various fields of applicaions[4] and used as a good alernaive mehod for boh linear and non-linear ime series forecasing. Since ANN can model boh non-linear and linear srucures of ime series, i is obvious ha i can give good resuls in forecasing. As[37] menioned, one of he mos popular neural ne paradigms is he feed forward neural nework ha is used in our sudy. [40]review he lieraure for forecasing ime series by ANN. Boh heoreical and empirical findings in lieraure show ha combining differen mehods can be an effecive and efficien way o improve forecass. Therefore, hybrid of * Corresponding auhor: erole@omu.edu.r (Erol Egrioglu) Published online a hp://journal.sapub.org/ajis Copyrigh 2012 Scienific & Academic Publishing. All Righs Reserved ARIMA and ANN mehods has been used for modelling boh linear and non-linear paerns equally well.[30] proposed hybrid ARIMA and suppor vecor machines model.[38] combined seasonal ime series ARIMA model and FNN.[41] inroduced a hybrid of ARIMA and FNN models. Forecasing ourism daa is an imporan ask since i is employed for fuure planning. In he lieraure, here have been many sudies using ime series analysis in ourism daa.[10],[39],[23-25], and[22] presened survey sudies. Recenly,[35] have summarized he sudies on his opic in he las decade such as[18, 19, 5, 6, 17, 29] and[31]. Empirical resuls in all of hese papers show ha arificial neural neworks can produce beer resuls han hose generaed by convenional ime series mehods. However, ARFIMA models, which are also applied in our sudy while modelling he ourism daa, have been found very successful for he analysis of ourism daa in[7]. In his paper, o model long memory ime series which have boh linear and non-linear srucures, a new hybrid mehod is proposed by modifying[41] hybrid mehod. This paper aims o obain beer forecasing accuracy by modelling boh linear and non-linear paerns of long memory ime series. Therefore, if more accurae forecass are obained, beer plans for aciviies of ourism will be made. The some hybrid mehods have been proposed in[15], [16],[20],[28],[32],[33] and[41] sudies. Alhough here have been some mehods combine ARIMA and ANN models, he combinaion of ARFIMA and ANN models haven' proposed in he lieraure. The proposed mehod is he firs sudy abou combining of ARFIMA and feed forward neural nework.

2 American Journal of Inelligen Sysems 2012, 2(2): The proposed mehod is applied o he ourism daa of Turkey. This monhly daa concerns he number of ouriss coming o Turkey beween he periods 1995:1-2005:12. The res of he paper is organized as follows. In Secions 2 and 3, ARFIMA models and elemens of FNN are presened, respecively. In Secion 4, he proposed hybrid approach is inroduced. In Secion 5, he proposed mehod is applied o he ourism daa of Turkey and compared wih ARIMA, ARFIMA and FNN models. In he las secion, he efficiency of he proposed mehod is discussed. 2. ARFIMA Models ARFIMA models are used o model long range dependen ime series. ARFIMA models were inroduced by[11]. AR- FIMA ( pdq,, ) model can be given by d ϕ( B)(1 B) X = θ( Be ), 1/2< d< 1/2 where B is he back-shif operaor such ha BX = X 1 and e is a whie noise process wih Ee ( ) = 0 and variance 2 p σ e. The polynomials ϕ( B) = (1 ϕ1b ϕpb ) and q θ( B) = (1 θ1b θqb ) have orders p and q respecively wih all heir roos ouside he uni circle.[3] exended he esimaion of ARFIMA models for any d > 1/2 by considering he following variaion of he ARFIMA model: δ m ϕ( B)(1 B) (1 B) X = θ( Be ) 1 / 2 < δ < 1 / 2 (1) The ineger m is he number of imes ha X mus be differenced o achieve saionary, and hus differencing parameer is given by d = δ + m. General properies of AR- FIMA models were given by Hosking[13, 14] and[2]. Sudies abou he parameer esimaion of ARFIMA models sill coninue. Many maximum likelihood (ML) mehods for ARFIMA are proposed in lieraure such as approximae ML mehods (AML) by[12, 14, 21] and[2]; exac ML mehod (EML) by[36]; condiional sum of square (CSS) mehod by[8]. Noe ha CSS mehod is as efficien as EML mehod and i is idenical wih AML mehod by[2] ha is based on infiniy auoregressive presenaion. 3. Elemens of FNN Elemens of he ANN are nework archiecure, learning algorihm and acivaion funcion. Deermining hese elemens ha affec he forecasing performance of FNN should be considered carefully. One criical decision is o deermine he appropriae archiecure, ha is, he number of layers, number of nodes in each layers and he number of arcs which inerconnec wih he nodes[42]. FNN has been used by many sudies in forecasing. However, deermining of he archiecure is a basic problem. Since, in he lieraure, here is no general rule for deermining he bes archiecure, many archiecures can be considered as he correc archiecure. Fig. 1 depics he broad FNN archiecure ha has single hidden layer and single oupu. Learning of FNN for a specific ask is equivalen o finding he values all of he weighs such ha he desired oupu is generaed o he corresponding inpu. Various raining algorihms have been used for he deerminaion of he opimal weighs values. The mos popularly used raining mehod is he back propagaion algorihm ([34]). In he back propagaion algorihm, learning of he ANN consiss of adjusing all weighs such as he error measure beween he desired oupu and acual oupu ([9]). Anoher elemen of FNN is he acivaion funcion. I deermines he relaionship beween inpus and oupus of a nework. In general, he acivaion funcion inroduces a degree of he non-lineariy ha is valuable mos of he FNN applicaions. The well known acivaion funcions are logisic, hyperbolic angen, sine (or cosine) and he linear funcions. Among hem, logisic ransfer funcion is he mos popular one ([40]). Figure 1. A broad FNN archiecure 4. The Proposed Hybrid ARFIMA and FNN Approach ARFIMA processes have been used o model long memory ime series. However, i is clearly ha using ARFIMA processes for modelling non-linear problems is no adequae because ARFIMA model is based on a linear srucure. Applicaions of hybrid mehods in he lieraure show ha combining differen mehods can be an effecive and efficien way o improve forecass. Since i is difficul o compleely know he characerisics of daa in a real problem, hybrid mehodology which has boh linear and non-linear modelling capabiliies can be a good approach for pracical purposes. Therefore, o model ime series having boh linear and non-linear srucures, hybrid approaches are proposed. To model long memory ime series, any hybrid approach has no been proposed in he lieraure. In his paper, by modifying Zhang s[41] hybrid approach, a new hybrid approach is proposed o solve his problem. The proposed hybrid ARFIMA and FNN approach is given as follows: I is assumed ha a ime series can be considered composing of wo componens, which are a linear auocorrelaion srucure par and a non-linear par respecively. The model is as follows: y = L + N (2) where y denoes original ime series, L denoes he linear componen and N denoes he non-linear componen.

3 14 Cagdas Hakan Aladag e al.: Improvemen in Forecasing Accuracy Using he Hybrid Model of ARFIMA and Feed Forward Neural Nework Linear componen is esimaed by ARFIMA model and residuals obained from he ARFIMA model. e ˆ = y L (3) are esimaed by FNN of[35]. Here L ˆ is he forecasing value for he period of he ime series y by ARFIMA. Residuals are vial in examining he lineariy assumpion of he model. Auocorrelaion coefficiens are used o decide wheher he residuals have linear relaion or no. On he oher hand, non-linear relaion canno be deermined since he auocorrelaion coefficien can be only employed for linear relaion. Thus, he residuals migh denoe non-linear relaion even hough he auocorrelaion coefficiens for he residuals are approximaely abou zero. Therefore, he residuals obained from he ime series model generaed by ARFIMA are analyzed by using FNN. Wih n inpu nodes, he FNN model for he residuals can be wrien as e= f( e 1, e 2,, e n) + ε (4) where f is a non-linear funcion deermined by he FNN and ε is he random error. The esimaion of e by (4) will yield he forecasing of non-linear componen of ime series, N. By his way, forecasing values of he ime series are obained as follows: y ˆ ˆ ˆ = L + N (5) Consequenly, our proposed mehod in his sudy consiss of wo phases. In he firs phase, he ime series is analysed by using ARFIMA models. In he nex phase, he residuals obained in he previous phase are examined by FNN and hen forecas values obained from hese wo models separaely are summed. Besides, in his sudy, wo differen hybrid models are employed since wo differen FNN models are used in he second phase. One FNN model includes logisic acivaion funcion in all layers of he nework. Second one includes logisic acivaion funcion in he hidden layer and linear acivaion funcion in he oupu layer. Figure 2. The ourism daa of Turkey 5. Empirical Resuls The proposed hybrid mehod is applied o he ourism daa of Turkey which is ploed in Fig. 2. This monhly daa concerns he number of ouriss coming o Turkey beween he periods 1995:1-2005:12. For comparison, he daa is also modeled by ARIMA, ARFIMA, and FNN. The las 24 observaions of he daa are used for comparing he mehods by obaining he values of roo mean square error (RMSE), mean absolue percenage error (MAPE), median absolue percenage error (MdAPE) and he res of he observaions are used for he parameer esimaion of he models. Firsly, he ourism daa of Turkey is analysed by AR- FIMA models. Tourism series has seasonaliy. Therefore, he saionary series y can be given by 12 y = (1 B ) x (6) where B is backshif operaor and x represens he ourism series. In his empirical sudy, we will ry o esimae he series, y. We see ha ransformed y series has a long memory srucure afer he ransformaions of x by (6) using R/S es by[26]. Therefore we use ARFIMA models o esimae y series. The mos appropriae model is deermined as ARFIMA (1, d, 3) by Bayesian informaion crierion[1] in S-Plus package program and his model is given as follows: ( B)(1 B) y = ( B 0.211B B ) e where e is found as a whie noise series using Box-Pierce Tes. RMSE value of forecass obained by using ARFIMA (1, d, 3) model for las 24 daa poins is given in Table 1. Secondly, he ourism ime series is direcly analysed by wo differen FNN models. When he bes archiecure design was deermining, rial and error mehod was used. Alhough some sysemaically approaches o deermine archiecure design exis in he lieraure, hey are no preferred generally since hey do no guaranee he bes archiecure. Thus, in he lieraure, he mos preferred and used mehod o deermine ANN srucures in ime series forecasing sudies is rial and error mehod. And his mehod is performed relevan o considered daa. ANN is also a mehod based on he daa examined. Therefore, rial and error mehod is used in our sudy. The firs used FNN model, which includes logisic acivaion funcion in he hidden layer and linear acivaion funcion in he oupu layer, is called FNN1. The oher one, which includes logisic acivaion funcion in all layers, is called FNN2. For each FNN model, 144 archiecures are examined by varying he number of neurons in he hidden layer and in he inpu layer 1 o 12. By hese rials, he bes archiecure, which has he lowes RMSE value for he es se, is deermined. For FNN1 model, he bes archiecure was found as FNN1(11-2-1), which includes 11 neurons in he inpu layer and 2 neurons in he hidden layer. FNN1(11-2-1) forecass and he original ourism series of Turkey for 2004 and 2005 are shown in Table 1. In addiion o his, calculaed RMSE, MAPE, and MdAPE values for FNN1(11-2-1) are presened in Table 2. Similarly, he bes archiecure for FNN2 was found as FNN2(9-2-1). The forecas values for 2004 and 2005 are presened in he Table 1 and calculaed RMSE, MAPE, and MdAPE values for FNN2(9-2-1) are given in he Table 3. Neural neworks ool box of Malab 7.0 version is used in he

4 American Journal of Inelligen Sysems 2012, 2(2): analysis. Finally, he ourism daa is examined by employing he proposed hybrid approach presened in Secion 4. In he firs phase of he proposed mehod, he ourism daa is analysed by ARFIMA model. As menioned, ARFIMA (1, d, 3) is deermined as he mos proper model. In he second phase, he residuals obained in he firs phase are analysed by using FNN1 and FNN2 models, separaely. For each model, 144 archiecures are examined by varying he number of neurons in he hidden layer and in he inpu layer 1 o 12. For FNN1 and FNN2 models, FNN1(9-1-1) and FNN2(1-1-1) are deermined as he bes archiecures, respecively. Then, he prediced values obained by ARFIMA (1, d, 3) and FNN1(9-1-1) are summed. These resuls belong o he firs hybrid mehod ha is called Hybrid1. Similarly, he prediced values obained by ARFIMA (1, d, 3) and FNN2(1-1-1) are summed and he resuls of Hybrid2 are obained. For Table 1. The forecas values all models for 2004 and 2005 years 2004 and 2005, he forecas values obained from he hybrid mehods are presened in he Table 1. The values of RMSE, MAPE, and MdAPE for hese wo hybrid mehods are given in Table 2. I is clearly seen from he resuls ha when he FNN2 model is used in he hybrid model, beer resuls are obained. Therefore, we prefer o employ FNN2 model in he proposed hybrid mehod. From Fig. 3, we can observe he forecas values of all models. In Table 3, he values of RMSE, MAPE, and MdAPE for all mehods are given for comparison. I is observed from his able ha he proposed hybrid mehod gives he bes resuls in erms of all used forecasing crieria since his mehod has he smalles values for all used crieria. Anoher imporan resul is he wo FNN models give he wors forecass. Alhough FNN have proved success in forecasing ime series, we see ha he FNN models are ineffecive for he daa long memory srucured. Years Monhs Arrived Touris FNN1 FNN2 ARFIMA Hybrid1 Hybrid , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,7 Table 2. RMSE, MAPE, and MdAPE values of he hybrid mehods Years Crieria Hybrid1 Hybrid RMSE MAPE MdAPE RMSE MAPE MdAPE Table 3. RMSE, MAPE, and MdAPE values of all mehods Years Crieria FNN1 FNN2 ARFIMA Proposed Mehod RMSE MAPE MdAPE RMSE MAPE MdAPE

5 16 Cagdas Hakan Aladag e al.: Improvemen in Forecasing Accuracy Using he Hybrid Model of ARFIMA and Feed Forward Neural Nework 6. Conclusions Long memory ime series have been analysed by using ARFIMA models which are based on linear srucure. AR- FIMA models are no always adequae for long memory ime series ha have boh linear and non-linear srucures. Therefore, he hybrid mehod which combines linear and non-linear models can be an effecive way o improve forecasing performance. Moivaed by his idea, in his paper, a hybrid model of ARFIMA and FNN is proposed o increase forecasing accuracy. Applying he proposed hybrid mehod o he ourism daa of Turkey, we see ha he proposed mehod is more successful han he oher mehods in erms of obaining beer forecass. All compuaional processes relaed o neural neworks are done wih Malab 7.0 sofware. As a resul, he bes forecass are obained by using he proposed hybrid ARFIMA and FNN2 (1-1-1) model. I is also seen ha using only FNN models are ineffecive for his long memory srucured ime series. This is also an imporan resul for he ANN sudies in he fuure. In furher sudies, we hope o improve he forecasing accuracy by changing he ype of ANN in hybrid models presened here. REFERENCES [1] Akaike, H., 1979, "A Bayesian exensions of he minimum AIC procedure," Biomerika 66, pp: [2] Beran, J., "Saisics for long-memory processes," Chapman&Hall/Crc. (1994). [3] Beran, J., "Maximum likelihood esimaion of he differencing parameer for inverible shor and long memory ARIMA models," Journal of Royal Saisical Sociey Series B 57 (4), (1995) pp: [4] Buhamra, S., Smaoui, N. and Gabr, M., "The Box-Jenkins analysis and neural neworks: predicion and ime series modeling," Applied Mahemaical Modelling 27 (2003) pp: [5] Burger, C.J.S.C., Dohnal, M., Kahrada, M. and Law, R., "A praciioners guide o ime series mehods for ourism demand forecasing a case sudy of Durban, " Souh Africa Tourism Managemen, 22 (2001) pp: [6] Cho, V., A comparison of hree differen approaches o ouris arrival forecasing. Tourism Managemen 24 (2003) pp: [7] Chu, F.L., "A fracionally inegraed auoregressive moving average approach o forecasing ourism demand.," Tourism Managemen 29 (2008) pp: [8] Chung, C.F. and Baillie, R.T. "Small sample bias in condiional sum of squares esimaors of fracionally inegraed ARMA models," Empirical Economics, 18, (1993) pp: [9] Cichocki, A. and Unbehauen, R., "Neural neworks for opimizaion and signal processing," John Willey & Sons, New York, (1993). [10] Crouch, G.I., "The sudy of inernaional ourism demand: A review of pracice," Journal of Travel Research 33 (1994) pp: [11] Granger, C.W.J. and Joyeux, R., "An inroducion o long-memory ime series models and fracionally differencing," Journal of Time Series Analysis 1(1) (1980) pp: [12] Hasle, J. and Rafery, A.E., "Space-ime modeling wih long-memory dependence: assessing Ireland s wind power resources.," Applied Saisics 38 (1989) pp:1-50. [13] Hosking, J.R.M., "Fracionally differencing," Biomerika 68 (1) (1981) pp: [14] Hosking, J.R.M., Modeling persisence in hydrological ime series using fracionally differencing. Waer Resources Research 20 (12) (1984) pp: [15] Khashei M. and Bijari M., "A new hybrid mehodology for nonlinear ime series forecasing," Modelling and Simulaion in Engineering (doi: /2011/379121) (2011). [16] Khashei M. and Bijari M., "Hybridizaion of he probabilisic neural neworks wih feed forward neural neworks for forecasing," Engineering Applicaions in Arificial Inelligence (doi: /j.engappai ) (2012). [17] Kon, S.C. and Turner, W.L., "Neural nework forecasing of ourism demand. Tourism Economics," 11(2005) pp: [18] Law, R., "Back-propagaion learning in improving he accuracy of neural nework based ourism demand forecasing," Tourism Managemen 21 (2000) pp: [19] Law, R, "The impac of he Asian financial crisis on Japanese demand for ravel o Hong Kong: A sudy of various forecasing echniques.," Journal of Travel &Tourism Markeing 10 (2001) pp: [20] Lee Y.S., Tong L.I., "Forecasing ime series using a mehodology based on auoregressive inegraed moving average and geneic programming," Knowledge-Based Sysems 24(1) (2011) pp: [21] Li, W.K. and McLeod, A.I., Fracional ime series modeling. Biomerika 73 (1986) pp: [22] Li, G., Song, H. and Wi, S.F., Recen developmens in economerics modeling and forecasing. Journal of Travel Research, 44, pp: [23] Lim, C., "Review of inernaional ourism demand models," Annals of Tourism Research 24 (1997) pp: [24] Lim,C. "An economeric classificaion and review of inernaional ourism demand models," Tourism Economics 3 (1997) pp: [25] Lim,C., "A mea analysis review of inernaional ourism demand," Journal of Travel Research 37 (1999) pp: [26] Lo, A.W., "Long memory in sock marke prices," Economerica 59, (1991) pp: [27] Man, K.S., "Long memory ime series and shor erm forecass," Inernaional Journal of Forecasing 19 (3) (2003) pp: [28] Min Gan, Hui Peng, "Sabiliy analysis of RBF nework-based sae dependen auoregressive model for nonlinear ime series," Applied Sof Compuing 12(1) (2012)

6 American Journal of Inelligen Sysems 2012, 2(2): pp: [29] Pai, P.F., Hong, W.C., Chang, P.T. and Chen C.T., "The applicaion of suppor vecor machines o forecas ouris arrivals in Barbados: An empirical sudy," Inernaional Journal of Managemen, 23 (2006) [30] Pai, P.F. and Lin, C.S., "A hybrid ARIMA and suppor vecor machines model in sock price forecasing," The Inernaional Journal of Managemen Science 33(2005) pp: [31] Palmer, A., Jose Monano, J.J. and Sese, A., "Designing an arificial neural nework for forecasing ourism ime-series," Tourism Managemen 27 (2006) pp: [32] Purwano, Eswaran, C., Logeswaran, R., "An opimally configured hybrid model for healhcare ime series predicion," Asian Journal of Informaion Technology 10 (6) (2011) pp [33] Samsudin, R., Saad, P., Shabri, A., "A hybrid GMDH and leas squares suppor vecor machines in ime series forecasing," Neural Nework World 21 (3) (2011) pp [34] Smih, K.A., "Neural neworks in business: Techniques and applicaions," Imprin Info Hershey: Idea Group, (2002). [35] Song, H. and Li, G., "Tourism demand modeling and forecasing- A review of recen research.," Tourism Managemen 29 (2008) pp: [36] Sowell, F., "Maximum likelihood esimaion of saionary univariae fracionally inegraed ime series models," Journal of Economerics 53 (1992) pp: [37] Tang, Z., Almeida, C. and Fishwick, P.A., "Time series forecasing using neural neworks vs. Box-Jenkins mehodology," Simulaion 57 (1991) pp: [38] Tseng, F.M., Yu, H.C. and Tzeng, G.H. "Combining neural nework model wih seasonal ime series ARIMA model," Technological Forecasing and Social Change 69 (2002) pp: [39] Wi, S.F. and Wi, C.A., "Forecasing ourism demand: A review of empirical research," Inernaional Journal of Forecasing, 11 (1995) pp: [40] Zhang, G., Pauwo, B.E. and Hu, Y.M., Forecasing wih arificial neural neworks: The sae of he ar. Inernaional Journal of Forecasing 14 (1998) [41] Zhang, G., "Time series forecasing using a hybrid ARIMA and neural nework model," Neurocompuing, 50, (2003) [42] Zurada, J.M., "Inroducion of arificial neural sysems" S. Paul: Wes Publishing, (1992).