A study on the ability of Support Vector Regression and Neural Networks to Forecast Basic Time Series Patterns

Transcription

1 A study on the ablty of Support Vector Regresson and Neural Networks to Forecast Basc Tme Seres Patterns Sven F. Crone, Jose Guajardo 2, and Rchard Weber 2 Lancaster Unversty, Department of Management Scence, Lancaster LA 4YX, Lancaster, UK s.crone@lancaster.ac.uk 2 Unversty of Chle, Department of Industral Engneerng, Republca 70, Santago, Chle {jguajard,rweber}@d.uchle.cl Abstract. Recently, novel learnng algorthms such as Support Vector Regresson (SVR) and Neural Networks (NN) have receved ncreasng attenton n forecastng and tme seres predcton, offerng attractve theoretcal propertes and successful applcatons n several real world problem domans. Commonly, tme seres are composed of the combnaton of regular and rregular patterns such as trends and cycles, seasonal varatons, level shfts, outlers or pulses and structural breaks, among others. Conventonal parametrc statstcal methods are capable of forecastng a partcular combnaton of patterns through ex ante selecton of an adequate model form and specfc data preprocessng. Thus, the capablty of sem-parametrc methods from computatonal ntellgence to predct basc tme seres patterns wthout model selecton and preprocessng s of partcular relevance n evaluatng ther contrbuton to forecastng. Ths paper proposes an emprcal comparson between NN and SVR models usng radal bass functon (RBF) and lnear kernel functons, by analyzng ther predctve power on fve artfcal tme seres: statonary, addtve seasonalty, lnear trend, lnear trend wth addtve seasonalty, and lnear trend wth multplcatve seasonalty. Results obtaned show that RBF SVR models have problems n extrapolatng trends, whle NN and lnear SVR models wthout data preprocessng provde robust accuracy across all patterns and clearly outperform the commonly used RBF SVR on trended tme seres. Introducton Support Vector Regresson (SVR) and Artfcal Neural Networks (NN) have found ncreasng consderaton n forecastng theory, leadng to successful applcatons n tme seres and explanatory forecastng n varous domans, ncludng busness and management scence [, 2]. Methods form computatonal ntellgence promse

2 2 Sven F. Crone, Jose Guajardo and Rchard Weber attractve features to busness forecastng, beng data drven, sem-parametrc learnng machnes, permttng unversal approxmaton of arbtrary lnear or nonlnear functons from examples wthout a pror assumptons on the model structure, often outperformng conventonal statstcal approaches of ARIMA- or exponental smoothng- methods. Despte ther theoretcal capabltes, NN as SVR are not establshed forecastng methods n busness practce. Recently, substantal theoretcal crtcsm of NN has rased skeptcsm regardng ther ablty to forecast even smple tme seres patterns of seasonalty or trends wthout pror data preprocessng [3]. Whle all novel methods must ultmately be evaluated n an objectve experment usng a number of emprcal tme seres, adequate error measures and multple orgns of evaluaton [4], the fundamental questons to ther ablty to approxmate and generalze basc tme seres patterns must be evaluated beforehand. Tme seres can generally be characterzed by the combnaton of basc regular patterns: level, trend, season and resdual errors. For trend, a varety of lnear, progressve, degressve and regressve patterns are feasble. For seasonalty, an addtve or multplcatve combnaton wth level and trend further determnes the shape of the fnal tme seres. Consequently, we evaluate SVR and NN on a set of artfcally created tme seres derved from prevous publcatons. We evaluate the comparatve forecastng accuracy of each method to reflect ther ablty of learnng and forecastng fundamental tme seres patterns relevant to emprcal forecastng tasks. Ths paper s organzed as follows. Frst, we provde a bref ntroducton to SVR and NN n forecastng tme seres of observatons. Secton three presents the artfcally generated tme seres and the expermental desgn. Ths s followed by the expermental results and ther dscusson. Conclusons are gven n secton 4. 2 Modellng SVR and NN for Tme Seres Predcton 2. Support Vector Regresson We apply the common Support Vector Regresson (SVR) algorthm as proposed by Vapnk [5], whch uses an ε-nsenstve loss functon for predctve regresson problems. Ths functon allows a tolerance degree to errors not greater than ε. The descrpton s based on the termnology used n [6, 7]. Let {(x,y ),.., (x l,y l )}, where x є R n and y є R, be the tranng data ponts avalable to buld a regresson model. The SVR algorthm apples a transformaton functon Φ to the orgnal data ponts from the ntal Input Space, to a hgher-dmensonal Feature Space F. In ths new space, we construct a lnear model, whch corresponds to a non-lnear model n the orgnal space : When Φ s the dentty functon, the Feature Space s equvalent to the Input Space, and the model constructed s lnear n the orgnal space.

3 A study on the ablty of Support Vector Regresson and Neural Networks 3 Φ : R n F, w F f ( x) = w, Φ( x) + b The goal when usng the ε-nsenstve loss functon s to fnd a functon that fts current tranng data wth a devaton less or equal to ε, and at the same tme s as flat as possble. Ths means that one seeks for a small weght vector w; one way to do that s e.g. by mnmzng the quadratc norm of the vector w [6]. As ths problem could be nfeasble, slack varables ξ, ξ* are ntroduced to allow error levels greater than ε, arrvng to the formulaton proposed n [5]: l 2 * Mn w + C ( ξ + ξ ) 2 = s. t. y w, Φ( x ) b ε + ξ w, Φ( x ) + b y * ξ, ξ 0, =,2,..., l ε + ξ Ths s known as the prmal problem of the SVR algorthm. The objectve functon takes nto account generalzaton ablty and accuracy n the tranng set, and embodes the structural rsk mnmzaton prncple [8]. Parameter C measures the trade-off between generalzaton ablty and accuracy n the tranng data, and parameter ε defnes the degree of tolerance to errors. To solve the problem stated above, t s more convenent to represent the problem n ts dual form. For ths purpose, a Lagrange functon s constructed, and once applyng saddle pont condtons, t can be shown that the followng soluton s obtaned [8]: l * w = ( α α ) Φ( x ) l = * * f ( x) = ( α α ) K( x, x) + b = Here, α and α * are the dual varables, and the expresson K(x,x) represents the nner product between Φ(x ) and Φ(x), whch s known as the kernel functon [8]. The exstence of such a functon allows us to obtan a soluton for the orgnal regresson problem, wthout explctly consderng the transformaton Φ(x) appled to the data. In our experments we use radal bass functons (RBF) and lnear kernel functons. Lmted research has been conducted to nvestgate the ablty of SVR for predctng dfferent tme seres patterns. Experments performed by Hansen et. al [9] compare SVR performance wth 3 statstcal methods (e.g. ARIMA) on predctng 9 dfferent patterns present n real world tme seres. Among other patterns, they tred trends, seasonalty, cycles, and combnatons of them. Ther experments show SVR models outperformng the other methods on 8 of the 9 patterns; partcularly, they obtaned very good results usng SVR for extrapolatng lnear and non lnear trends. Guajardo et al. [0] compared SVR wth ARMAX models for predctng seasonal tme seres n a weekly sales forecastng doman for 5 dfferent products. Ther experments show that SVR were slghtly better than ARMAX models, succeedng n extrapolatng seasonal patterns (wthout trends) wth SVR.

4 4 Sven F. Crone, Jose Guajardo and Rchard Weber 2.2 Neural Networks Forecastng wth non-recurrent NN may encompass predcton of a dependent varable ŷ from lagged realzatons of the predctor varable y t n, or explanatory varables x of metrc, ordnal or nomnal scale as well as lagged realzatons thereof, xt, n. Therefore, NNs offer large degrees of freedom towards the forecastng desgn, permttng explanatory or causal forecastng through estmaton of a functonal relatonshp of the form yˆ = f ( x, x,..., x 2 z ), as well as general transfer functon models and smple tme seres predcton. Followng, we present a bref ntroducton to modellng NN for tme seres predcton; a general dscusson s gven n [, 2]. Forecastng tme seres wth NN s generally based on modellng the network n analogy to a non-lnear autoregressve AR(p) model [2, 3]. At a pont n tme t, a one-step ahead forecast y ˆ t+ s computed usng p=n observatons yt, yt, K, yt n+ from n precedng ponts n tme t, t-, t-2,, t-n+, wth n denotng the number of nput unts of the NN. Ths models a tme seres predcton as of ( y, y y ) ˆ = f,..., t + t t t n+ y. The archtecture of a feed-forward Multlayer Perceptron (MLP), a well researched NN paradgm, of arbtrary topology s dsplayed n fgure y ˆt+ y ˆt+ 2 y ˆt+ h Fg.. Autoregressve MLP applcaton to tme seres forecastng wth a MLP of arbtrary topology, usng n nput neurons for observatons n t, t-, t-2,, t-n-, m hdden unts, h output unts for tme perods t+, t+2,, t+h and a two layers of tranable weghts. The bas s dsplayed wthn the unts. Data s presented to the MLP as a sldng wndow over the tme seres observatons. The task of the MLP s to model the underlyng generator of the data durng tranng, so that a vald forecast s made when the traned NN s subsequently presented wth a new nput vector value.

5 A study on the ablty of Support Vector Regresson and Neural Networks 5 The network paradgm of MLP offers extensve degrees of freedom n modellng for predcton tasks. Structurng the degrees of freedom, each expert must decde upon the selecton and samplng of datasets, the degrees of data preprocessng, the statc archtectural propertes, the sgnal processng wthn nodes and the learnng algorthm n order to acheve the desgn goal, characterzed through the objectve functon or error functon. For a detaled dscusson of these ssues and the ablty of NN to forecast unvarate tme seres, the reader s referred to [2]. 3 Experments and Results 3. Descrpton of the Artfcal Tme Seres We evaluate a set of fve artfcal tme seres of monthly retal sales motvated from Pegel s orgnal classfcaton, later extended by Gardner to ncorporate degressve trends. Tme seres are composed of regular patterns of dfferent forms of lnear, progressve, degressve or regressve trends T, addtvely or multplcatvely combned wth seasonalty S, a constant level L and resdual nose E. In addton, emprcal tme seres are mpacted by rregular patterns such as level shfts and pulses, whch are dsregarded. To evaluate the ablty of dfferent computatonal ntellgence methods we create a set of benchmark tme seres for the most common regular tme seres patterns: lnear trend and dfferent forms of seasonalty. Consequently, we create ndvdual tme seres patterns and combne them accordngly, overlayng each wth addtve nose. No Trend (L) No Seasonalty (E) Addtve Seasonalty (S A ) Multplcatve Seasonalty (S M ) Lnear Trend (T L ) Fg. 2. Basc tme seres patterns of artfcal tme accordng to the Pegels- and Gardnerclassfcaton, combnng Level, Trend and Seasonalty wth a medum addtve nose level. In contrast to Pegel s classfcaton, a tme seres wth multplcatve seasonalty L+S M +E cannot dsplay an ncreasng seasonalty n the absence of level changes, t equals the pattern of addtve seasonalty and was consequently omtted from further analyss. Consequently, we create a set of fve tme seres ncludng a statonary tme seres L+E (E), seasonalty wthout trend L+S A +E (S A ), lnear trend L+T L +E (T L ), lnear trend wth addtve seasonalty L+T L +S A +E (T L S A ) and lnear trend wth multplcatve seasonalty dependng on the level of the tme seres L+T L *S M +E

6 6 Sven F. Crone, Jose Guajardo and Rchard Weber N applyng a medum level of nose σ 2 = 25. The orgnal tme seres data was taken from the experments of [3] and represent monthly retal sales. All tme seres consdered an addtve nose term to allow an estmaton of fnal forecastng accuracy n relatonshp to the orgnal nose level. Each tme seres conssts of 228 observatons. 2 (T L S M ). The resdual error term follows a Gaussan dstrbuton ( 0, σ ) 3.2 Expermental Desgn Ths research nvestgates whether the fve patterns descrbed above can be accurately predcted wth RBF SVR, Lnear SVR and NN models. For each seres, we defned a lag structure ncludng the 3 prevous observatons as attrbutes for predctng the next seres value (one perod ahead predcton); thus, a total of 25 data ponts reman to buld and parameterze models. Data was sequentally dvded nto tranng, valdaton and test sets usng 9, 48 and 48 observatons respectvely; tranng data s used to buld the model, valdaton data for parameter selecton purposes, and test data to evaluate the accuracy on a hold-out data set. All models are parameterzed usng only tranng and valdaton data, wthholdng all nformaton n the test set (also for scalng etc.) to assure vald ex ante testng. Data was transformed only by applyng lnear scalng nto a [-0.5, 0.5] nterval to avod saturaton effects, usng mnmum and maxmum values only from the tranng and valdaton data. No other preprocessng procedures such as deseasonalzaton or detrendng were carred out. As mentoned n secton 2.., SVR models requre settng of two parameters: C and ε. In addton, one needs to select an approprate kernel functon to carry out the transformaton to a hgher dmensonal feature space. The RBF kernel functon, whch s the kernel functon most wdely utlzed for regresson (see e.g. [6, 4, 5]), requres the defnton of an addtonal parameter σ. Our heurstc approach for RBF SVR parameter selecton can be summarzed as follows: - Frst, we determne startng values for the C and ε parameters on each tme seres by usng the emprcal rules proposed by Cherkassky and Ma [4], leadng to E {C= ; ε= }, S A {C= ; ε= }, T L {C= ; ε= }, T L S A {C=0.7464; ε= } and T L S M {C= ; ε= }. - Second, we search for good values of the RBF kernel parameter σ usng the predetermned parameters C and ε, and evaluate 45 dfferent alternatves for σ={0.00; 0.0; 0.03; 0.05; 0.08; 0.; 0.3; 0.5; 0.8; ;.3;.5;.8; 2; 2.3; 2.5; 2.8; 3; 3.3; 3.5; 3.8; 4; 4.3; 4.5; 4.8; 5; 5.3; 5.5; 5.8; 6; 7; 8; 9; 0; 5; 20; 25; 50; 80; 00; 200; 300; 400; 500; 000}. The value of σ whch generates the model wth the lowest mean absolute error (MAE) n the valdaton set s defned as the base parameter for the kernel functon. As result, we now have heurstc startng values for the three parameters of the SVR model, C, ε and σ. - Thrd, we defne a grd around base parameters C, ε and σ, and retan the best combnaton of parameters to be the fnal values used n the SVR model. In our experments, we tred fve dfferent values for each parameter C, ε, σ (factors

7 A study on the ablty of Support Vector Regresson and Neural Networks 7 0.5, 0.75,,.25 and.5 over the ntal values), thus creatng a grd of 25 possble parameter settngs. The parameter canddate of the grd s selected by usng the lowest MAE on the valdaton set as before. The scheme for Lnear SVR s very smlar, but wthout consderng parameter σ. Thus, second step for base parameter σ s not carred out, and the thrd step nvolves only 25 dfferent combnatons for C and ε. (for addtonal detals see [0]). For NN models, we used the backpropagaton algorthm to tran multple canddates of multlayer perceptron (MLP) networks. The network topology was obtaned usng a grd search of dfferent hdden nodes {0, 2,, 20} and actvaton functons {sgmod; tanh} wth fxed number of nput and output nodes, selectng the archtecture wth the lowest MAE on the valdaton set. The fnal model was ntalzed 20 tmes usng an (3-8-) archtecture comprsed of 3 nput nodes, 8 hdden nodes and a sngle output node for t+ predctons, applyng a sgmod transfer functon between the nput and hdden layers, and a lnear functon between hdden and output layers. As for SVR models, we selected the network wth the lowest valdaton mean absolute error (MAE) to calculate the test error results. 3.3 Expermental Results and Dscusson To evaluate our models we used the root mean squared error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). Test set errors obtaned usng SVR and MLP models for each one of the fve seres analyzed n ths paper are shown n Table. As can be seen from Table, RBF SVR has the best performance (denoted n bold) on a level tme seres supermposed wth whte nose (E) and addtve seasonalty (S A ) patterns across all error measures. Lnear SVR s the best method for predctng lnear trend (T L ) and lnear trend wth multplcatve seasonalty patterns (T L S M ), whle MLPs provde best results for lnear trends wth addtve seasonalty (T L S A ) pattern. Table. Forecastng accuracy on the test set for RBF and lnear SVR models and MLP Seres RBF SVR Lnear SVR MLP RMSE MAE MAPE RMSE MAE MAPE RMSE MAE MAPE Seres E Seres S A Seres T L Seres T L S A Seres T L S M Sum Snce we evaluated artfcally constructed tme seres we can estmate the part of the forecastng errors caused by the artfcally created nose, whch due to ts random nature cannot be forecasted. Ths permts an analyss to what extent each method was capable of separatng nose from structure of varyng complexty on the unbased error measure of MAE. In applyng the true mean of the Gaussan resduals

8 8 Sven F. Crone, Jose Guajardo and Rchard Weber as an optmal forecast, we estmate a MAE of 3.80 as a lower bound forecast error for all tme seres on the test set. It becomes apparent, that RBF SVR exceeds even a perfect forecast for seres E and S A, whch can be attrbuted to the randomness of the data nherent n all ex ante evaluatons of forecastng experments. In contrast, RBF SVR sgnfcantly underperform on trended tme seres patterns, ndcatng nadequaces of the chosen kernel functon. On the contrary, lnear SVR shows a more robust predcton of all tme seres patterns. As the forecasts devates only slghtly from the lower bound n comparson to the level of the tme seres, as would be reflected n the MAPE, SVR wth lnear kernel functons may be consdered a robust method n forecastng arbtrary tme seres patterns wthout preprocessng. Smlarly, MLPs forecast all tme seres patterns robustly and wthout preprocessng wth a comparatve hgh accuracy close to lnear SVR and the lower bound. In summarzng over all tme seres, applyng an equal weght to each of the tme seres patterns, MLPs robustly outperform RBF SVR on all three error measures of MAE, MAPE and RMSE, whereas MLPs also moderately outperform lnear SVR. Ths ndcates that whle partcular kernel functons enable the SVR to outperform alternatve parameterzatons, MLPs or lnear SVR may prove a more robust alternatve n usng a sngle method to forecast a set of tme seres of dfferent patterns. In addton to these dstance based error measures, we evaluate the relatve performance by rankng each method by the ndvdual error measure, provded n Table 2. Table 2. Forecastng accuracy measured by ranks of methods for each error measure Rank by RMSE Rank by MAE Rank by MAPE SVR SVR MLP SVR SVR MLP SVR SVR MLP RBF lnear RBF lnear RBF lnear Seres E Seres S A Seres T L Seres T L S A Seres T L S M Sum of Ranks The fndngs by ranked error measures confrm lttle dfferences between lnear SVR and MLPs, wth MLPs provdng the best results for the lmted test desgn provded across all error measures. SVR wth RBF kernel, the most frequently used mplementaton n tme seres predcton wth SVR to date, performs sgnfcantly worse than the other methods. As must be expected, dfferent error measures dentfy dfferent best methods. In partcular, RMSE and MAPE are consdered to be based error measures. To lmt bases n the absence of a true objectve functon whch could motvate the use of a partcular error measure, we assume equal weght to each error and focus our conclusons on the MAE. To confrm the results of model accuracy from a statstcal pont of vew, we performed a pared-samples t test on the absolute values of the resduals over the test set data ponts. Results obtaned show that dfferences between

9 A study on the ablty of Support Vector Regresson and Neural Networks 9 model errors are statstcally sgnfcant when comparng RBF SVR to Lnear SVR (t=7.337; df=239; p<0.00), and RBF SVR to NN (t=6.999; df=239; p<0.00), although not when comparng NN to Lnear SVR (t=-0.989; df=239; p=0.324). Ths ndcates that no sgnfcant dfference n forecastng accuracy between the methods of lnear SVR and MLP may be derved from these experments. Consequently, we need to extend ths evaluaton on addtonal tme seres and varatons of MLPs. Results suggest that RBF SVR can predct seasonal patterns but no trends, whle lnear SVR and NN seem to be able to extrapolate trend as well as seasonal patterns accurately and wthout preprocessng. By examnng the resduals of the models, t can be observed that RBF SVR systematcally underestmate hold-out sample observatons for trended seres, whch corresponds to saturaton effects. 4 Concluson We have examned the ablty of RBF SVR, lnear SVR and MLP for predctng fve basc artfcal tme seres patterns: statonary, seasonalty, lnear trends, lnear trend wth addtve seasonalty, and lnear trend wth multplcatve seasonalty. Results obtaned usng multple error measures show that whle RBF SVR outperform other methods on non-trended data, they do not provde robust results across all patterns. For tme seres wth trend components, lnear SVR and MLP sgnfcantly outperform RBF SVR models, whch severely underestmate out-of-sample observatons, consstently laggng behnd upward trends. RBF SVR errors have shown to be statstcally sgnfcantly hgher than lnear SVR and NN errors. MLP demonstrate robust performance, provdng the hghest overall forecastng accuracy n across tme seres and dfferent statstcal error measures and rank based metrcs. Our results confrm prevous fndngs by Guajardo et. al [0], demonstratng accurate forecasts of seasonal tme seres wthout trends usng RBF SVR, even outperformng establshed statstcal methods such as ARIMAX. Also, they confrm results by Hansen et. al [9], who accurately predcted both lnear and nonlnear trends usng SVR, outperformng other methods such as ARIMA on several patterns. We assume that Hansen et al. also used lnear kernels, as they dd not fully document the kernel functons appled. A prelmnary hypothess for our poor results obtaned wth RBF SVR n extrapolatng trend patterns les n the lnear nature of ths trend. Prevous publcatons report smlar problems of closely related RBF-neural networks n predctng trends and nstatonary tme seres. Whle SVR wth lnear kernel functons and MLP wth lnear actvaton functons n the output unts may be partcularly suted to extrapolate lnear trends, we dd not conduct experments as to ther ablty to extrapolate non-lnear trends. These ssues wll be evaluated n an extended set of experments currently under nvestgaton by the authors, ncreasng the number of tme seres patterns and consderng addtonal knds of trend patterns, also evaluatng results aganst establshed statstcal forecastng methods as benchmarks. Addtonally, we wll

10 0 Sven F. Crone, Jose Guajardo and Rchard Weber evaluate the nfluence of preprocessng procedures such as deseasonalzaton to evaluate alternatve perspectves on the problem of extrapolatng tme seres patterns. Acknowledgement: Ths work has been supported n part by the Mllennum Nucleus Complex Engneerng Systems ( References. K. P. Lao and R. Fldes, The accuracy of a procedural approach to specfyng feedforward neural networks for forecastng, Computers & Operatons Research 32 (8) (2005) G.P. Zhang, B.E. Patuwo, and M.Y. Hu, Forecastng wth artfcal neural networks: The state of the art, Internatonal Journal of Forecastng,, 4, (998) 3. G.P. Zhang and M. Q, Neural network forecastng for seasonal and trend tme seres, European Journal of Operatonal Research 60, (2005). 4. L. J. Tashman, Out-of-sample tests of forecastng accuracy: an analyss and revew, Internatonal Journal of Forecastng 6 (4) (2000) V.Vapnk, The nature of statstcal learnng theory (Sprnger, New York, 995). 6. A.J. Smola and B. Schölkopf, A Tutoral on Support Vector Regresson, NeuroCOLT Techncal Report NC-TR , 998 (Royal Holloway College, Unversty of London, UK). 7. K. Müller, A. Smola, G. Rätsch, B. Schölkopf, J. Kohlmorgen, and V. Vapnk, n: Advances n Kernel Methods: Support Vector Learnng/ Usng Support Vector Machnes for Tme Seres Predcton, edted by B. Schölkopf, J. Burges, and A. Smola (MIT Press, 999), pp V.Vapnk, Statstcal Learnng Theory (John Wley and Sons, New York, 998). 9. J.V. Hansen, J.B. McDonald, and R.D. Nelson, Some evdence on forecastng tme-seres wth Support Vector Machnes, Journal of the Operatonal Research Socety,, -, J. Guajardo, J. Mranda, and R. Weber, A Hybrd Forecastng Methodology usng Feature Selecton and Support Vector Regresson, 5 th Internatonal Conference on Hybrd Intellgent Systems HIS 2005 (Ro de Janero, Brazl, 2005), pp C. M. Bshop, Neural networks for pattern recognton. Clarendon Press; Oxford Unversty Press, Oxford, S. Haykn, Neural networks: a comprehensve foundaton, 2nd ed. Prentce Hall, Upper Saddle Rver, N.J., A. Lapedes, R. Farber, and Los Alamos Natonal Laboratory, Nonlnear sgnal processng usng neural networks: predcton and system modellng, Los Alamos Natonal Laboratory, Los Alamos, N.M. LA-UR , V. Cherkassky and Y. Ma, Practcal selecton of SVM parameters and nose estmaton for SVM regresson, Neural Networks 7(), 3-26 (2004). 5. D. Mattera and S. Haykn, n: Advances n Kernel Methods: Support Vector Learnng/ Support Vector Machnes for Dynamc Reconstructon of a Chaotc System, edted by B. Schölkopf, J. Burges and A. Smola (MIT Press, 999), pp