DESIGN A NEURAL NETWORK FOR TIME SERIES FINANCIAL FORECASTING: ACCURACY AND ROBUSTNESS ANALISYS

Transcription

1 DESIGN A NEURAL NETWORK FOR TIME SERIES FINANCIAL FORECASTING: ACCURACY AND ROBUSTNESS ANALISYS LEANDRO S. MACIEL, ROSANGELA BALLINI Insiuo de Economia (IE), Universidade Esadual de Campinas (UNICAMP) Rua Piágoras, 65 Cidade Universiária Zeferino Vaz CEP Campinas São Paulo Brasil s: [email protected]; [email protected] ABSTRACT Neural Neworks are an arificial inelligence mehod for modeling complex arge funcions. For cerain ypes of problems, such as learning o inerpre complex realworld sensor daa, Arificial Neural Neworks (ANNs) are among he mos effecive learning mehods currenly know. During he las decade hey have been widely applied o he domain of financial ime series predicion and heir imporance in his field is growing. This paper aims o analyze he neural neworks for financial ime series forecasing. Specifically he abiliy o predic fuure rends of Norh American, European and Brazilian Sock Markes. Accuracy is compared agains a radiional forecasing mehod, generalized auoregressive condiional heeroscedasiciy (GARCH). Furhermore, i is examined he bes choice of nework design for each sample of daa. I was concluded ha ANNs do have he capabiliy o forecas he sock markes sudied and, if properly rained, can improve he robusness according o he nework srucure. Key words: Arificial Neural Neworks, Finance Forecasing, Economic Forecasing, Sock Markes. 1. INTRODUCTION There is a long hisory of research on finance and economic modeling. Time series analysis is one of he mos widely used radiional approaches in his field. There are wo kinds of models o describe he behavior of ime series as follows. The firs are he Linear Models: A linear approach o ime series analysis is ypically effeced hrough one of he following echniques: (a) Box-Jenkins echniques, (b) Kalman filers, (c) Brown s heory of exponenial smoohing, (d) piecewise regression. The second are he Nonlinear Models: (a) Taken s Theorem, (b) he Mackey-Glass equaion. These echniques aemp o reconsruc he ime series based upon a sampling of he daa in order o forecas fuure values. Alhough hese echniques are saisically powerful, hey produce low success raes when hey are used o forecasing financial markes. Recen evidence shows ha financial markes are nonlinear; however, hese linear mehods sill provide good ways of describing nonlinear sysems found in he financial marke ime series analysis (Fang e al., 1994). Bollerslev (1986) provide an excellen survey of he exisence of nonlineariies in he financial daa, and developed a model o predic financial ime series called Generalized Auoregresssive Condiional Heerocedasiciy (GARCH) ha combines all he feaures observed in hese series. Bu, he economy is evolving (raher slowly) over ime. This feaure canno easily be capured by fixed specificaion linear models, however, and manifess iself in he form of an evolving coefficien esimae. Many facors inerac in finance and economics

2 including poliical evens, general economic condiions, and rader s expecaions. Therefore, predicing finance and economics movemens is quie difficul. Arificial Neural Neworks (ANNs) are a very powerful ool in modern quaniaive finance and have emerged as a powerful saisical modeling echnique. ANNs provide an aracive alernaive ool for boh researches and praciioners. They can deec he underlying funcional relaionships wihin a se of daa and perform asks such as paern recogniion, classificaion, evaluaion, modeling, predicion and conrol (Anderson and Rosenfeld, 1988; Hech-Nielsen, 1990; Herz e al., 1991; Hiemsra and Jones, 1994). Several disinguishing feaures of ANNs make hem valuable and aracive in forecasing. Firs, ANNs are nonlinear daa-driven. They are capable o perform nonlinear modeling wihou an a priori knowledge abou he relaionships beween inpu and oupus variables. The non-parameric ANN model may be preferred over radiional parameric saisical models in siuaions where he inpu daa do no mee he assumpions required by he parameric model, or when large ouliers are eviden in daase (Lawrence, 1991; Rumelhar and Mcclelland, 1986; Waie and Hardenbergh, 1998; Wasserman, 1993). Second, ANNs are universal funcions approximaion. I has been shown ha a neural nework can approximae any coninuous funcion o any desire accuracy (Hornik, 1993; Hornik e al., 1989). Third, ANNs can generalize. Afer learning he daa presened o hem, ANNs can ofen correcly infer he unseen par of a populaion even if he sample daa conain noisy informaion. Neural Neworks are able o capure he underlying paern or auocorrelaion srucure wihin a ime series even when he underlying law governing he sysem is unknown or oo complex o describe. Some aricles have reviewed journal aricles on how ANNs can be applied o finance and economic. Wong and Selvi (1998) classified he aricles by year of publicaion, applicaion area, journal, various decision characerisics (problem domain, decision process phase, level of managemen, level of ask inerdependence), means of developmen, inegraion wih oher echnologies, and major conribuion. Zang e al. (1998) surveyed aricles ha addressed modeling issues when ANNs are applied o forecasing. They summarized he mos frequenly cied advanages and disadvanages of he ANN models. Chaerjee e al. (2000) provided an overview of he ANN sysem and is wide-ranging used in he financial markes. Their work has furher discussed he superioriy of ANN over radiional mehodologies. The sudy concluded wih a descripion of he successful use of ANN by various financial insiuions. Edward Gaely, in his book, Neural Neworks for Financial Forecasing, describes he general mehodology required o build, rain, and es a neural nework using commercially available sofware. In his paper we aim o analyze and examine he use of neural neworks o predic fuure rends of Norh American, European and Brazilian Sock Markes Indexes. The indexes are: Dow Jones and S&P 500 (Unied Saes), DAX (Germany), CAC40 (France), FTSE (UK), IBEX35 (Spanish), PSI20 (Porugal) and IBOVESPA (Brazil). We hope o give he deailed discussion of he applicaion of a neural neworks ool o forecasing sock markes economic indicaors. As a comparison, we analyze GARCH model applied o each series o evaluae he accuracy of ANNs. A discussion abou how ANNs can incorporae he heeroscedasiciy of financial ime series was performed o verify he robusness of he model. This paper is organized as follows. Secion 2 discusses applicaions o sock marke index prices forecasing wih neural neworks. Secions 3-4 describe GARCH and Neural Neworks models respecively. Secion 5 shows he srucure of neural nework applied. Performance comparison beween he mehods is described in Sec. 6. Finally, conclusions are given in Sec. 7.

3 2. APPLICATIONS STOCK MARKET INDEX FORECASTING The sock marke is one of he mos popular invesmens owing o is highexpeced profi. However, he higher expeced profi, he higher is he risk implied. The sock marke, which has been invesigaed by various researches, is a raher complicaed environmen. There are hree degrees of marke efficiency. The srong form of he efficien markes hypohesis sae ha all informaion ha is knowable is immediaely facored ino he marke price for a securiy. If his is rue, hen all of hose price predicors are definiely wasing heir ime, even if hey have access o privae informaion. In he semi-srong form of he efficien markes hypohesis, all public informaion is considered o have been refleced in price immediaely as i became known, bu possessors of privae informaion can use ha informaion for profi. The weak form holds only ha any informaion gained from examining he securiy pas rading hisory of refleced in price. Of course, he pas rading hisory is public informaion implying ha he weak form is a specializaion of he semi-srong form, which iself is a specializaion of he srong form of he efficien markes hypohesis. Sock marke flucuaions are he resul of complex phenomena, whose effec ranslaes ino a blend of gains and losses ha appear in a sock marke ime series ha is usually prediced by exrapolaion. The periodic variaions follow eiher seasonal paerns of he business cycle in he economy. Shor-erm and day-o-day variaions appear a random and are difficul o predic, bu hey are ofen he source for sock rading gains and losses, especially in he case of day raders. Numerous invesigaions gave rise o differen decision suppor sysems for he sake of providing he invesors wih an opional predicion. Many expers in he sock markes have employed he echnical analysis for beer predicion for a long ime. Generally speaking, he echnical analysis derives he sock movemen from he sock s own hisorical value. The hisorical daa can be used direcly o form he suppor level and he resisance or hey can be plugged ino many echnical indicaors for furher invesigaion. Convenional researches addressing his research problem have generally employed he ime series analysis echniques (i.e. mixed auo regression moving average (ARMA)) as well as muliple regression models (Huang e al., 2005). Considerable evidence exiss and shows ha sock marke price is o some exen predicable (Lo and Mackinlay, 1988). 2.1 DISCUSSION OF INPUT VARIABLES There are wo kinds of heoreical approaches o deermine he inpu variables for he sock marke index forecasing wih neural neworks. The firs one inroduces he relaionship among he sock marke index price and oher macroeconomic indicaors. The second one inroduces nonlineariy in he relaion among sock prices, dividends and rading volume. Chen (1991) sudied he relaion beween changes in financial invesmen opporuniies and change in he economy. This paper provided addiional evidence ha variables such as he defaul spread, erm spread, one-monh T-bill rae, lagged indusrial producion growh rae and dividend-price raio are imporan deerminans of he fuure sock marke index. This sudy inerpreed he abiliy of hese variables o forecasing he fuure sock marke index in erms of heir correlaions wih changes in he macroeconomic environmen. Fama and French (1993) idenified hree common risk

4 facors: he overall marke facor, facors relaed o firm size and book-o-marke equiy, which seem o explain average reurns on socks and bonds. Ferson and Schad (1996) showed ha he omission of variables like lagged sock index and previous ineres raes could lead o misleading resuls. Sie and Sie (2000) discussed he predicive abiliy of ime delay neural neworks for he S&P 500 index ime series. The vecor auoregression (VAR) mehod is mainly used o invesigae he relaionship beween variables. Is advanage in ha muliple variables can be invesigaed a he same ime and he inerdependence can be esed auomaically wih he sophisicaed saisically significance level. Ao (2003a; 2003b) found ha: (1) HK depends on is pas price, JP, Nasdaq, S&P and DJ; (2) AU depends on he pas price, S&P and DJ; (3) depends on is pas price, HK, Nasdaq, S&P and DJ; (4) JP depends on is pas price, Nasdaq, S&P and DJ; (5) DJ depends on is pas price and Nasdaq; (6) S&P depends on is pas price and Nasdaq. The resuls from he VAR modeling sugges ha, for he Asian markes, he relevan informaion is he own hisorical value as well as he sock movemens from he US markes. I is also posiive o know he exen and ime-dependan naure of he markes dynamic when we draw he correlaion diagram of he local marke wih he US markes. Furher invesigaion can ell us ha, a he ime of low correlaion like he lae 90s of he Asian Financial crisis, he Hong Kong marke (and similarly oher Asian markes) is dominaed by he local evens like he currency problems. A oher periods, he local marke is grealy correlaed wih he US markes. In summary, he se of poenial macroeconomic indicaors are as follows: erm srucure of ineres raes (TS), shor erm ineres rae (ST), long erm ineres rae (LT), consumer price index (CPI), indusrial producion (IP), governmen consumpion (GC), privae consumpion (PC), gross naional produc (GNP), gross domesic produc (GDP). These are he mos easily available inpu variables ha are observable o a forecaser. Though oher macroeconomic variables can be used as inpus, he general consensus in he lieraure is ha he majoriy of useful informaion for forecasing is subsumed by ineres raes and he lagged predicive variable. The erm srucure of ineres raes, i.e. he spread of long-erm bond yields over shor-erm bond yields, may have some power in forecasing he sock index. In his paper we uilized as inpu variables he hisorical daa of each series sudied 1, which are showed in Table 1. The goal is o analyze he influence of he ANNs srucure in he resuls of forecasing. Afer chosen he beer srucure we compared he performance of he ANNs mehod and GARCH model. Table 1 Sock Marke Indexes ha form he Sample. Counry Index Variable Unied Saes Dow Jones DOW Unied Saes S&P 500 S&P Germany DAX DAX France CAC40 CAC Unied Kingson FTSE FTSE Spanish IBEX35 IBEX Porugal PSI20 PSI Brazil IBOVESPA IBOV 1 The daa was obained in hp://finance.yahoo.com/ URL accessed on Augus 17, 2008

5 3. GARCH MODELS The ARIMA models have one severe drawback: hey assume ha he volailiy 2 of he variable being modeled (e. g. sock price) is consan over ime. In many cases his is no rue. Large differences (of eiher sign) end o be followed by large differences. In oher words, he volailiy of asse reurns appears o be serially correlaed (Campbell e al., 1997). ARCH (Auoregressive Condiional Heerocedasiciy) model were developed in order o capure his propery of financial ime series. The ARCH 3 process is defined as ARCH (q): y q i= 1 = σ ε (1) σ = α 0 + α y (2) i 2 i where σ is he condiional sandard deviaion of y given he pas values of his process. The ARCH(q) process is uncorrelaed and has a consan mean, a consan uncondiional variance ( α 0 ), bu is condiional variance is nonconsan. This model has a simple inuiive inerpreaion as a model for volailiy clusering: large values of pas 2 squared reurns ( y i ), give rise o a large curren volailiy (Marin, 1998). The ARCH(q) model is a special case of he more general GARCH(p,q) model defined as (GARCH means Generalized ARCH ) GARCH(p,q): y p q = α0 αi y i β jσ j i= 1 j= 1 = σ ε (3) σ (4) In his model, he volailiy oday depends upon he volailiies for he previous q days and upon he squared reurns for he previous p days. A long and vigorous line of research followed he basic conribuions of Engle and Bollerslev (developers of ARCH and GARCH model respecively), leading a number of varians of he GARCH(p,q) model. These include power GARCH (PGARCH) models, exponenial GARCH (EGARCH) models, hreshold GARCH (TGARCH) model and oher models ha incorporae so-called Leverage effecs. Leverage erms allow a more realisic modeling of he observed asymmeric behavior of reurns according o which a good-news price increase yields lower subsequen volailiy, while bad-news decrease in price yields a subsequence increase in volailiy. I is also worh menioning wo-componen GARCH models which reflec differing shor erm and long erm volailiy dynamics, and GARCH-in-he-mean (GARCH-M) models which allow he mean value of reurns o depend upon volailiy (MARTIN, 1998) 4. 2 Volailiy is he synonym of sandard deviaion. 3 This secion is based upon Rupper (2001). 4 In his work was uilized GARCH(1,1) model as a resul of correlaion and auocorrelaion analysis.

6 4. NEURAL NETWORKS Neural Nework learning mehods provide a robus approach o approximaing real-valued, discree-valued and vecor-value arge funcions. For cerain ypes of problems, such as learning o inerpre complex real-world sensor daa, arificial neural neworks (ANNs) are among he mos effecive learning mehods currenly known (Michell, 1997). The sudy of ANNs has been inspired in par by he observaion ha biological learning sysems are buil of very complex webs of inerconneced neurons. In rough analogy, ANNs are buil ou of a densely inerconneced se of sample unis, where each uni akes a number of real-valued inpus (possibly he oupus of oher unis) and produces a single real-valued oupu, which may become inpu o oher unis (Michell, 1997). One moivaion for ANN sysems is o capure his kind of highly parallel compuaion based on disribued represenaions. Mos ANN sofware runs on sequenial machines emulaing disribued processes, alhough faser versions of he algorihms have also been implemened on highly parallel machines and on specialized hardware designed specifically for ANN applicaions. 4.1 BASIC DEFINITIONS The srucure of an arificial nework of mos commonly used ype is he mulilayer perceprons. I consiss of several layers of processing unis (also ermed neurons or nodes). The inpu values (inpu daa) are fed o he neurons in he so-called inpu layer. The inpu values are processed wihin he individual neurons of he inpu layer and hen he oupu values of hese neurons are forwarded o he neurons in he hidden layer. Each connecion has an associaed parameer indicaing he srengh of his connecion, he so-called weigh. By changing he weighs in a specific manner, he nework can learn o map paerns presened a he inpu layer o arge values on he oupu layer. This descripion of he procedure, by means of which his weigh adapaion is performed, is called learning or raining algorihm. Usually, he daa available for raining he nework is divided in (a leas) wo non-overlapping pars: he so-called raining and esing ses. The commonly large raining se is used o each he nework o desire arge funcion. Then he nework is applied o daa in he es se in order o available is generalizaion abiliy, i. e. he abiliy o derive correc conclusions abou he populaion properies of he daa from he sample properies of he raining se (e. g. if a nework has o learn a sine funcion, i should produce correc resuls for all real numbers and no only for hose in he raining se). If he nework is no able o generalize, bu insead learns he individual properies of he raining paerns wihou recognizing he general feaures of he daa (i. e. produce correc resuls for raining paerns, bu has a high error rae in he es se), i is said o be overfied or o be subjec o overfiing. 4.2 PROPERTIES OF NEURAL NETWORKS ANN learning is well suied o problems in which he raining daa corresponds o noisy, complex sensor daa, such as inpus from cameras and microphones. I is also

7 applicable o problems for which more symbolic represenaions are ofen used, such as he decision ree learning asks. In his case ANN and decision ree learning produce resul of comparable accuracy (Haykin, 2001). The backpropagaion algorihm is he mos commonly used ANN learning echnique. I is appropriae for problems wih he following characerisics (Michell, 1997): Insances are represened by many value pairs. The arge funcion o be learned is defined over insances ha can be described by a vecor of predefined feaures, such as he pixel values. These inpu aribues may be highly correlaed or independen of one anoher. Inpu values can be any real values. The arge funcion oupu may be discree-valued, real-valued, or a vecor of several real- or discree-valued aribues. The raining examples my conain errors. ANN learning mehods are quie robus o noise in he raining daa. Long raining imes are accepable. Nework raining algorihms ypically require longer raining imes han, say, decision ree learning algorihms. Training imes can range from a few seconds o many hours, depending on facors such as he number of weighs in he nework, he number of raining examples considered, and he seings of various learning algorihm parameers. Fas evaluaion of he learning arge funcion may be required. Alhough ANN learning imes are relaively long, evaluaing he learning nework, in order o apply i o a subsequen insance, is ypically very fas. The abiliy of humans o undersand he learning arge funcion is no imporan. The weighs learned by neural neworks are ofen difficul for humans o inerpre. Learned neural neworks are less easily communicaed o humans han learned rules. 4.3 MULTI-LAYER PERCEPTRONS The nework consiss of a se of nodes ha consiue he inpu layer, one or more hidden layers of nodes and an oupu layer of nodes. The inpu propagaes hrough he nework in a forward direcion, on a layer-by-layer basis. These neural neworks are referred o as mulilayer perceprons (MLPs). In he 1960s here was a grea euphoria in he scienific communiy abou ANN based sysems ha were promised o deliver breakhroughs in many fields. Single-layer neural neworks such as ADALINE 5 were used widely, e. g. in he domain of signal processing. This euphoria was given an end by he publicaion of Minsky and Paper (1969), who showed ha he ANNs used a ha ime were no capable of approximaing arge funcions wih cerain properies (arge funcions ha are no linearly separable such as he exclusive or (XOR) funcion). In he 1970s only a small amoun of research was devoed o ANNs. In he mid-1980s, he ANNs were revived by employmen of he error back-propagaion (EBP) learning algorihm in combinaion wih muli-layer neworks (Rumelhar and McClelland, 1986). Basically, he error back-propagaion process consiss of wo phases hrough he differen layers of he nework: a forward pass and a backward pass. In he forward pass, an inpu vecor is applied o he nodes of he nework, and is effec propagaes hrough he nework, layer by layer. Finally, a se of oupus is produce as he acual response of 5 ADALINE = ADApive LInear NEuron.

8 he nework. During his phase he weighs are all fixed. During he backward pass he weighs are all adjused in accordance wih he error-correcion rule. Specifically, he acual response of he nework is subraced from a desired response o produce an error signal. This error is propagaed backward hrough he nework, agains he direcion of synapic connecions hence he name error back-propagaion. The synapic weighs are adjused so as o make he acual response of he nework move closer o desired response (Haykin, 2001). MLP nework consiss of a leas hree layers: inpu layer, one or more hidden layers and oupu layer. The nodes are conneced by links associaed o real number named weighs. Each node akes muliple values as inpu, processes hem, and produces an oupu, which can be forwarded o oher nodes. Given a node j, is oupu is equal o ( ( x jiw ) o (5) j = ransfer ji where o j is he oupu of node j, x ji is he ih inpu o uni j, w ji he weigh associaed wih ih inpu o j and ransfer is he non-linear ransfer funcion responsible for ransferring he weighed sum of inpus o some value ha is given o he nex node 6. A neuron may have an arbirary number of inpus, bu only one oupu. By changing he weighs of he links connecing nodes, he ANN can be adjused for approximaing a cerain funcion. 4.4 LEARNING ALGORITHMS Usually, he weighs of he ANN mus be adjused using some learning algorihm in order for he ANN o be able o approximae he arge funcion wih a sufficien precision. In his secion is presened sochasic gradien descen backpropagaion learning algorihm as follows 7. The erm neural nework refers o a MLP rained wih his learning algorim, ofen called back-propagaion or error back-propagaion (EBP). Assume an ANN uses he following error funcion E( w r ) = 1 2 ( kd o kd ) d D k oupus 2 (6) where o kd is he oupu value produced by oupu neuron k, kd he desire (correc) value his neuron should produce and D denoes he se of all raining paerns, i. e. E(w r ) is he sum of predicion error for all raining examples. Predicion errors of individual raining examples are in urn equal o he sum of he differences beween oupu values produced by he ANN and he desire (correc) values, where w r is he vecor conaining he weighs of he ANN. The goal of a learning algorihm is o minimize E(w r ) for a paricular se of raining examples. There are several ways o achieve his, one of hem being he socalled gradien descen mehod. Basically, i works in he following way (Schraudolph and Cummins, 2002): 6 There are a several ypes of ransfer funcions and hey can be seen in Haykin (2001). 7 See more learning algorihms in Haykin (2001).

9 1. Choose some (random) iniial values for he model parameers. 2. Calculae he gradien G of he error funcion wih respec o each model parameer. 3. Change he model parameers so ha we move a shor disance in he direcion of he greaes rae of decrease of he error, i. e., in he direcion of G. 4. Repea seps 2 an 3 unil G ges close o zero. Le G f ( x) = he gradien of funcion f is he vecor of firs parial derivaives ( x) f ( x) f ( x) f f ( x) =,,..., x1 x2 x n r In our case, G = E( w) (i. e. he derivaive of he error funcion E wih respec o he weigh vecor w r ). Having his in mind, we will now explore he gradien descen back-propagaion (error back-propagaion) learning algorihm. Firs, a neural nework is creaed and he parameers are iniialized (he weighs are se o small random numbers). Then, unil he erminaion condiion (e.g. he mean squared error of ANN is less han a cerain error hreshold) is me, all raining examples are augh he ANN. Inpus of each raining example are fed o he ANN, and processed from he inpu layer, over he hidden layer(s) o oupu layer. In his way, vecor o of oupu values produced by he ANN is obained. In he nex sep, he weighs of he ANN mus be adjused. Basically, his happens by moving he weigh in he direcion of seepes descen of he error funcion. This happens by adding o each individual weigh he value w = ηδ x (8) j ji where η is he learning rae ha deermines he size of he sep ha we use for moving owards he minimum of E, and δ j represens he error erm of neuron j 8. The learning rae can be hrough of as he lengh of he arrows 9. There are many improvemens of his algorihms such as momenum erm, weigh decay ec described in he appropriaed lieraure (Bishop, 1996). Neverheless, MLP in combinaion wih sochasic gradien descen learning algorihm is he mos popular ANN used in pracice 10. Anoher imporan feaure of his learning algorihm is ha i assumes a quadraic error funcion, hence i assumes here is only one minimum. In pracice, he error funcion can have apar from he global minimum muliple local minima. There is a danger for he algorihm o land in one of he local minima and hus no be able o reduce he error o highes exen possible by reaching a global minimum. Nex secion shows an ANN design o our daa, and sep by sep a comparison wih GARCH model. (7) 8 Sochasic gradien descen backpropagaion learning algorihm derivaive can be see in Haykin (2001). 9 Usually, η R, slow convergence of he ANN. 10 This srucure was uilized in he presen work. <η. Noe ha oo large η leads o oscillaion around he minimum, while o small η can lead o a

10 5. DESIGN OF ANN IN STOCK MARKET FORECASTING The mehodology described in his secion is based upon Kaasra and Boyd (1996). The design of a neural nework successfully predicing a financial ime series is a complex ask. The individual seps of his process are lised bellow: 1. Variable Selecion 2. Daa Collecion 3. Daa Preprocessing 4. Daa Pariioning 5. Neural Nework Design 6. Training ANN Deailed descripion of each sep is presened below. 5.1 VARIABLE SELECTION Success in designing a neural nework depends on a clear undersanding of he problem (Nelson and Illingworh, 1991). Knowing which inpu variables are imporan in he marke being forecased is criical. This is easier said han done because he very reason for relaying on a neural is for is powerful abiliy o deec complex nonlinear relaionships among a number of differen variables. However, economic heory can help in choosing variables which are likely imporan predicors. A his poin in design process, he concern is abou he raw daa from which a variey of indicaors will be developed. These indicaors will from he acual inpus o he neural neworks (Kaasra and Boyd, 1996). The financial researcher ineresed in forecasing marke prices mus decide wheher o use boh echnical and fundamenal economic inpus from one or more markes. Technical inpus are defined as lagged 11 values of dependen variable 12 or indicaors calculed from he lagged values. The model applied in his paper uses lagged values of he dependen variables as a resul of correlaion and auocorrelaion analysis 13. Table 2 shows inpu srucures performed for each daa uilized. 11 Lagged means an elemen of he ime series in he pas. For example, a ime, he values y, 1 y 2, y p are said o be lagged values of he ime series y. 12 Dependen variable is he variable whose behavior should be modeled or prediced (Doughry, 1992). 13 Such models have ouperformed radiional ARIMA-based models in price forecasing, alhough no in all sudies (Sharda and Pail, 1994; Tang e al., 1990).

11 Table 2 Variables Selecion Variables Inpu Pas Closing Values DOW DOW 1, DOW 2, DOW 3 S&P S & P 1, S & P 2 DAX DAX, 1 DAX 2 CAC CAC 1, CAC 2, CAC 3, CAC 4 FTSE FTSE 1, FTSE 2, FTSE 3 IBEX IBEX, 1 IBEX 2 PSI PSI 1, PSI 2, PSI 3, PSI 4 IBOV IBOV 1 IBOV 2, IBOV 3, The frequency of he daa depends on he objecives of he researcher. A ypical of-floor rader in he sock or commodiy fuures markes would likely use daily daa if design a neural nework as a componen of an overall rading sysem. An invesor wih a longer erm horizon may use weekly or monhly daa as inpus o he neural nework o formulae he bes asse mix raher han using a passive buy and hold sraegy (Kaasra and Boyd, 1996). 5.2 DATA COLLECTION The research mus consider cos and availabiliy when collecing daa for he variables chosen in he previous sep. Technical daa is readily available from many vendors a a reasonable cos whereas fundamenal informaion is more difficul o obain. Time spend collecing daa canno be used for preprocessing, raining and evaluaing nework performance. The vendor should have a repuaion of providing high qualiy daa; however, all daa should sill be checked for errors by examine day o day changes, ranges, logical consisency and missing observaions (Kaasra and Boyd, 1996). Missing observaions which ofen exis can be handled in a number of ways. All missing observaions can be dropped or a second opion is o assume ha he missing observaions remain he same by inerpolaing or averaging from nearby values. In his work, we assume ha here are no missing observaions in he sample and, some values ha can be seen as ouliers are presens in he daa, because we aim o modeling sock markes mainly in urbulence scenes, characerized by low losses DATA PROCESSING As in mos oher neural neworks applicaions, daa processing is crucial for achieving a good predicion performance when applying neural neworks for finance ime series predicion. The inpu and oupu variables for which he daa was colleced are rarely fed ino hee nework in raw form. As he very leas, he raw daa mus be scaled beween he upper and lower bonds of he ransfer funcions (usually beween zero and one minus one and one). Two of he mos common daa ransformaions in boh radiional and neural nework forecasing are firs differencing and aking logarihm of a variable. Firs 14 The sample beginning on January 12, 2000 and finished on July 27, 2008.

12 differencing, or using changes in a variable, can be use o remove a linear rend of daa. Logarihmic ransformaion is useful for daa which can ake on boh small and large values. Logarihmic ransformaions also conver muliplicaive or raio relaionships o addiive which is believed o simplify and improve he nework raining (Masers, 1993) 15. In his work we used he logarihmic ransformaion of reurn Index Index 1 R = ln (9) Index 1 where R represens he normal logarihmic of reurns. This approach is especially useful in financial ime series analysis and produce good resuls according o he lieraure (see Fama, 1965; Granger and Morgensern, 1970). Also, he reurns behavior is more approximaed o a Normal probabiliy disribuion, bu, as will be show in his work, i is a very hardly hypohesis. 5.4 DATA PARTIONING Common pracice is o divide he ime series ino hree disinc ses called he raining, esing and validaion 16 (ou-of-sample) ses. The raining se is he larges se and is used by neural nework o learn he paerns presen in daa. The esing se, ranging in size from 10% o 30% of he raining se, is used o evaluae he generalizaion abiliy of a supposedly rained nework. A final check on he validaion se chosen mus srike a balance beween obaining a sufficien sample size o evaluae a rained nework and having enough remaining observaions for boh raining and esing. The validaion se should consis of he mos recen coniguous observaions. In his work he approach in evaluaion neural neworks used as fallows: 1. Training Se: 80% 2. Tesing Se: 15% 3. Validaion Se: 5% 5.5 NEURAL NETWORK DESIGN There are an infinie number of ways o consruc a neural nework. Neurodynamics and archiecure are wo erms used o describe he way in which a neural nework is organized. The number of inpu neurons is one of he easies parameers o selec once he independen variables have been reprocessed because each independen variable is represened by is own inpu neuron 17. The asks of selecion of he number of hidden layers, he number of he neurons in he hidden layers, he number of inpu neurons as well as he ransfer funcions are much more difficul. 15 Anoher popular daa ransformaion is o use raios of inpu variables. See Tomek and Querin (1984). 16 In some of sudies, he erm esing se is used as a name for he validaion se. 17 Each daa has is specific inpu variables as described in Table 1.

13 5.5.1 NUMBER OF HIDDEN LAYERS The hidden layer(s) provide he nework wih is abiliy o generalize. In pracice, neural neworks wih one and occasionally wo hidden layers are widely used and have performed very well. Increasing he number of hidden layers also increases compuaion ime and he danger of overfiing which leads o poor ou-of-sample forecasing performance. In he case of neural neworks, he number of weighs, which is inexorably linked o he number of hidden layers and neurons, and he size of he raining se (number of observaions), deermine he likelihood of overfiing (Baum and Haussler, 1989). I was applied neural neworks srucure wih one and wo hidden layers o a comparison NUMBER OF HIDDEN NEURONS Despie is imporance, here is no magic formula for selecing he opimum number of hidden neurons. Therefore researches fall back on experimenaions. However, some rules of humb have been advanced. A rough approximaion can be obained by he geomeric pyramid rule proposed by Masers (1993). For a hree-layer nework wih n inpu neurons and m oupu neurons, he hidden layer would have n m neurons. Baily and Thompson (1990) sugges he he number of hidden layer neurons is a hree-layer neural nework should be 75% of he number of inpu neurons. Kaz (1992) indicaes ha he opimal number of hidden neurons will generally be found beween one-half o hree imes he number of inpu neurons. Ersoy (1990) proposes doubling he number of hidden neurons unil he nework s performance on he esing se deerioraes. Klimasauskas (1993) suggess ha here should be a leas five imes as many raining facs as weighs, which ses an upper limi on he number of inpu and neurons. Because of hese feaures, his work applied differen srucures o all daa, chosen randomly, wih 22, 34, 40, 52 and 60 neurons in he hidden layer as o describe he bes srucure according o he index NUMBER OF OUTPUT NEURONS Deciding on he number of neurons is somewha more sraighforward since here are compelling reasons o always use only one oupu neuron. Neural nework wih muliple oupus, especially if hese oupus are widely spaced, will produce inferior resuls as compared o a nework wih a single oupu (Masers, 1993). The modeling applied in his work aims o one day pas closing value in he fuure forecasing, and as cied above, was uilized one oupu layer srucure TRANSFER FUNCTION The majoriy of curren neural nework models use he sigmoid ransfer funcion, bu ohers such as he angens hyperbolicus, arcus angens and linear ransfer funcions have also been proposed (Haykin, 2001). Linear ransfer funcions are no useful for nonlinear mapping and classificaion. Levich and Thomas (1993) and Kao and Ma (1992) found ha financial markes are nonlinear and have memory suggesing ha nonlinear ransfer funcions are more

14 appropriae. Transfer funcions such as he sigmoid are commonly used for ime series daa because hey are nonlinear and coninuously differeniable which are desirable properies for nework learning. In his sudy, sigmoid ransfer funcion was applied in he nework proposed TRAINING THE ANN Training a neural nework o learn paerns in he daa involves ieraively presening i wih examples o he correc known answers. The objecive of raining is o find he se of weighs beween he neurons ha deermine he global minimum of he error funcion. Unless he model is overfied, his se of weighs should provide good generalizaion. The backpropagaion nework, applied in his work, uses he gradien descen raining algorihm which adjuss he weighs o move down he seepes slope of he error surface. Finding he global minimum is no guaraneed since he error surface can include many local minima in which he algorihm can become sruck. This secion will discuss when o sop raining a neural nework and he selecion of learning rae and momenum values NUMBER OF TRAINING ITERATIONS Many sudies ha menion he number of raining ieraions repor convergence from 85 o 5000 ieraions (Deboeck, 1994; Klaussen and Uhrig, 1994). However, he range is very wide as and ieraions (Klimasauskas, 1993; Odom and Sharda, 1992) and raining imes of 60 hours have also been repored. Training is affeced by many parameers such as he choice of learning rae and momenum values, proprieary improvemens o he backpropagaion algorihm, among ohers, which differ beween sudies and so i is difficul o deermine a general value for he maximum number of runs. Also, he numerical precision of he neural nework sofware can affec raining because he slope of he error derivaive can become very small causing some neural neworks programs o move in he wrong direcion due o round off errors which can quickly build up in he highly ieraive raining algorihm. I is recommended ha researches deermine for heir paricular problem and es as many randomly saring weighs as compuaional consrains allow (Kaasra and Boyd, 1996). We uilized 500, 1000, 2500, 5000, 8000 and ieraions randomly o choose he bes perform o each index LEARNING RATE During raining, a learning rae ha is oo high is revealed when he error funcion is changing wildly wihou showing a coninued improvemen. A very small learning rae also requires more raining ime. In eiher case, he research mus adjus he learning rae during raining or brainwash he nework by randomizing all weighs and changing he learning rae for he new run hrough he raining se. Iniial learning raes used in his work vary widely from 0.1 o 0.9. Mos neural nework sofware programs provide defaul values for learning rae ha ypically work 18 Sigmoid ransfer funcion is defaul in MATLAB neural nework oolbox.

15 well. Common pracice is o sar raining wih a higher learning rae such as 0.7 and decrease as raining proceeds. Many nework programs will auomaically decrease he learning rae as convergence is reached (Haykin, 2001). 6. COMPARISON ANALYSIS In his secion we will go o presen he neural nework srucure implemened for he daa ha resuled in a minimum error. Also, he resuls of ANNs and GARCH model are described o a comparison. Table 3 shows he bes neural nework srucure performed for each index sudied. Table 3 Neural Nework Design Index Inpus Hidden Layer(s) Hidden Neurons Ieraions Learning Rae DOW S&P DAX CAC FTSE IBEX PSI IBOV The resuls show ha he choice of srucure is differen according o he daa. Then, do no have any magic formula o describe a srucure ha minimize he error and resul in a bes resul. The bes choice have mus be search by he randomly alernaives according o he daa. The experimenal resuls revealed ha he proposed algorihm provide a promising alernaive o sock marke predicion resuling in low errors (see Table 4). Table 4 compares he ranked Coefficiens of Muliple Deerminaion for each model. The R square value represens he proporion of variaion in he dependen variable ha is explained by he independence variables. The beer he model explains variaion in he dependen variable, he higher he R squared value. Wihou furher comparison, he Neural Nework bes explains variaion in he dependen variable, followed by he Regression Model. The ranked error saisics are provided for comparison. These saisics are all based on reurns errors beween a desire and a neural nework oupu value.

16 Table 4 Error Comparison Index R Squared Percenage Mean Error Mean Square Roo Error ANN GARCH ANN GARCH ANN GARCH DOW S&P DAX CAC FTSE IBEX PSI IBOV In Table 4 is relaively easy o visually verify ha he neural nework model perform beer han he regression model. This differs from he model ranking due o R squared values. Neural nework model predic he closing value relaively accuraely. In a enaive o evaluae he robusness of he ANN model applied, we analyze he error dimension in ses performed (raining, es and validaion). The resuls are measured by Maximum Percen Error (MPE) and Mean Squared Roo Error (MSRE) denoed n 100 yi yˆ i MPE = max (10) n i= 1 yi MSRE 1 n y i yˆ i 2 = ( ) (11) where yi denoe he desire value i and ŷ i he neural nework oupu. The comparison among he nework ses is showed in Table 5. Table 5 Nework ses comparison Index Ses Training Tes Validaion MPE MSRE MPE MSRE MPE MSRE DOW S&P DAX CAC FTSE IBEX PSI IBOV We can see in Table 5 ha a neural nework srucure applied o all indexes had he predicion capabiliy and, how has been seen in he low rae of validaion errors, a neural nework learn wih he daa and can process good resuls o forecasing. Finally, he resuls in Tes and Validaion ses confirm he generalizaion capabiliy of a neural nework.

17 One quesion ha we proposed in his work is: Can Neural Nework incorporaes heeroscedasiciy phenomena? Table 6 provides he residual analysis of each series sudied in his work. I includes mean es, es of Normaliy (Jarque-Bera), es of correlaion presen in he residuals (Ljung-Box-Pierce Q-Tes) and verify he heeroscedasiciy in he residuals (Engle s ARCH Tes) Index Value Table 6 Residuals Analysis Tess Mean Jarque-Bera Ljung-Box-Pierce Engle s ARCH Saisically zero? Value Normal? Value Correlaion? Value Homocedasiciy? DOW Ok Ok No Ok S&P Ok Ok No Ok DAX Ok Ok No Ok CAC Ok Ok No Ok FTSE Ok Ok No Ok IBEX Ok Ok No Ok PSI Ok Ok No Ok IBOV Ok Ok No Ok Analyzing Table 6 we can see ha he neural neworks residuals for all indexes sudied have mean saisically equal o zero, have Normal disribuion, here is no correlaion beween he residuals and, finally, he residuals are homocedasiciy. The resuls show ha a neural nework srucure proposed was capable o series modeling and forecasing, capuring he heeroscedasiciy phenomena and confirm he robusness of he mehod. 7. CONCLUSIONS This research examined and analyzed he use of neural neworks as a forecasing ool. Specifically a neural nework's abiliy o predic fuure rends of Sock Marke Indexes was esed. Norh American, European and Brazilian Sock Markes Indexes were sudied. Accuracy was compared agains a radiional forecasing mehod (GARCH). While only briefly discussing neural nework heory, his research deermined he feasibiliy and pracicaliy of using neural neworks as a forecasing ool for he individual invesor. I was concluded ha neural neworks do have a powerful capaciy o forecas all sock marke indexes sudied and, if properly rained, he individual invesor could benefi from he use of his forecasing ool agains curren echniques for he following reasons: When using muliple linear regression, he governing regression assumpions mus be rue. The lineariy assumpion iself and normal disribuion my no hold 19 For his ess see Brockwell (1991). 20 Chebyschev Inequaliy Tes was applied o confirm he resuls abou residuals probabiliy disribuion. For all index was confirmed a Normal disribuion.

18 in mosly financial ime series. Neural Neworks can model nonlinear sysems and do no have any assumpion abou inpu probabiliy disribuion. ANNs are universal funcions approximaion. I has been shown ha a neural nework can approximae any coninuous funcion o any desire accuracy. ANNs can generalized. Afer learning he daa presened o hem, ANNs can ofen correcly infer he unseen par of a populaion even if he sample daa conain noisy informaion. Compared wih GARCH model, neural neworks are significanly more accurae. Heerocedasiciy phenomena can be capured by ANNs. The nex sep in fuure works is o inegrae neural neworks and oher echniques such as geneic echniques, wavele analysis, fuzzy inference, paern recogniion and, radiional ime series models, for finance and economic forecasing. The advanages of geneic echniques include adapiveness and robusness, which avoid neural neworks o ge suck a a local opimum. Once he nework was rained, esed and idenified as being good, a geneic algorihm was applied o i in order o opimize is performance. The process of geneic evoluion worked on he neuron connecion of a rained nework by applying wo procedures: muaion and crossover. The applicaion of hybrid sysems seemed o be well suied for he forecasing of financial daa. On he oher hand, he discussion abou inpu variables can be aken according o each daa sudied. ACKNOWLEDGMENT This work was suppored by he Brazilian Naional Research Council (CNPq) grans / REFERENCES Ao, S. I. (2003a), Analysis of he ineracion of Asian Pacific indices and forecasing opening prices by hybrid VAR and neural nework procedures. In: Proc. In. Conf. on Compuaional Inelligence for Modelling, Conrol and Auomaion 2003, Vienna, Ausria. Ao, S. I. (2003b), Incorporaing correlaed markes prices ino sock modeling wih neural nework. In: Proc. IASTED In. Conf. on Modelling and Simulaion 2003, Palm Springs, USA, pp Anderson, J. A. and Rosenfeld, E. (1988), Neurocompuing: Fundaions of research. MIT Press, Cambridge, MA. Baum, E. B. and Haussler, D. (1989), Wha size ne gives valid generalizaion?. Neural Compuaion, 6, pp Baily, D. and Thompson, D. M. (1990), Developing neural nework applicaions. AI Exper, 12, pp Bishop, C. (1996), Neural Neworks for Speech and Sequence Recognaion. Thompson, London. Bollerslev, T. R. (1986), Generalized Auoregressive Condiional Heeroskedasiciy. Journal of Economerics, 51, pp

19 Brockwell, P. J and Davis, R. A. (1991), Time Series: Theory and Mehods. Second Edion, New York: Springer. Campbell, J. Y.; Lo, A. W. and Maclinkay A. C. (1997), The Economerics of Financial Markes. Princeon Universiy Press, Unied Kingson. Chaerjee, A.; Ayadi, O. F. and Boone, B. E. (2000), Arificial neural nework and he financial markes: A survey. Managerial Finance, 26, pp Chen, N. (1991), Financial invesmen opporuniies and he macroeconomy. Journal of Finance, 46, pp Deboeck, G. J. (1994), rading on he Edge: Neural, Geneic and Fuzzy Sysems for Chaoic Financial Markes. Wiley, New York. Doughery, C. (1992), Inroducion o Economerics. Oxford Universiy Press, New York. Ersoy, O. (1990), Tuorial a Hawaii Inernaional Conference on Sysems Sciences. January 1990, Hawaii. Fang, H; Lai, S. and Lai, M. (1994), Fracal srucure in currency fuures price dynamics. Journal of Fuures Markes, 14, pp Fama, E. F. (1965), The behavior of sock markes prices. Journal of Business, 14, pp Fama, E. F. and French, K. (1993), Common risk facors in he reurns on socks and bonds. Journal of Financial Economics, 33, pp Ferson, W. and Schad, R. (1996). Measuring fund sraegy and performance in changing economic condiions. Journal of Finance, 51, pp Gaely, Edward J. (1996), Neural Neworks for Financial Forecasing. John Wiley & Sons, New York. Granger, C. e Morgensern, O. (1970), Predicabiliy of sock marke prices. Healh Lexingon, Massachuses, USA. Haykin, S. (2001). Neural Neworks A Comprehensive Foundaion. IEEE Press, New York. Hech-Nielsen, R. (1990), Neurocompuing. Addison-Wesley, Reading, MA. Herz, J.; Krogh, A. and Palmer, R. G. (1991), Inroducion o he Theory of Neurocompuaion. Addison-Wesley, Reeading, MA. Hiemsra, C. and Jones, J. D. (1994), Tesing for linear and nonlinear Granger causaliy in he sock price volume relaion. Journal of Finance, 49, May, pp Hornik, K. (1993), Some new resuls on neural nework approximaion. Neural Neworks, 6, pp Hornik, K.; Sinchcomber, M. and Whie, H. (1989) Mulilayer feedforward neworks are universal approximaions. Neural Neworks, 2, pp

20 Huang, W.; Nakamori Y. and Wang, S. Y. (2005), Forecasing sock marke movemen direcion wih suppor vecor machine. Compuers & Operaions Research, 32, pp Kaasra, I and Boyd, M. (1996) Designing a neural nework for forecasing financial and economic ime series. Neurocompuing, 10, pp Kao, G. W. and Ma, C. K. (1992), Memories, heeroscedasiciy and price limi in currency fuure markes. Journal of Fuure Markes, 12, pp Karz, J. O. (1992), Developing neural nework forecasers for rading. Technical Analysis of Socks and Commodiies, 8, pp Klaussen, K. L. and Uhrig, J. W. (1994), Cash soybean price predicion wih neural neworks. In Conference on Applied Commodiy Analysis, Price, Forecasing and Marke Risk Managemen Proceedings, pp , Chicago. Klimasauskas, C. C. (1993), Applying Neural Neworks. In: R. R. Trippi and E. Turban, ediors, Neural Neworks in Finance and Invesing: Using Arificial Inelligence o Improve Real World Performance, pp Probus, Chicago. Lawrence, J. (1991), Inroducion o Neural Neworks. California Scienific Sorware: Grass Valley, CA. Levich, R. M. and Thomas, L. R. (1993), The significance of echnical rading rule profis in he foreign exchange marke: A boosrap approach. In Sraegic Currency Invesing Trading and Hedge in he Foreign Exchange Marke, pp , Probus, Chicago. Lo, A. W. and Mackinlay, A. C. (1988), Sock marke prices do no follow random walks: Evidence from a simple specificaion es. Review of Financial Sudies, 1, pp Masers, T. (1993), Pracical Neural Nework Recipes in C++. Academic Press, New York. Marin, R. D. (1998), Garch modeling of ime-varying volailiies and correlaions. URL: hp://fenews.com/1998/issue4/ hm. (URL accessed on July 5, 2008). Minsky, M. and Paper, S. (1969), Perceprons. MIT Press, Cambridge, MA. Michel, T. M. (1997), Machine Learning. McGraw-Hill. Nelson, M. M. and Illingworh. (1991), A Pracical Guide o Neural Nes. Addison Wesley, Reading, MA. Odom, M. D. and Sharda, R. (1992), A neural nework for bankrupcy predicion. In Proc. IEEE In. Conf. on Neural Neworks, pp. II163-II168, San Diego. Rumelhar, D. E. and Mcclelland, J. L. (1986), Parallel Disribued Processing, Exploraions in he Microsrucure of Cogniion. MIT Press: Cambridge, MA. Rupper, D. (2001), GARCH models. URL: hp:// (URL accessed on April 25, 2008).

21 Sharda, R. and Pail, R. B. (1994), A connecionis approach o ime series predicion: An empirical es. In: Deboeck, G. J., edior, Trading on he Edge: Neural, Geneic, and Fuzzy Sysems for Chaoic Financial Markes, pp , Wiley, New York. Schraudolph, N. and Cummins, F. (2002), Inroducions o Neural Neworks. URL: hps:// (URL accessed on Sepember 13, 2008). Sie, R. and Sie, J. (2000), Analysis of he predicive abiliy of ime delay neural neworks applied o he S&P 500 ime series. IEEE Transacion on Sysems, Man and Cyberneics, 30, November, pp Tang, Z; Almeida, C. and Fishwick, P. A. (1990), Time series forecasing using neural neworks vs. Box-Jenkins Mehodology. In: Inernaional Workshop on Neural Neworks, Auburn, AL. Topek, W. G. and Querin, S. F. (1984), Random process in prices and echnical analysis. Journal of Fuure Markes, 4, pp Waie, T. and Hardenbergh, H. (1989), Neural nes. Programmer s Journal, 7, pp Wasserman, P. D. (1993), Advanced Mehods in Neural Compuing. Van Nosrand Reinhold, New York. Wong, B. K. and Selvi, Y. (1998), Neural nework applicaions in business: A review and analysis of he lieraure. Informaion & Managemen, 34, pp Zang, G.; Pauwo, B. E. and Hu, M. Y. (1998), Forecasing wih arificial neural neworks: The sae of he ar. Inernaional Journal of Forecasing, 14, pp