Nonlinearity or Structural Break? - Data Mining in Evolving Financial Data Sets from a Bayesian Model Combination Perspective

Transcription

1 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences Nonlneary or Srucural Break? - Daa Mnng n Evolvng Fnancal Daa Ses from a Bayesan Model Combnaon Perspecve Hao Davd Zhou Managemen Deparmen Drexel Unversy [email protected] Absrac Much work n he exploraon of daa mnng has been focused on qualy daa. Daa non-saonary s prevalen n realy and s furher complcaed by model uncerany. The emphass on organzaonal mpac and benef maxmzaon of daa mnng urges us o develop models ha are easy o be undersood by manageral decson makers. To provde a beer soluon o hese applcaon challenges, we propose a new approach negrang Bayesan srucural break models wh change pon deecon mehods o chronologcally ordered observaons. We apply our approach o hree exchange rae predcons. Our approach ncorporaes boh srucural break and model uncerany explcly. I no only has a clear nuve appeal bu also has a farly frm sascal foundaon. The benchmark comparson shows srong emprcal evdence ha our approach could mach he approxmang ably of neural neworks n mnng daa wh srucural break. Furhermore, comparng o neural neworks, our approach provdes beer nerpreably.. Inroducon In he pas, much work n he heorecal and mehodologcal exploraon of daa mnng has been focused on qualy daa. The npu o he daa mnng algorhm or echnque s assumed o be saonary and o conan no ncorrec or anomaly observaons. The lkelhood funcons of he approxmang models beween npu and oupu are assumed o be smooh and snglemodal. As we accumulae more and more daa and are able o access rcher classes of models, wo new applcaon challenges emerge and could severely nhb our capably of usng daa mnng o mprove manageral decson-makng. They are: (a) how o explo he evolvng daa under a dynamc busness envronmen (srucural break) and (b) how o selec he bes model under he coexsence of numerous compeng alernaves (model uncerany) o maxmze he benefs of echnology soluons. Srucural break and srucural nsably characerze many forecasng models fed o economc and fnancal daa and have been wdely documened n economcs and fnance leraure (Pesaran and Tmmermann [5]). Alhough evolvng relaonshp beween varables n dynamc busness envronmen s prevalen n realy, here s very lle research akng no accoun hs problem explcly n knowledge dscovery and daa mnng applcaon. Sung e al [9] frs jusfy he necessy for dfferen models n bankrupcy predcon under dfferen economc condons. They classfy economc condons o normal condon and crss condon. When we apply he predcon models for he normal condon o daa analyss under he crss condon, he accuracy rae of he bankrupcy predcon model drops sgnfcanly - from 83.3 percen o 66.7 or worse. Sung e al [9] sae ha he non-robusness of modelng requres connuous remodelng as daa or suaon changes. In daa mnng applcaon, he non-saonary challenge s furher complcaed by he presence of non-lneary and oulers. Many applcaon felds lke fnancal me seres predcon are domans characerzed by nonlneary, srong nose, weak sgnal and lack of funconal srucure. In many suaons, he lkelhood funcons of he approxmang models beween npu and oupu are non-smooh and mul-modal. Granger and Tmmermann [9] argue ha as we are lkely o search over a much larger space of models hrough flexble modelng approach such as neural neworks, he lkelhood of overfng he daa-generang process s much hgher. Granger and Tmmermann s saemen are furher elaboraed by Andrew, Lo, a fnance professor a MIT, as he warns: "Gven enough me, enough aemps, and enough magnaon, almos any paern can be eased ou of any daa se." (Lo [6]) Mos of he exsng approaches n mnng daa wh srucural break and model uncerany could no ncorporae model uncerany explcly and offer good nerpreably. Black-box modelng grealy resrcs her

2 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences applcaon because many manageral decson makers prefer o adop he models ha are easy o undersand (Dhar e al [5]). Densy forecasng has won more aenon recenly (Pasley e al [4]). In many busness applcaons, a predcon nerval wh acceped confdence level s more preferred because one wans o know he assocaed rsk o make he opmal decson. In hs paper we propose a new approach n mnng daa wh srucural break and model uncerany. We apply our approach o he predcon of exchange rae, one of he mos exensvely suded fnancal varables. Our new approach negraes Bayesan srucural break models and he change pon deecon mehods o chronologcally ordered observaons. We use he change pon deecon mehods o deec he poenal srucural breaks. Focused on he mos recen breaks, we hen use Bayesan srucural models o ncorporae he model uncerany of he srucural breaks. Our research queson s wheher he negraed approach beween Bayesan srucural break models and change pon deecon mehods could mach he approxmang ably of neural neworks n mnng daa wh srucural break. We argue our approach has a clear nuve appeal and a farly frm sascal foundaon. Our resuls show srong suppor ha our Bayesan srucural break model negraed wh change pon deecon mehods could mach he approxmang ably of neural neworks. Furhermore, comparng o neural neworks, our approach provdes beer nerpreably, a compeve advanage n daa mnng applcaon. Afer hs nroducon, we organze our paper as follows. In secon we formally defne he non-lneary models, srucural break models and ouler models we are gong o use and revew he exsng approach n mnng daa wh srucural break. In secon 3 we descrbe he wo mehods we adop n evolvng daa sream mnng. In secon 4 we repor he resuls. We conclude and dscuss he mplcaon and he fuure work n secon 5.. Background. Non-lneary models, srucural break models and ouler models Before we proceed, we need clarfy some defnons ha we are gong o use. In her fundamenal conrbuon, Koop and Poer [5] defne he models where dynamcs change permanenly n a way ha canno be predced by he hsory of he seres as srucural break models. They defne he models ha allow for dynamcs varyng over he busness cycle n a predcable way as nonlnear models. They furher defne models ha have apparen deparures from lneary and are due o unpredcable large shocks wh only emporary effecs as ouler models. I s no surprsng ha here s some exen of abuse of ermnology n he leraure. For nsance, Lubrano [7] classfes nonlnear model o srucural break models and hreshold regresson models. Lubrano defnes he models ha combne an abrup ranson funcon wh a me ndex as he srucural break model. The breakng pon could be unknown. Lubrano defnes he models where an abrup ranson s no combned wh a me ndex bu combned wh a connuous varable, no necessarly growng over me, as hreshold models. In order o avod confuson, we wll use he defnon adoped by Koop and Poer [5] hroughou he remander of hs paper. Pesaran and Tmmermann sae Fnancal me seres are lkely o undergo sudden, large changes reflecng nsuonal changes, regme swches or breakdowns n a marke mechansm as observed durng fnancal crses (Pesaran and Tmmermann [5] Page 508). Breaks or jumps n he parameers ha relae secury reurns o sae varables could arse due o a number of facors such as major changes n marke senmens, burs or creaon of speculave bubbles, regme swches n a moneary and deb managemen polces ( Pesaran and Tmmermann [5] Page 496). Tmmermann and Granger [33] furher argue ha sable forecasng paerns are unlkely o perss for long perods of me and wll self-desruc when dscovered by a large number of nvesors. Pesaran and Tmmermann [6] llusrae ha can be very cosly o gnore breaks and ha forecasng approaches condonng on he mos recen break are lkely o perform beer over uncondonal approaches. Anomaly or aypcal observaons could affec me seres predcon and sascal models grealy. Franses and Djk [7] furher pon ha neglecng aypcal observaons wll have even more mpac on ou of sample forecass n nonlnear me seres han n lnear me seres and we need pay consderable aenon o ake no accoun such observaons whle consrucng nonlnear models. In hs paper, we wll focus on models for fnancal me seres ha mpose a regme-swchng srucure. Followng Koop and Poer [5], we wll resrc our focus on models ha have a clear nerpreaon and are plausble from an economc perspecve. The followng model s poseror probables are well known. Because of he me consran, we only nvesgae model,, 3 and her combnaons o predc exchange rae change:. Lnear model. Homoscedasc srucural break model wh one break 3. Heeroscedasc srucural break model wh one break 4. Ouler models wh one ouler 5. Homoscedasc nonlnear models wh one hreshold

3 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences Heeroscedasc nonlnear models wh one hresholds A formal specfcaon of he models of he fnancal me seres predcon s descrbed by Koop and Poer [5]. We descrbe hese models brefly here. β 0 + β p + σ V f I = Y = β00 + β0 p + σ 0V f I = 0 where I s an ndcaor varable for he regme and β s 0 nercep coeffcens and β s he slope coeffcens. V p - s assumed o be sandard normal and ndependen over me. There are four ways of defnng I () Model s obaned f we se I =0 for all. () The srucural break model wh one breaks s obaned f we se I = when < τ and I = 0 when τ <, (3) The ouler model s obaned f we se I 0 for only one values of and β p = β, 0 p σ = σ bu 0 β0 β 00 (4) β 0 + βp, + σ V f I = Y = β 00 + β 0 p, + σ 0V f I = 0, s obaned by keepng all predcors n excep for The nonlnear model s obaned f we se I = f,, > γ, I =0 f r,.,. Srucural break deecon mehods In he curren knowledge dscovery and daa mnng leraure, he common approach o srucural break s usng an arbrary sze of recen daa o generae he updaed models. Whle forecasng bas and forecasng varance ogeher deermne he accuracy level when he forecasng varable s connuous, we have o face a rade-off beween reducng bas and reducng varance n processng he evolvng daa ses: f we oally abandon he daa n he long me ago, we may have lower bas, bu hgher varance n he predcon. If we use he daa n he long me ago whou any preprocessng, we may have lower varance, bu hgher bas n he predcon. Usng dfferen ranng horzons wh daly daa for he S&P 500 ndex, Meha and Bhaacharyya [8] repor he sensvy of he performance of he dscovered paerns wh respec o he ranng duraons. Mos of he prevous research n modelng srucural break has a focus on he n-sample model f nsead of he forecas of he fuure (Oh and Han [] []). In her fundamenal conrbuon, Pesaran and Tmmermann [5] analyze he sably of a model relang US sock reurns o lagged values of he dvdend yeld, shor-run neres rae and defaul premum by usng reversed ordered Cusum (ROC) breakpon mehod. They compared he ROC mehod o exsng uncondonal approaches such as expandng or rollng wndow, me varyng parameers and ec. The reversed ordered Cusum breakpon mehod seems o work suffcenly well o conssenly denfy hree major breaks n a forecasng model for he sock reurns. In her semnal work, Csorgo and Horvah [3] enumerae hree major change-pon deecon mehods for deecng varous ypes of changes n chronologcally ordered observaons - () he lkelhood raon es for a paramerc mehod based on lkelhood () he Pe es for a nonparamerc approach based on he Mann- Whney ype sasc and (3) he Chow es for a lnear model wh lnear model resrcon. In her nnovaon, Oh and Han [] [] furher propose a new cluserng forecasng sysem whch negraes change-pon deecon and a unversal approxmang mehodology - arfcal neural neworks. They adop he above hree change-pon deecon mehods o deec a seres of change pons and group he successve ranng daa o dfferen homogenous groups based on he change pons deeced. In he followng sage, neural neworks are raned based on he ranng npus a me wh caegorcal group oupu for +. The raned neworks are appled o forecas he caegorcal group oupu o he new comng daa. In he fnal sage, neural neworks raned wh he correspondng group daa are appled o forecas he new comng daa s magnude oupu. In machne learnng feld, an acve research feld rackng conex drf (Schlmmer e al [30]) s closely relaed o our research. Wdmer e al [35] presen a general wo-level learnng model ha could effecvely rack srucural break by ryng o deec srucural break clue and use hs clue o focus he learnng process. Harres e al [] argue ha concep drf due o hdden changes n concep complcaes learnng n many applcaons and presen a new approach whch use an exsng bach learner and he process of conexual cluserng o denfy hdden conexs. Srvasava e al [3] and Wegend [34] nroduce a new ool called Scale-Sensve Gaed Expers (SSGE) o analyze me seres wh srucural break: SSGE consss of a nonlnear gang neural nework and several compeng nonlnear expers (modeled by usng neural nework). The gang nework s ask s o learn o assocae npus wh parcular expers and he se of exper neworks ask s o predc he value a he regresson surface gven he npu. The assocaon probably - he probably of assocang an npu-oupu par o a parcular exper s derved by usng he prncple of maxmum enropy. Mos of he exsng research does no consder model uncerany explcly. A srucural break could be eher a sudden large change or a slow small change. A srucural break model could be mul-modal as we wll show n 3

4 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences fgure 3 and fgure 4. Mos of he exsng approaches n modelng srucural break ry o fnd an opmal srucural break pon and hus om he model uncerany compleely. Mos of he exsng approach could no offer a good nerpreably. 3. Mehod specfcaon In hs paper, we adop wo mehodologes Neural neworks wh Bayesan regularzaon and Bayesan srucural models. We compare he advanage and dsadvanage of boh of hese wo mehods. As Pesaran and Tmmermann [5] noe, usng a fxed rollng wndow sze, c, o forecas may make he wndow oo shor or oo long, f c s no seleced appropraely. We use wo mehods o deermne he rollng wndow sze. One s a consan rollng wndow sze wh 0 mos recen observaons. Anoher s a dynamc rollng wndow sze deermned by he change pon deecon mehods. 3. Reversed Pe es and Cusum es In hs paper, we adop wo change pon deecon mehods he reversed Pe es mehod and he reversed Cusum (ROC) es. As Oh and Han [] sae, hese wo ess are frequenly offered by sascal packages and are represenave n nonparamerc approach and lnear model approach. Furhermore, as Oh and Han [] and Pe [7] sae, Pe es s more preferable n forecasng chaoc es because provdes a robus mehod ressan o anomaly or aypcal observaon frequenly caugh n fnancal me seres daa. We devae from he prevous research n ha we use he change pon deecon mehod o deec he mos recen wo breaks (f more han one break could be deeced) or he only break (f only one break could be found). We hen use he breaks deeced o segmen he daa and apply neural nework and Bayesan srucural break model o approxmae he wo mos recen break daa (f more han one break s deeced) or he whole daa se (f only one break could be found). Change pon deecon mehod requres our decson on sgnfcan level. For nsance, n he fgure 4, dfferen sgnfcan level could deermne wheher he frs break happens a pons around pon 40, pon or pon. As noed by Pesaran and Tmmermann [5], may be opmal o also nclude pre-break daa o esmae a forecasng wndow. By sng he negraon of Bayesan srucural break model and change pon deecon mehod, we do no have hs concern because Bayesan srucural break model could ncorporae hs uncerany explcly. 3. Arfcal neural Neworks wh Bayesan regularzaon ANN has araced many scholars from many dfferen felds. ANN could approxmae a nonlnear (or lnear) funcon o an arbrary degree of accuracy hrough he composon of a nework of relavely smple funcon, f we selec an approprae number of ANN s hdden-layer uns (Hornk e al [4]). In oher words, ANN s a unversal approxmaor. However, s also well known ha he black-box modelng makes neural nework unpraccal when he applcaon doman needs clear nerpreaon. Some excng work exracs rules from neural nework (Baesens []). However, n many suaons, a predcon nerval wh acceped confdence level s more preferred because one wans o know he assocaed rsk o make he opmal nvesmen decson. Furhermore, he number of he nodes n he hdden layer, he nal value of he parameers and he ranng perod all grealy affec he performance of neural nework. However, here s no rgorous procedure or wdely acceped rule o denfy, selec and es he model srucure of neural neworks (Hase []). I s sll an ar o deermne he srucure of neural nework and we need balance he rade-off beween over-fng and underfng. 3.3 Bayesan srucural models and Bayesan model combnaon The semnal work of Baes and Granger s The combnaon of forecass (Baes and Granger []) has nspred exensve works on model combnaon n boh managemen scence, economerc and arfcal nellgence leraure In mos examples of nference and predcon, a model M s used o descrbe he relaonshp beween dependen varable(s) and ndependen varable(s). Darper [5] sae ha a model M ypcally nclude wo pars: () srucural descrpon S such as a parcular lnk funcon n a generalzed lnear model or a parcular form of heeroscedascy and () parameer descrpon θ whose meanng s specfc o he chosen srucure. In modelng srucural breaks, a model could be eher heeroscedasc or homoscedasc. A srucural break model could be eher sngle modal or mul-modal. In pracce mos sascal mehods acknowledge paramerc uncerany abou θ whou acknowledgng srucural uncerany abou S, and only search a sngle bes choce * s accordng o some crera such as R, AIC, SIC, FIC and PIC o make nferences and predcons as f S were known o be correc. Darper [5] demonsrae ha Bayesan model averagng approach solve he problem of falure o assess and propagae srucural uncerany by reang he enre 4

5 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences model M=(S,θ ) as a nusance parameer and negrang over uncerany abou S and θ, as n he expresson m y x, ϕ ) = y x, S, θ ) S, θ x) dθ = = = m S x) y x, S ) The frs facor on he rgh-hand sde s he poseror probably of S j, whch s gven by: P( S x) = m P( x S ) P( S ) = P( x S ) P( S ) where P ( S ) s he decson-maker s pror belef ha S s he correc model, and P ( x S ) s he margnal lkelhood of he daa. In knowledge dscovery and daa mnng leraure, model uncerany problem has long been gnored. In sead, a lo of effors have been pu on opmzaon ssue. For nsance, Hand e al [0, page 5] sae ha daa mnng componens nclude model/paern srucure deermnaon, score funcon judgmen, opmzaon and search mehod. Padmanabhan and Tuzhln [3] argue ha combnng he opmzaon mehods wh he daa mnng can resul n more powerful analycal approaches. We agree ha opmzaon provdes an opporuny n maxmzng he benefs of daa mnng echnology. However, we beleve overesmang opmzaon mehods could lead us o draw spurous concluson especally when he model uncerany s very hgh. Koop and Poer [5] noe ha n he suaon when here are many nonlnear models, srucural break models and ouler models o explan he same daa ses, lkelhood funcons could be non-smooh and mulmodal. Bayesan mehods reduce he uncerany because could no only use he nformaon from he enre parameer space bu also use poseror model probables o combne models. Bayesan mehods could be compuaonally demandng. As we could access more and more fas compuer, he neres n Bayesan mehods also surge very quckly n recen years. Suppose we have a model specfcaon Y = k j j = β + ε Followng George [8] and Holmes e al [3], we use he wdely adoped Normal Inverse Gamma Dsrbuon (NIG) as he conjugae choce of jon pror for β and σ. In parcular, The prors s p ( β, σ ) = β σ ) σ ) = N ( m, σ V ) IG( a, b) = a b ( a+ ( k / ) + ) ( σ ) k / / (π ) V Γ( a) ' ex {( β m ) V ( β m) + b}/(σ )] where V = c( ' ) The model poseror probably of lnear model s j D β σ ) β, σ ) D M ) = β where * ' ' m = ( V + ) ( V m + Y ), V ' = ( V + ) * * / a * ( ) ( ) *, V b Γ a * a = ( b ) / n /, σ D) V π Γ( a), * a = a + n / ' ' * ' * * b* = b + { mv m + Y Y ( m ) ( V ) m }/ Based on he poseror probably of every model and s esmaon, we could aan he combned model s predcve nference from he followng formula: where M y x, D) = D) = M = y x, D, M ) M D) D M ) M ) D M ) M ) j j j Deal model descrpon could be found n George [8], Holmes e al [3] and many oher books on Bayesan model averagng, As s well known, c s crcal because deermnes he preferable sze of he model. We refer deal dscusson abou he mporance of seng he value of c o Holmes e al [3] and George [8]. We se c value followng he way proposed by George [8]. Alhough George [8] se he c value o selec he sngle bes model, we beleve has a clear nuve appeal and a farly frm sascal foundaon n combnng models. We assume each poenal model has an equal pror. In oher words, we adop a oal daa drven approach. We adop he model specfcaon n secon. because of he exsence of analycal resul. Condonal on knowng I, he srucural break model breaks no wo sandard lnear regresson models. The analycal poseror resuls exs for lnear regresson f we adop naural conjugae pons for each regme. To oban poseror resuls whch are no condonal on I, we should know he margnal poseror for he parameers defnng I. The deals o reach he poseror resuls for he srucural break models and ouler models could be found a Koop and Poer [5]. Furhermore, followng Koop and Poer [5], we se a dscree unform pror over all possble sample breaks. We also resrc ha a leas 30 daa observaons le n each regme. Ths rule s used o ensure ha an adequae amoun of daa s avalable n each regme. 4. Applcaon o he foregn exchange rae predcon 4. Daa descrpon In hs sudy, we apply he proposed mehodology o he predcon of foregn exchange rae, one of he mos exensvely suded varables n he fnancal economy 5

6 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences Followng Q and Wu [8], we employ a smple verson of he moneary model of exchange rae deermnaon o gude he choce of forecasng varables. h h s + h s = b + c ( m a0 a y + ar s ) + ε + h where s + and h s s he logarhm of exchange rae (domesc currency prce of one un foregn currency) a h h me +h and me. b and c are regresson parameers a horzon h and ε s he h perod ahead forecas error. + h m and y are respecvely, naural logarhms of he relave money supply and relave real ncome beween he domesc and foregn counres; r s her neres rae dfferenal. More model specfcaon could be found n Q and Wu [8]. We sudy wo forecasng horzons (h= 6 monhs and monhs) n hs paper. In her rgorous examnaon, Q and Wu [8] provde srong emprcal evdence of he nably of exsng heorecal models of exchange rae deermnaon o ouperform a random walk n forecasng 6 and monh ahead changes n exchange raes even by usng neural neworks ncorporang model non-lneary and model uncerany. Usng neural nework and a well know sascal resamplng echnques - Boosrap, Whe and Racne [36] fnd ha exchange raes do appear o conan nformaon ha s exploable for enhanced pon predcon, bu he naure of he predcve relaons evolves hrough me. In he fnancal leraure, here are srong emprcal evdence show ha superor forecass could be obaned a longer horzons by allowng coeffcens o change (Schnas and Swamy [3] ) or by modelng boh long memory and srucural change (Morana and Belra [9]). In hs paper, we nvesgae he predcably of exchange rae over shor horzons by usng Bayesan srucural break models and change pon deecon mehods. Table. Descrpon of varables Varable Descrpon Arbue Name s exchange rae h perod Oupu + h change m relave money supply Inpu beween he domesc and foregn counres; s domesc currency prce of Inpu one un foregn currency y relave real ncome Inpu beween he domesc and foregn counres; r neres rae dfferenal Inpu beween he domesc and foregn counres; All daa are monhly and are obaned from IMF s Inernaonal Fnancal Sascs. Our sample sars n March 973 and ends n July 997 wh 9 observaons. We use he las 90 observaon as es daa se. We selec exchange raes beween he U.S. dollar and he Japanese yen, he Deusche mark and he Canadan dollar. Exchange raes are end-of-monh U.S. dollar prces of he foregn currences. Followng Q and Wu [8], we measure money supply by M, and real ncome by ndusral producon n each of he counres. We use Treasury-bll raes for Canada and he U.S. (lne 60c) and call money raes (lne 60b) for Germany and Japan as alernave measure of neres rae. The followng able shows he dependen varable and ndependen varables used n hs sudy M3 973 M 975 M 976 M9 978 M7 980 M5 98 M3 984 M 986 M 987 M9 989 M7 99 M5 993 M3 995 M 997 M 998 Germany Canada Fgure. End-of-monh U.S. dollar prces of Deusche mark and Canadan dollar from Mar. 973 o Dec M3 973 M 974 M9 976 M6 978 M3 980 M 98 M9 983 M6 985 M3 987 M 988 M9 990 M6 99 M3 994 M 995 M9 997 Japan Fgure. End-of-monh U.S. dollar prces of Japanese Yen from Mar. 973 o Dec. 998 Fgure and Fgure show he hsorcal rend of endof-monh U.S. dollar prces of Canadan dollars, Deusche 6

7 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences mark and Japanese Yen from Mar. 973 o Dec As has wdely been documened n he leraure, comparng o oher fnancal varables, exchange raes are characerzed wh hgher volaly, large nose and regme swchng. To predc exchange rae, we should explcly ake no accoun of he model uncerany and model nsably. The parameers of he neural nework are esmaed by mnmzng he sum of squared errors ε. We use + h Bayesan regularzaon, a modfcaon of he Levenberg Marquard ranng algorhm o produce neworks ha generalze well and o reduce he dffculy of deermnng he opmum nework archecure (MacKay [0]). Furhermore, o he npus o neural nework, we use one fourh of he daa as valdaon se and hree fourh of he daa as ranng se. We pck he ses as equally spaced pons hroughou he orgnal daa. I has already been wdely documened n he leraure ha he number of processng uns n hdden layer has grea mpac on he performance of neural nework s forecas. To avod he model msspecfcaon of neural nework, we also red dfferen confguraon of neural nework wh dfferen number of processng uns n he hdden layer and dfferen nal values of parameers. Inally we se he number of processng uns n he hdden layer as a large number (0) and hen we keep reducng he number of processng uns. Our conclusons are robus o dfferen neural nework archecure. To make he forecas from Bayesan model combnaon and neural nework more effcen, we scale he npus (ndependen varables) and arges (dependen varables) o boh Bayesan model averagng and neural nework,.e. we normalze he npus and arges so ha hey wll have zero mean and uny sandard devaon. 4. Forecasng performance comparson wh a fxed wndow sze We frs compare he es daa forecasng performance of Bayesan srucural models and neural nework by usng a fxed rollng wndow sze wh 0 mos recen observaons. Table and able 3 show he es daa se resul of forecasng U.S. dollar prces of Japanese Yen, Canadan dollar and Deusche mark predcon a and 6 monh horzon ahead. The resul s descrbed n percenage of RMSE (Roo of Mean Square Error * 00). Overall, We ls he predcon from random walk models, neural neworks, lnear models, srucural break models wh one homoscedasc break or one heeroscedasc break and he hose models combnaon based on her poseror probables. From able and able 3, we could see afer ncorporang model nsably, boh neural nework and Bayesan model combnaon could predc exchange rae of Japanese Yen and Canadan dollar very well. Overall, Bayesan srucural break model could mach he sae of he ar mehodology neural nework whch has been wdely acknowledged as a unversal approxmang mehodology. Some neresng observaon should be furher nerpreed. As we could see, o Deusche mark, he Bayesan srucural break model could no offer a superor approach. We would call for he specal aenon ha due o he me consran, we only ncorporae a very lmed number of models no Bayesan combnaon pools. For nsance, we dd no ncorporae he nonlnear models. We dd no ncorporae he srucural break models wh more han one break. Ignorng hese models could grealy affec he performance of he predcon of Deusche mark. Table. monh ahead U.S. dollar prces of Japanese Yen, Canadan dollar and Deusche mark predcon wh fxed rollng wndow sze Japan Canada Germany Lnear model () Homo srucural break model () Heo srucural break model (3) ()+() ()+()+(3) ANN Random Walk Table 3. 6 monh ahead U.S. dollar prces of Japanese Yen, Canadan dollar and Deusche mark predcon wh fxed rollng wndow sze Japan Canada Germany Lnear model () Homo srucural break model () Heo srucural break model (3) ()+() ()+()+(3) ANN Random Walk In fgure 3 and fgure 4, we show he margnal poseror probably a dfferen poenal break pons. As we could see n Fgure 3 and Fgure 4, here s srong evdence ha one break model probably s no approprae. A srucural model wh more han one break s more preferable accordng o he mul-modal margnal poseror probably. In oher words, he fxed rollng wndow sze wh 0 mos recen observaons may be oo long for he Bayesan srucural break models. An nellgen rollng wndow sze should be deeced o 7

8 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences mprove he Bayesan srucural break model f o he daa. 8.00E E E E E E-8.00E-8.00E-8 Margnal Model Poseror Probably Model fed o Germany Currency 0.00E Fgure 3. Margnal model poseror probably condonng on he parameers defnng I. 4.00E E E-30.50E-30.00E-30.50E-30.00E E-3 Margnal Model Poseror Probably Model 3 fed o Germany Currency 0.00E Fgure 4. Margnal model poseror probably condonng on he parameers defnng I. 4.3 Forecasng performance comparson wh he wndow sze deeced wh change deecon mehod In he comng expermens, we hen compare he es daa forecasng performance of Bayesan srucural models and neural nework by usng a rollng wndow sze deeced wh change pon deecon mehods. Table 4 and able 5 show he es daa se resul of forecasng U.S. dollar prces of Japanese Yen, Canadan dollar and Deusche mark predcon a and 6 monh horzon ahead. The resul s descrbed n percenage of RMSE (Roo of Mean Square Error). We could see by usng he wndow sze deeced wh change pon deecon mehod, he performance of forecasng Germany mark could be grealy mproved. Our Bayesan srucural break model negraed wh change pon deecon mehod could mach he approxmang ably of neural nework. Furhermore, f we only adop he Bayesan srucural models, we could see Bayesan srucural break models ouperform he neural nework n all he forecas. Fgure 5 show he rollng wndow sze deeced wh reversed ROC es n forecasng. We could see change pon deecon fnd srong evdence of Deusche mark s srucural break n he 980 s. Table 4. monh ahead U.S. dollar prces of Japanese Yen, Canadan dollar and Deusche mark predcon wh deeced wndow sze Japan Canada Germany Lnear model () Homo srucural break model () Heo srucural break model (3) ()+() ()+()+(3) ANN Random Walk Table 5. 6 monh ahead U.S. dollar prces of Japanese Yen, Canadan dollar and Deusche mark predcon wh deeced wndow sze Japan Canada Germany Lnear model () Homo srucural break model () Heo srucural break model (3) ()+() ()+()+(3) ANN Random Walk ROC Tes Deeced Wndow Szes 50 Wndow Szes Japanese Yen Canadan dollar Deusche mark Forecasng Pons Fgure 5. ROC es deeced wndow szes of 6 monh ahead U.S. dollar prces predcon of Japanese Yen, Canadan dollar and Deusche mark 8

9 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences Conclusons Daa mnng applcaon challenges lke non-saonary and model uncerany are prevalen n realy. In hs paper we propose an nvocave approach negrang Bayesan srucural break model and change pon deecon mehods. Our emprcal resuls provde srong suppor ha our proposed approach could mach he unversal approxmang mehodology arfcal neural neworks when here s srucural break n he evolvng daa. Our approach s superor performance s due o s capably o ncorporae model uncerany when here are mulple models and break pons compeng o explan he same daa se and model nsably when here are srucural break n he relaonshp beween he varables suded. As we could access more daa and more powerful compuer echnology, model nsably and model uncerany could become crcal ssues o he success of daa mnng applcaon. Our proposed approach has no only a clear nuve appeal bu also a farly frm sascal foundaon n solvng hese challenges. In hs sudy, we only ncorporae a very lmed number of model specfcaons. I s promsng ha we could fully benef from Bayesan model combnaon f we ncorporae more model specfcaon lke ouler models and nonlnear models specfed n secon.. We beleve Bayesan model combnaon offer a very compeve mehodology n explcly ncorporang model uncerany and model nsably. In a relaed work, we compare he performance of Bayesan model averagng, neural neworks wh Bayesan regularzaon, decson rees and suppor vecor machne n several fnancal predcon problems. Our resuls show ha Bayesan model combnaon s very compeve comparng o hose sae of he ar nonlnear approxmang mehodologes. Bayesan mehod punshes he model wh larger sze. As we could see, alhough model and model 3 (he models wh srucural break) have beer model f, lnear model aan hgher poseror probably because parsmonous sze. When we make Bayesan model combnaon, how many models are oo many s sll an open queson. However, hs only makes our resul more conservave. Many daa mnng applcaon felds lke fnancal me seres predcon are doman characerzed by nonsaonary, srong nose, weak sgnal and lack of funconal srucure. The emphass on organzaonal mpac and benef maxmzaon of daa mnng urges us o develop models ha could be undersood by manageral decson makers. Comparng o he radonal black box modelng, Bayesan srucural break models negraed wh change pon deecon mehods could offer beer nerpreably. Ths s a compeve advanage because manageral decson makers prefer o adop he models ha are easy o undersand and o assocae reurn wh rsk.. We plan o conduc more rgorous examnaon on he performance of our proposed approach n he fuure. For nsance, n addon o sandard sascal error measures, a radng smulaon would measure he benefs of echnology more accuraely. Fuure work les on more applcaon of Bayesan srucural break model negraed wh change pon deecon mehods o oher felds. For nsance, nruson deecon, cusomer purchasng behavor change analyss, healh scence and qualy conrol all could be promsng felds o apply our proposed approach. References [] B. Baesens, R. Seono, C. Mues, and J. Vanhenen, Usng Neural Nework Rule Exracon and Decson Tables for Credrsk Evaluaon, Managemen Scence, vol. 49, no. 3, 003, pp [] J. M. Baes, and C.W. J. Granger, The Combnaon of Forecass, Operaons Research Quarerly, vol. 0, 969, pp [3] M. Csorgo and L. Horvah, Lm Theorems n Change Pon Analyss, New York: John Wley & Sons, 997 [4] D.G. T. Denson, C.C. Holmes, B.K. Mallck, and A.F.M. Smh, Bayesan Mehods for Nonlnear Classfcaon and Regresson, John Wley & Sons, Ld, Wes Sussex, England. 00 [5] V. Dhar, D, Chou, and F. Provos, Dscoverng Ineresng Paerns for Invesmen Decson Makng wh Glower A Genec Learner Overlad wh Enropy Reducon, Daa Mnng and Knowledge Dscovery, 4, 000, pp [6] D. Draper, Assessmen and Propagaon of Model Uncerany, J. R. Sas. Soc. B. vol. 57 no., 995, pp [7] P. H. Franses and D. V. Djk, Non-lnear Tme Seres Models n Emprcal Fnance, The Unversy of Cambrdge, Cambrdge, Uned Kngdom, 000. [8] E. I. George and D. P. Foser, Calbraon and Emprcal Bayes Varable Selecon, Bomerka, vol. 87, 000, pp [9] C. Granger and A. Tmmermann, Daa Mnng wh Local Model Specfcaon Uncerany: A Dscusson of Hoover and Perez, Economercs Journal, 000, pp. 0-5 [0] D. Hand, H. Mannla and P. Smyh, Prncple of Daa Mnng. MIT Press, Cambrdge: Massachuses, 00. [] M. B. Harres and C. Sammu, Exracng Hdden Conex, Machne Learnng, vol. 3, pp. 998, 0-6 9

10 Proceedngs of he 38h Hawa Inernaonal Conference on Sysem Scences [] T. Hase, R. Tbshran and J. Fredman, The Elemens of Sascal Learnng: Daa Mnng, Inference, and Predcon, New York: Sprnger. 00. [3] C.C. Holmes, and D.G.T. Denson, Classfcaon wh Bayesan MARS, Machne Learnng, vol. 50, 003, pp [4] K. Hornk, M. Snchcombe and H. Whe, Mullayer Feed Forward Neworks Are Unversal Approxmaors, Neural Neworks, 989, pp [5] G. Koop and S. Poer, Nonlneary, Srucural Breaks, or Oulers n Economc Tme Seres?, pp n Nonlnear conomerc Modelng n Tme Seres Analyss Proceedngs of he Elevenh Inernaonal Symposum n Economc Theory and Economercs Eded by W. A. Barne, D. F. Hendry, S. Hylleberg, T. Teräsvra, D. TjØshem, and A. Würz Cambrdge UK, 000 [6] A. Lo, Daa-Snoopng Bases n Fnancal Analyss, n H. R. Fogler, ed.: Blendng Quanave and Tradonal Equy Analyss, 994. Charloesvlle, VA: Assocaon for Invesmen Managemen and Research. [7] M. Lubrano, Bayesan Analyss of Nonlnear Tme Seres Models wh a Threshold, 79-8 n Nonlnear Economerc Modelng n Tme Seres Analyss Proceedngs of he Elevenh Inernaonal Symposum n Economc Theory and Economercs Eded by W. A. Barne, D. F. Hendry, S. Hylleberg, T. Teräsvra, D. TjØshem, and A. Würz Cambrdge UK, 000 [8] K. Meha and S. Bhaacharyya, Adequacy of Tranng Daa for Evoluonary Mnng of Tradng Rules, Decson Suppor Sysems, 37, 004, pp [9] C. Morana and A. Belra, Srucural Change and Longrange Dependence n Volaly of Exchange Raes: Eher, Neher or Boh, Journal of Emprcal Fnance, forhcomng. [0] D.J.C., MacKay, Bayesan Inerpolaon, Neural Compuaon, vol. 4, 99, pp [] K. J. Oh and I. Han, Usng Change-pon Deecon o Suppor Arfcal Neural Neworks for Ineres Raes Forecasng, Exper sysems wh applcaon, vol. 9, 000, pp [] K. J. Oh and I. Han, An Inellgen Cluserng Forecasng Sysem based on Change-Pon Deecon and Arfcal Neural Neworks: Applcaon o Fnancal Economcs, Proceedngs of he 34 h Hawa Inernaonal Conference on Sysem Scences 00. [3] B. Padmanabhan, and A. Tuzhln, On he Use of Opmzaon for Daa Mnng: Theorecal Ineracons and ecrm Opporunes, Managemen Scence, vol. 49, no. 0, 003, pp [4] A. Pasley and J. Ausn, Dsrbuon Forecasng of Hgh Frequency Tme Seres, Decson Suppor Sysems, vol. 37, 004. pp [5] M. H. Pesaran and A. Tmmermann, A Marke Tmng and Reurn Predcon under Model Insably, Journal of Emprcal Fnance, vol. 9, 00, pp [6] M. H. Pesaran and A. Tmmermann, How Cosly Is I o Ignore Breaks When Forecasng he Drecon of A Tme Seres? Inernaonal Journal of Forecasng, In Press, [7] A. N. Pe, Some Resuls on Esmang a Change-Pon Usng nonparamerc ype sascs, Journal of Sascal Compuaon and Smulaon,, 980, pp. 6-7 [8] M. Q and Y. Wu, Nonlnear Predcon of Exchange Raes wh Moneary Fundamenals, Journal of Emprcal Fnance, vol. 0, 003, pp [9] T. K. Sung, N. Chang, and G. Lee, Dynamcs of Modelng n Daa Mnng: Inerpreve Approach o Bankrupcy Predcon, Journal of Managemen Informaon Sysems, vol. 6, no., 999, pp [30] J. C. Schlmmer and R. H. Granger, JR., Incremenal Learnng from Nosy Daa, Machne Learnng, vol., 986, pp [3] G. J. Schnas and P. A. V. B. Swamy, The Ou-of-sample Forecasng Performance of Exchange Rae Models When Coeffcens Are Allowed o Change, Journal of Inernaonal Money and Fnance, vol. 8, no , pp [3] A. N. Srvasava, R, Su and A. W. Wegend, Daa Mnng for Feaures Usng Scale-Sensve Gaed Expers, IEEE Transacons on Paern Analyss and Machne Inellgence, vol., no. 999, pp [33] A. Tmmermann and C. W. J. Granger, Effcen Marke Hypohess and Forecasng, Inernaonal Journal of Forecasng, vol. 0, no., 004, pp [34] A.S. Wegend, M. Mangeas, and A. N. Srvasava, Nonlnear Gaed Expers for Tme Seres: Dscoverng Regmes and Avodng Overfng, Inernaonal Journal of Neural Sysems, vol. 6, 995, pp [35] G. Wdmer, Trackng Conex Changes hrough Mea- Learnng, Machne learnng, vol. 7, 997, pp [36] H. Whe, and J. Racne, Sascal Inference, he Boosrap, and Neural-nework Modelng wh Applcaon o Foregn Exchange Raes, IEEE Transacon on Neural Nework, vol., no. 4, 00, pp