PERFORMANCE COMPARISON OF TIME SERIES DATA USING PREDICTIVE DATA MINING TECHNIQUES

Transcription

1 , pp Available olie a hp:// PERFORMANCE COMPARISON OF TIME SERIES DATA USING PREDICTIVE DATA MINING TECHNIQUES SAIGAL S. 1 * AND MEHROTRA D. 2 1Deparme of Compuer Sciece, Krisha Egieerig College, Ghaziabad , UP, Idia. 2Deparme of MCA, Amiy School of Compuer Scieces, Noida , UP, Idia. *Correspodig Auhor: - saigalsheeal9@gmail.com Received: November 09, 2012; Acceped: December 03, 2012 Absrac- This paper focuses o he mehodology used i applyig he Time Series Daa Miig echiques o fiacial ime series daa for calculaig currecy exchage raes of US dollars o Idia Rupees. Four Models amely Muliple Regressio i Excel, Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka, Vecor Auoregressive Model i R ad Neural Nework Model usig NeuralWorks Predic are aalyzed. All he models are compared o he basis of he forecasig errors geeraed by hem. Mea Error (ME), Mea Absolue Error (MAE), Mea Squared Error (MSE), Roo Mea Square Error (RMSE), Mea Perceage Error (MPE) ad Mea Absolue Perceage Error (MAPE) are used as a forecas accuracy measure. Resuls show ha all he models accuraely predic he exchage raes, bu Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka ouperforms he oher hree models. Keywords- Exchage Rae Predicio, Time Series Models, Regressio, Predicive Daa Miig, Weka, VAR, Neural Nework Ciaio: Saigal S. ad Mehrora D. (2012) Performace Compariso of Time Series Daa Usig Predicive Daa Miig Techiques. Advaces i Iformaio Miig, ISSN: & E-ISSN: , Volume 4, Issue 1, pp Copyrigh: Copyrigh 2012 Saigal S. ad Mehrora D. This is a ope-access aricle disribued uder he erms of he Creaive Commos Aribuio Licese, which permis uresriced use, disribuio ad reproducio i ay medium, provided he origial auhor ad source are credied. Iroducio Oe of he mos eicig applicaio areas of daa miig i hese emergig echologies is i fiace, becomig more ameable o daa-drive modelig as large ses of fiacial daa become available. I he field of fiace he exesive use of daa miig applicaios icludes he area of forecasig sock marke, pricig of corporae bods, udersadig ad maagig fiacial risk, radig fuures, predicio of exchage raes, credi raig ec. Mohly daa is colleced for he las 10 years from 2000 o 2010, for predicig exchage raes of 2011 [5,14,16].The origial rae of 2011 is available ad he compared wih he prediced values for calculaig he accuracy of he models. Mea Error (ME), Mea Absolue Error (MAE), Mea Squared Error (MSE), Roo Mea Square Error (RMSE), Mea Perceage Error (MPE) ad Mea Absolue Perceage Error (MAPE) is used as a forecas accuracy measure. The muliple variables used o which exchage rae depeds are CPI, Trade Balace (i millio US dollars), GDP, Uemployme ad Moeary Base (i billio dollars) [16]. Four Models amely Muliple Liear Regressio i Excel [14], Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka [6, 9], Vecor Auoregressive Model i R [6-8, 10] ad Neural Nework Model [4,5,13-15] usig NeuralWorks Predic are aalyzed. All he models are compared o he basis of he forecasig errors geeraed by hem. The paper is orgaized as follows. Secio II covers predicive daa miig. Secio III covers he four predicive ime series models amely Muliple Liear Regressio i Excel, Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka, Vecor Auoregressive Model i R ad Neural Nework Model usig Neural- Works Predic. Secio IV covers he daases used for he aalysis ad he seps ad resuls obaied by usig he four models. Secio V shows he performace compariso of he four models usig Mea Error (ME), Mea Absolue Error (MAE), Mea Squared Error (MSE), Roo Mea Square Error (RMSE), Mea Perceage Error (MPE) ad Mea Absolue Perceage Error (MAPE). Secio VI cocludes he work, followed by refereces i he las secio. Predicive Daa Miig Predicive daa miig aalyzes daa i order o cosruc oe or a se of models ad aemps o predic he behavior of ew daa ses. Predicio is a form of daa aalysis ha ca be used o exrac models describig impora daa classes or o predic fuure daa reds. Such aalysis ca help provide us wih a beer udersadig of he daa a large. Predicio ca also be viewed as a mappig or fucio, y = f (X), where X is he ipu (e.g., a uple describig a loa applica), ad he oupu y is a coiuous or ordered value (such as he prediced amou ha he bak ca safely loa he applica); Tha is, we wish o lear a mappig or fucio ha models he relaioship bewee X ad y. There are wo issues regardig predicio: firs is preparig he daa for predicio which ivolves he preprocessig seps like daa cleaig, relevace aalysis, daa rasformaio ad daa reducio, secod issue is comparig he differe predicio models. The models are compared accordig o he crieria give below: Bioifo Publicaios 57

2 Performace Compariso of Time Series Daa Usig Predicive Daa Miig Techiques Accuracy: The accuracy of a predicor refers o how well a give predicor ca guess he value of he prediced aribue for ew or previously usee daa. Speed: This refers o he compuaioal coss ivolved i geeraig ad usig he give predicor. Robusess: This is he abiliy of he predicor o make correc predicios give oisy daa or daa wih missig values. Scalabiliy: This refers o he abiliy o cosruc he predicor efficiely give large amous of daa. Ierpreabiliy: This refers o he level of udersadig ad isigh ha is provided by he predicor. Alhough accuracy, speed, robusess, scalabiliy ad ierpreabiliy are he various facors for comparig he predicio models, bu i his paper he predicio models are compared o he basis of heir accuracy. Measures like ME, MAE, MSE, RMSE, MPE ad MAPE are used o compare he performace of predicive models [18]. Predicive daa-miig ools are desiged o help us udersad wha he useful iformaio looks like ad wha has happeed durig pas procedures. Therefore, he ools use he descripio of he useful iformaio o fid similar examples of hidde iformaio i he daabase ad use he iformaio leared from he pas o develop a predicive model of wha will happe i he fuure. Differe predicive models are aalyzed ad he bes model is chose for predicig he daa. [Fig-1] below shows ha he bes model is chose amog differe sized models o ge required soluio. Fig. 1- Fid he bes model amog predicive model[17] Predicive Time Series Models A Time Series is a ime-orieed or chroological sequece of observaios o a variable of ieres. Time series aalysis is he process of usig saisical echiques o model ad explai a imedepede series of daa pois. Time series forecasig is he process of usig a model o geerae predicios (forecass) for fuure eves based o kow pas eves. Time series daa has a aural emporal orderig - his differs from ypical daa miig/ machie learig applicaios where each daa poi is a idepede example of he cocep o be leared, ad he orderig of daa pois wihi a daa se does o maer. Time series predicio proposes algorihms for which pas daa (maily fiie observaio sequeces of daa pois relaed o uiform ime ierval) are used o geerae models o forecas fuure daa pois of he series [1-3,11,12]. There are oly wo broad ypes of forecasig echiques- Qualiaive mehods ad Quaiaive mehods. Qualiaive forecasig echiques are ofe subjecive i aure ad require judgme o he par of expers. Quaiaive forecasig echiques make formal use of hisorical daa ad a forecasig model. I his aalysis, four models amely Muliple Liear Regressio i Excel, Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka, Vecor Auoregressive Model i R ad Neural Nework Model usig NeuralWorks Predic is aalyzed. Muliple Liear Regressio i Excel Regressio models make use of relaioships bewee he variable of ieres ad oe or more relaed predicor variables. Someimes regressio models are called causal forecasig models, because he predicor variables are assumed o describe he forces ha cause or drive he observed values of he variable of ieres. Regressio aalysis is a saisical echique for modelig ad ivesigaig he relaioships bewee a oucome or respose variable ad oe or more predicor or regressor variables. The ed resul of a regressio aalysis sudy is ofe o geerae a model ha ca be used o forecas or predic fuure values of he respose variable give specified values of he predicor variables. The Muliple Liear Regressio Model is 2 y β β x β (1) 0 1 1x The parameers β0, β1 β2 i his model are ofe called parial regressio coefficies because hey covey iformaio abou he effec o y of he predicor ha hey muliply give ha all of he oher predicors i he model do o chage. The regressio models i [Eq-1] is liear regressio models because hey are liear i he ukow parameers (he β's), ad o because hey ecessarily describe liear relaioships bewee he respose ad he regressors. Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka WEKA, formally called Waikao Evirome for Kowledge Learig, is a compuer program ha was developed a he Uiversiy of Waikao i New Zealad for he purpose of ideifyig iformaio from raw daa gahered from agriculural domais. WEKA suppors may differe sadard daa miig asks such as daa preprocessig, classificaio, cluserig, regressio, visualizaio ad feaure selecio. Weka (>= 3.7.3) ow has a dedicaed ime series aalysis evirome ha allows forecasig models o be developed, evaluaed ad visualized. This evirome akes he form of a plugi ab i Weka's graphical "Explorer" user ierface ad ca be isalled via he package maager. Weka's ime series framework akes a machie learig/daa miig approach o modelig ime series by rasformig he daa io a form ha sadard proposiioal learig algorihms ca process. I does his by removig he emporal orderig of idividual ipu examples by ecodig he ime depedecy via addiioal ipu fields. These fields are someimes referred o as "lagged" variables. Various oher fields are also compued auomaically o allow he algorihms o model reds ad seasoaliy. Afer he daa has bee rasformed, ay of Weka's regressio algorihms ca be applied o lear a model. A obvious choice is o apply muliple liear regressio, bu ay mehod capable of predicig a coiuous arge ca be applied - icludig powerful o-liear mehods such as suppor vecor machies for regressio ad model rees. This approach o ime series Bioifo Publicaios 58

3 Saigal S. ad Mehrora D. aalysis ad forecasig is ofe more powerful ad more flexible ha classical saisical echiques such as ARMA ad ARIMA. Vecor Auoregressive Model i R VAR aalysis has evolved as a sadard isrume i ecoomerics for aalyzig mulivariae ime series. A VAR cosiss of a se of K edogeous variables y = (y1, yk,.. yk) for k = 1,... K. The VAR (p)-process is he defied as: y A1 y 1... Ap y p u (2) wih Ai are (K K) coefficie marices for i = 1,..., p ad u is a K- dimesioal whie oise process wih ime ivaria posiive defiie covariace marix E (uu ) = u. [Eq-2] is someimes wrie i he form of a lag polyomial A( L) ( lk A1... Ap ) as A( L) y CD u (3) where he marix C is he coefficie marix of poeially deermiisic regressors wih dimesio (K M), ad D is a (M 1) colum vecor holdig he appropriae deermiisic regressors, such as a cosa, red, ad dummy ad/or seasoal dummy variables Oe impora characerisic of a VAR (p)-process is is sabiliy. The sabiliy of a empirical VAR(p)-process ca be aalyzed by cosiderig he compaio form ad calculaig he eigevalues of he coefficie marix. If he moduli of he eigevalues of A are less ha oe, he he VAR (p)-process is sable. The iformaio crieria are implemeed i he fucios VAR () ad VARselec () coaied i he package vars. I he former fucio, a appropriae VAR (p)-model will be esimaed by providig he maximal lag umber, lag.max, ad he desired crierio. The calculaios are based upo he same sample size. Tha is, lag.max values are used as sarig values for each of he esimaed models. The resul of he fucio VARselec () is a lis objec wih elemes selecio ad crieria. The eleme selecio is a vecor of opimal lag legh accordig o he above-meioed iformaio crieria. The eleme crieria is a marix coaiig he paricular values for each of hese crieria up o he maximal lag order chose. Neural Nework Model usig NeuralWorks Predic Arificial eural eworks were ispired i large par by research io he fucio of euros i he huma brai. Arificial eural eworks process iformaio i a way ha resembles he way he brai works. Like he brai, eural eworks lear from experiece durig a process called raiig. We ca use eural eworks whe ukow relaioships exis i hisorical daa. A hisorical daase cosiss of recorded ipu values ad heir correspodig oupu values. Neural eworks ca deec paers i daa, geeralize abou relaioships i daa, ad geerae a oupu value whe give a ew se of ipu values from he problem domai. Aalyss who lack exesive domai kowledge ca use eural eworks o solve problems ha prove oo complex for more coveioal aalyical echiques. How a Neural Nework Lears The huma brai is a very complex sysem of iercoeced euros. Similarly, a eural ework is a iercoeced sysem of arificial euros. I eural ework ermiology, euros are called Processig Elemes or odes. Like a euro i he brai, each Processig Eleme (PE) ca accep ipu daa, process he daa, ad pass i o he ex PE. A PE processes daa usig oe of several ypes of mahemaical fucios. I effec a eire eural ework represes a composie of he fucios represeed by all PEs. The key o buildig a robus eural ework is o collec may examples (or records) of ipu values ad correspodig oupu values over ime. The eural ework uses his hisorical daa o deermie (lear) a mahemaical relaioship bewee he ipu daa ad he oupu daa. Nework Archiecure I a eural ework, PEs ca be iercoeced i various ways. Typically, PEs are srucured io layers ad he oupu values of PEs i oe layer serve as ipu values for PEs i he ex layer. Each coecio has a weigh associaed wih i. I mos cases, a Processig Eleme calculaes a weighed sum of icomig values (he sum he oupus of he PEs coeced from he ex lower level muliplied by heir coecio weighs). This sum is called he acivaio value. The acivaio value is he passed o he PE s o-liear rasfer fucio o produce a oupu for ha PE. This combiaio of PEs, coecios, weighs, ad rasfer fucios form he ework archiecure. This archiecure he represes a complex mahemaical formula ha has bee derived from hisorical daa. Traiig a Neural Nework Neural eworks lear from experiece (exposure o iformaio). From repeaed exposure o hisorical daa, a eural ework lears o sreghe coecio weighs from PEs ha have a greaer edecy o accuraely predic he desired oupu. The sregh of each coecio (he magiude of he coecio weigh) icreases or decreases based o is ifluece i producig he oupu associaed wih each ipu daa record i he hisorical daase. Coecio weighs are adjused durig raiig. Traiig is he process of repeaedly (ieraively) exposig a eural ework o examples of hisorical daa. Each example coais ipu variables, ad he associaed desired oupu. The desired oupu is he value ha he eural ework should predic, give he associaed ipu values i he hisorical daa. I eural ework ermiology, he desired oupu is called he arge value. Durig raiig, ework weighs are adjused accordig o a learig rulea algorihm ha specifies how he error bewee prediced oupus ad arge values should be used o modify ework weighs. Before raiig begis, a simple ework wih he ecessary umber of ipu ad oupu PEs is creaed. All coecio weighs are iiialized o small radom values. Daa examples from he raiig se are passed o he ework ad he ework produces a oupu value. This value is compared o he arge value. The weighs are adjused i order o decrease he error bewee he ework oupu ad he arge value. I addiio, more processig elemes are added o he ework if doig so helps decrease he error of he ework. Traiig coiues uil he eural ework produces oupu values ha mach he arge values wihi a specified accuracy level, or uil some oher soppig crierio is saisfied. Tesig ad Validaig a Neural Nework Jus as you migh es a perso s skill i a corolled evirome, you es a eural ework usig hisorical daa i has o see. Tes Bioifo Publicaios 59

4 Performace Compariso of Time Series Daa Usig Predicive Daa Miig Techiques resuls are good if he prediced values are close o he arge values. Oe difficuly for eural eworks (ad oher o-liear esimaio echiques) is he possibiliy ha he ework will over-fi he raiig daa. This meas ha he ework migh closely predic he arge values o he raiig daa bu produce iferior resuls for ew daa. Traiig for oo log (oo may passes hrough he raiig se) ca cause he fucio o become very complex i order o produce he arge values a he expese of geeralizig well o usee daa. However, if a ework is o raied log eough, i does fully lear reds ad relaioships i he daa. Oe way of kowig whe o sop raiig is o auomaically ad periodically es he performace o a es se durig raiig. Whe he performace sars o degrade o he es se, i's ime o sop raiig-he eural ework has sared o lear relaioships which are specific o oly he raiig se! A he ed of raiig, he eural ework ca be furher esed o a addiioal idepede es se referred o as a validaio se. Usig a Neural Nework Afer he eural ework has bee raied, i ca be used o make predicios give ipu daa for which he desired oupu value is ukow. NeuralWorks Predic NeuralWorks Predic is a complee applicaio developme evirome for creaig ad deployig real-ime applicaios for forecasig, modelig, classificaio ad cluserig or groupig. This powerful sysem combies eural ework echology wih fuzzy logic, saisics ad geeic algorihms o fid soluios o forecasig, modelig ad classificaio problems auomaically. Key feaures of NeuralWorks Predic are: NeuralWorks Predic Udersads Daa: NeuralWorks Predic kows how o rasform daa io forms ha maximize he power of eural ework echiques. This saves ime ad also resuls i beer performace. NeuralWorks Predic Applies he Rigors of Saisics o Model he Real World: The algorihms used i NeuralWorks Predic are grouded i saisics. Key coceps such as ridge regressio, maximum likelihood ad cross-validaio are seamlessly iegraed wih eural echiques. NeuralWorks Predic models are saisically based eural models. Simple is Beer: NeuralWorks Predic ackles complex problems, ideifies he mos salie feaures ad builds jus he righ size model o solve your problem. NeuralWorks Predic has bee used o sor hrough housads of variables. Simple models perform beer o ew daa, are faser ad more reliable ha more complex models buil by rial ad error. NeuralWorks Predic Models are a Hybrid of Polyomial Regressio, Fuzzy Logic ad Neural Neworks: This icludes polyomial eural ad fuzzy-eural regressio, daa disribuio compesaio, o-parameric oulier deecio ad rasformaio, sochasic gradie search ad mixed-mode hidde-layer fucios (sigmoid, ah, sie, expoeial, liear). NeuralWorks Predic picks he righ buildig blocks o build he bes soluio. NeuralWorks Predic Hadles he Big Problems: The commad lie versio of NeuralWorks Predic elimiaes he barriers o daa se size allowig 4,000 fields. The umber of records is oly limied by he sysem s memory. Empirical Resuls Daases Time Series Daa Miig mehods are applied o fiacial ime series for calculaig currecy exchage raes of US dollars o Idia Rupees. Mohly daa is colleced for he las 10 years from 2000 o 2010, for predicig exchage raes of The origial rae of 2011 is available ad he compared wih he prediced values for calculaig he accuracy of he models. The muliple variables used o which exchage rae depeds are CPI, Trade Balace (i millio US dollars), GDP, Uemployme ad Moeary Base (i billio dollars). The daa for all he variables is colleced from various fiacial sies like radigecoomics, ychars, cesus.gov ec. The explaaio of he variables is: CPI: Cosumer Price Idex (CPI) daa measures iflaio i a ecoomy. Whe iflaio ges oo high i a coury, he Ceral bak may icrease ieres raes i order o esure price sabiliy. This may cause he currecy o rise i value as he addiioal ieres received makes he currecy more desirable. Trade Balace: A coury's balace of rade is he oal value of is expors, mius he oal value of is impors. If his umber is posiive, he coury is said o have a favorable balace of rade. If he differece is egaive, he coury has a rade gap, or rade defici. GDP: Gross Domesic Produc is a measure of he overall ecoomic oupu wihi a coury s borders over a paricular ime, ypically a year. GDP is calculaed by addig ogeher he oal value of aual oupu of all ha coury s goods ad services. GDP ca also be measured by icome by cosiderig he facors producig he oupu-he capial ad labor-or by expediure by goverme, idividuals, ad busiess o ha oupu Uemployme: Employme levels have a immediae impac o ecoomic growh. A icrease i uemployme sigals a slowdow i he ecoomy ad possible devaluaio of a coury's currecy because of decliig cofidece ad lower demad. Moeary Base: I ecoomics, he moeary base (also base moey, moey base, high-powered moey, reserve moey, or, i he UK, arrow moey) is a erm relaig o (bu o beig equivale o) he moey supply (or moey sock), he amou of moey i he ecoomy. The moeary base is highly liquid moey ha cosiss of cois, paper moey (boh as bak vaul cash ad as currecy circulaig i he public), ad commercial baks' reserves wih he ceral bak. Seps ad Resuls of Time Series Daa The seps of he models ad he correspodig resuls obaied by usig he four models are: Muliple Liear Regressio i Excel Seps for performig his Model are: Load he daa i excel file. Geerae he regressio equaio by applyig he Muliple Liear Regressio. Forecas he daa for he specified year. Compare he forecased value wih he acual value. Bioifo Publicaios 60

5 Saigal S. ad Mehrora D. Compue ME, MAE, MSE, RMSE, MPE ad MAPE. [Fig-2] represes graph showig compariso bewee acual exchage rae of 2011 (show by blue lie) ad prediced exchage rae of 2011 (show by red lie) usig Muliple Liear Regressio i Excel. The x-axis of he graph shows ime period i mohs ad he y-axis of he graph shows exchage rae values. Fig. 2- Graph showig compariso bewee acual exchage rae of 2011 wih prediced exchage rae of 2011 usig Excel The mohwise acual exchage rae of 2011 is kow o us. [Table-1] shows he acual ad prediced exchage rae of 2011 mohwise usig Muliple Liear Regressio i Excel. Table 1- Compares he acual exchage rae of 2011 wih prediced exchage rae of 2011 Time Period Acual Exchage Rae Prediced Exchage Rae Ja Feb Mar Apr May Ju Jul Aug Sep Oc Nov Dec The regressio equaio obaied by applyig Muliple Liear Regressio i Excel is: Exchage Rae = *CPI *Trade Balace *GDP *Uemployme *MoeoryBase Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka Seps for performig his Model are: Load he daa io Weka s Explorer. Selec Advaced Cofiguraio pael. I gives he user full corol over a umber of aspecs of he forecasig aalysis. These iclude he choice of uderlyig model ad parameers, creaio of lagged variables, creaio of variables derived from a dae ime samp, specificaio of "overlay" daa, evaluaio opios ad corol over wha oupu is creaed. Selec Base learer pael. I provides corol over which Weka learig algorihm ad is used o model he ime series. I also allows he user o cofigure parameers specific o he learig algorihm. Selec liear suppor vecor machie for Regressio (Weka's SMOreg). Selec he Lag creaio pael. I allows he user o corol ad maipulae how lagged variables are creaed. Lagged variables are he mai mechaism by which he relaioship bewee pas ad curre values of a series ca be capured by proposiioal learig algorihms. For mohly daa lags upo 12 imes ahead are creaed. I Lag Creaio pael, selec Adjus for variace check box which allows he user o op o have he sysem compesae for variace i he daa. I does his by akig he log of each arge before creaig lagged variables ad buildig he model. This ca be useful if he variace (how much he daa jumps aroud) icreases or decreases over he course of ime. Adjusig for variace may, or may o, improve performace. Selec he Evaluaio pael, i allows he user o selec which evaluaio merics hey wish o see, ad cofigure wheher o evaluae usig he raiig daa ad/or a se of daa held ou from he ed of he raiig daa. The available merics are Mea absolue error (MAE), Mea squared error (MSE), Roo mea squared error (RMSE), Mea absolue perceage error (MAPE), Direcio accuracy (DAC), Relaive absolue error (RAE), Relaive absolue error (RAE) ad Roo relaive squared error (RRSE). Selec checkboxes of required merics. Selec he Oupu pael, i provides opios ha corol wha exual ad graphical oupu are produced by he sysem. Selecig Oupu predicios a sep causes he sysem o oupu he acual ad prediced values for a sigle arge a a sigle sep. Selecig Oupu fuure predicios beyod he ed of series will cause he sysem o oupu he raiig daa ad prediced values (up o he maximum umber of ime uis) beyod he ed of he daa for all arges prediced by he forecaser. The predicios a a specific sep ca be graphed by selecig he Graph predicios a sep check box. Selecig he Graph arge a seps checkbox allows a sigle arge o be graphed a more ha oe sep - e.g. a graph ca be geeraed ha shows 1-sep-ahead, 2-sep-ahead ad 5-sep ahead predicios for he same arge. The seleced resuls will be displayed by ruig he Model. Compare he forecased value wih he acual value. Compue ME, MAE, MSE, RMSE, MPE ad MAPE. [Fig-3] represes graph showig compariso bewee acual exchage rae of 2011 (show by blue lie) ad prediced exchage rae of 2011 (show by red lie) usig Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka. The x-axis of he graph shows ime period i mohs ad he y-axis of he graph shows exchage rae values. The mohwise acual exchage rae of 2011 is kow o us. [Table-2] shows he acual ad prediced exchage rae of 2011 mohwise usig Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka [Fig-4], [Fig-5] ad [Fig-6] shows he graphs showig exchage raes of 2000 o 2011, predicios a seps ad predicios a arges respecively. These graphs are geeraed by Dedicaed Time Series Aalysis i Weka. Bioifo Publicaios 61

6 Performace Compariso of Time Series Daa Usig Predicive Daa Miig Techiques Fig. 3- Graph showig compariso bewee acual exchage rae of 2011 wih prediced exchage rae of 2011 usig Weka Table 2- Compares he acual exchage rae of 2011 wih prediced exchage rae of 2011 Time Period Acual Exchage Rae Prediced Exchage Rae Ja Feb Mar Apr May Ju Jul Aug Sep Oc Nov Dec Fig. 6- Weka graph predicio a arges [Fig-7] shows evaluaio resuls usig mea absolue error (MAE), roo mea square error (RMSE), mea absolue perceage error (MAPE) ad mea squared error (MSE) for each sep ahead i Weka. Fig. 7- Weka screesho of evaluaio resuls usig various performace measures. Fig. 4- Graph showig exchage rae for years 2000 o 2011 Fig. 5- Weka graph predicio a sep Vecor Auoregressive Model i R Seps for performig his Model are: Load he vars package. Load he daa available i csv forma. Plo he ime series daa of each variable. A opimal lag-order ca be deermied accordig o a iformaio crieria or he fial predicio error of a VAR (p) wih he fucio VARselec (). This fucio reurs a lis objec wih he opimal lag-order accordig o each of he crieria, as well as a marix coaiig he values for all lags up o lag.max. I a ex sep, he VAR (lag.max) is esimaed wih he fucio VAR () ad as deermiisic regressors a cosa is icluded. Names () fucio reurs a lis objec of class vares. Summary () mehod gives he regressio resuls for various variables equaios. A plo cosisig of a diagram of fi, a residual plo, he auocorrelaio ad parial auocorrelaio fucio of he residuals. Bioifo Publicaios 62

7 Saigal S. ad Mehrora D. We should check sabiliy of VAR (p) process. Here, sabiliy does o refer o he coefficies sabiliy, i.e. he sabiliy of he regressios, bu raher he sabiliy of he sysem of differece equaios. As poied ou above, if he moduli of he eigevalues of he compaio marix are less ha oe, he sysem is sable. The moduli of he eigevalues are foud usig roos () fucio. A predic-mehod for objecs wih class aribue vares is available for forecasig. The.ahead forecass are compued recursively for he esimaed VAR, begiig wih h = 1, 2,...,. ahead. The defaul value of forecas cofidece ierval is The predic-mehod does reur a lis objec of class varprd wih hree elemes. The firs eleme, fcs, is a lis of marices coaiig he prediced values, he lower ad upper bouds accordig o he chose cofidece ierval, ci ad is size. The secod eleme, edog is a marix objec coaiig he edogeous variables ad he hird is he submied vares objec. A plo-mehod for objecs of class varprd does exis as well as a fachar() fucio for ploig fa chars. The fa char () fucio has colors ad cis argumes, allowig he user o ipu vecors of colors ad criical values. If hese argumes are lef NULL, he as defauls a hea.map color scheme is used ad he criical values are se from 0.1 o 0.9 wih a sep size of 0.1. Compare he forecased value wih he acual value. Compue ME, MAE, MSE, RMSE, MPE ad MAPE. [Fig-8] represes graph showig compariso bewee acual exchage rae of 2011 (show by blue lie) ad prediced exchage rae of 2011 (show by red lie) usig Vecor Auoregressive Model i R. The x-axis of he graph shows ime period i mohs ad he y-axis of he graph shows exchage rae values. Table 3- Prediced currecy exchage rae 2011 usig R FCST LOWER UPPER CI Table 4- Compares he acual exchage rae of 2011 wih prediced exchage rae of 2011 Time Period Acual Exchage Rae Prediced Exchage Rae Ja Feb Mar Apr May Ju Jul Aug Sep Oc Nov Dec Fig. 9- Graph showig forecas of prediced exchage raes 2011 Fig. 8- Graph showig compariso bewee acual exchage rae of 2011 wih prediced exchage rae of 2011 usig R [Table-3] shows fcs which is he lis of prediced exchage raes of 2011, he lower ad upper rages for he resul ad he cofidece ierval. The mohwise acual exchage rae of 2011 is kow o us. [Table-4] shows he acual ad prediced exchage rae of 2011 mohwise usig Vecor Auoregressive Model i R. [Fig-9] ad [Fig-10] shows he graphical represeaio ad fachar represeaio of forecas of prediced exchage rae of 2011 usig Vecor Auoregressive Model i R respecively. Fig. 10- Fa char forecas of prediced exchage rae 2011 Bioifo Publicaios 63

8 Performace Compariso of Time Series Daa Usig Predicive Daa Miig Techiques Neural Nework Model usig NeuralWorks Predic Seps of applyig he model are Loadig daa io Excel. Specifyig he problem ype. Specifyig daa ad ework characerisics. Savig he model. Traiig he model. Tesig he model. Aalyzig he resuls. Ruig ew daa hrough your model. Compue ME, MAE, MSE, RMSE, MPE ad MAPE. The mohly daa of year 2000 o 2010 is used by he model, so a oal of 120 records are accessed, ou of which 83 records are used by he raiig daa ad 37 records are used by es daa. [Table-5] shows he followig saisical resuls usig Neural Nework Model: R- The liear correlaio bewee he arge values ad he correspodig prediced oupu values. Ne-R- The liear correlaio bewee he arge values ad he raw ework oupu values (before hey are rasformed io he measureme uis of he problem). Avg. Abs.- The average absolue error bewee prediced oupu values ad he correspodig arge values. Max. Abs.- The maximum absolue error bewee prediced oupu values ad he correspodig arge values. RMS- The roo mea square error bewee prediced oupu values ad he correspodig arge values. Accuracy- The perceage of prediced oupu values ha are wihi he specified olerace (20%) of he correspodig arge values. Cof. Ierval- The cofidece ierval is he rage [arge value ± cofidece ierval] wihi which he correspodig prediced oupu occurs 95% of he ime. Table 5- Records accessed by raiig ad es daa wih 95% cofidece ierval Exchage Rae All Trai Tes R Ne - R Avg. Abs Max. Abs RMS Accuracy (20%) CI (95%) Records The mohwise acual exchage rae of 2011 is kow o us. [Table-6] shows he acual ad prediced exchage rae of 2011 mohwise usig Neural Nework Model usig NeuralWorks Predic [Fig-11] is a NeuralWorks Predic model screesho geeraed whe he raiig daa is complee. I shows ha six ipus, five hidde uis ad oe oupu ui is ake ad he exchage rae of 2011 is prediced. [Fig-12] showig compariso bewee acual exchage rae of 2011 (show by blue lie) ad prediced exchage rae of 2011 (show by red lie) usig NeuralWorks Predic of Neural Nework Model. The x-axis of he graph shows ime period i mohs ad he y-axis of he graph shows exchage rae values. Table 6- Compares he acual exchage rae of 2011 wih prediced exchage rae of 2011 Time Period Acual Exchage Rae Prediced Exchage Rae Ja Feb Mar Apr May Ju Jul Aug Sep Oc Nov Dec Fig. 11- shows ha i he Neural Model 6 ipus are ake, wih 5 hidde uis ad oe oupu ui exchage rae is prediced Fig. 12- Graph showig compariso bewee acual exchage rae of 2011 wih prediced exchage rae of 2011 usig NeuralWorks Predic Performace Compariso Afer fiig a ime series model, oe ca evaluae i wih forecas fi measures. Whe more ha oe forecasig echique seems rea- Bioifo Publicaios 64

9 Saigal S. ad Mehrora D. soable for a paricular applicaio, he he forecas accuracy measures ca also be used o discrimiae bewee compeig models. Oe ca subrac he forecas value from he observed value of he daa a ha ime poi ad obai a measure of error. To evaluae he amou of his forecas error, he researcher may employ he mea error or he mea absolue error. Le A is he acual daa ad F represes he forecas ad deoes he umber of forecass made. The he mea error (ME) is merely he average error. ME ( 1 ) 1 (A F ) The mea absolue error (MAE) or mea absolue deviaio (MAD) is calculaed by akig he absolue value of he differece bewee he esimaed forecas ad he acual value a he same ime so ha he egaive values do o cacel he posiive values. The average of hese absolue values is ake o obai he mea absolue error. ( 1 MAE ) (A F ) 1 The mea squared error (MSE) measure he variabiliy i forecas errors. Obviously, he variabiliy i forecas errors mus be small. ME, MAE ad MSE are all scale depede measures of forecas accuracy, ha is, heir values are expressed i erms of he origial uis of measureme. MSE ( 1 ) 1 The roo mea square error (RMSE) measures he average magiude of he error. I i, he differece bewee forecas ad correspodig observed values are each squared ad he averaged over he sample. Fially, he square roo of he average is ake. Sice he errors are squared before hey are averaged, he RMSE gives a relaively high weigh o large errors. This meas he RMSE is mos useful whe large errors are paricularly udesirable. The perceage error (MPE) is he proporio of error a a paricular poi of ime i he series. The average perceage error i he eire series is a geeral measure of fi useful i comparig he fis of differe models. This measure adds up all of he perceage errors a each ime poi ad divides hem by he umber of ime pois. (A F ) ( 100% MPE ) A Because he posiive ad egaive errors may ed o cacel hemselves, MPE saisic is ofe replaced by he mea absolue perceage error (MAPE). MAPE is he more objecive saisic idicaor because he measure is i relaive perceage ad will o be affeced by he ui of he forecasig series. The closer MAPE approaches zero, he beer he forecasig resuls The predicio errors are calculaed usig all meioed performace measures i.e. ME, MAE, MSE, RMSE, MPE ad MAPE for all he four models amely Muliple Liear Regressio i Excel, Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka, Vecor Auoregressive Model i R ad Neural Nework Model usig NeuralWorks Predic ad show i [Table-7]. 2 (A F ) RMSE SQRT(MSE) 1 MAPE ( 100% ) 1 (A F ) A Table 7- Performace of all Models are compared hrough ME, MAE, MSE, RMSE, MPE ad MAPE MODELS ME MAE MSE RMSE MPE MAPE Muliple Liear Regressio i Excel % 7.27% Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka % 3.41% Vecor Auoregressive Model i R % 5.04% Neural Nework Model usig Neural- Works Predic % 5.94% Which Performace Measure Is Bes? Followig observaios are made from he performace measures: I is usually bes o repor he roo mea squared error (RMSE) raher ha mea squared error (MSE), because he RMSE is measured i he same uis as he daa, raher ha i squared uis, ad is represeaive of he size of a "ypical" error. The mea absolue error (MAE) is also measured i he same uis as he origial daa, ad is usually similar i magiude o, bu slighly smaller ha, he roo mea squared error. The mahemaically challeged usually fid his a easier saisic o udersad ha he RMSE. The mea absolue perceage error (MAPE) is also ofe useful for purposes of reporig, because i is expressed i geeric perceage erms which will make some kid of sese eve o someoe who has o idea wha cosiues a "big" error i erms of dollars spe or widges sold. The MAPE ca oly be compued wih respec o daa ha are guaraeed o be sricly posiive. The mea error (ME) ad mea perceage error (MPE) ha are repored i some saisical procedures are siged measures of error which idicae wheher he forecass are biased--i.e., wheher hey ed o be disproporioaely posiive or egaive. Bias is ormally cosidered a bad hig, bu i is o he boom lie. Bias is oe compoe of he mea squared error--i fac mea squared error equals he variace of he errors plus he square of he mea error. Tha is: MSE = VAR (E) + (ME) ^ 2. The roo mea squared error is more sesiive ha oher measures o he occasioal large error: he squarig process gives disproporioae weigh o very large errors. If a occasioal large error is o a problem i decisio siuaio, he he MAE or MAPE may be a more releva crierio. I may cases hese saisics will vary i uiso--he model ha is bes o oe of hem will also be beer o he ohers--bu his may o be he case whe he error disribuio has "ouliers." If oe model's RMSE is 30% lower ha aoher's, ha is probably very sigifica. If i is 10% lower, ha is probably somewha sigifica. If i is oly 2% beer, ha is probably o sigifica. These disicios are especially impora whe we are radig off model complexiy agais he error measures: i is probably o worh addig aoher idepede variable o a regressio model o decrease he RMSE by oly a few more perce. The cofidece iervals for oe-sep-ahead forecass are based almos eirely o RMSE, he cofidece iervals for he loger-horizo forecass ha ca be produced by imeseries models deped heavily o he uderlyig modelig assumpios, paricularly assumpios abou he variabiliy of he red. The cofidece iervals for some models wide relaive- Bioifo Publicaios 65

10 Saigal S. ad Mehrora D. ly slowly as he forecas horizo is legheed (e.g., simple expoeial smoohig models wih small values of "alpha", simple movig averages, seasoal radom walk models, ad liear red models). The rae a which he cofidece iervals wide is o a reliable guide o model qualiy: wha is impora is he model should be makig he correc assumpios abou how ucerai he fuure is. I is very impora ha he model should pass he various residual diagosic ess ad "eyeball" ess i order for he cofidece iervals for loger-horizo forecass o be ake seriously. So he observaio saes ha mosly he performace of compared models depeds o RMSE, bu someimes MAE or MAPE. Also, If wo models are geerally similar i erms of heir error saisics ad oher diagosics, you should prefer he oe ha is simpler ad/or easier o udersad. The simpler model is likely o be closer o he ruh, ad i will usually be more easily acceped by ohers. Forecass from he four models are compared ogeher wih he acual usage. I is observed ha forecass from he Dedicaed Time Series Model usig Weka are closer o he acual demads ha he forecass from oher hree models. Coclusio I owadays echology, fiacial isiuios are able o produce huge daases ha build a foudaio for approachig hese eormously complex ad dyamic problems wih daa miig ools. Wih vas daa ad lieraure i his field he poeial sigifica beefis of solvig hese problems moivaed exesive research for years. The mosly used daa miig applicaio applied o he field of fiace is he modelig (predicig or forecasig) of fiacial daa. Almos every saisical ad mahemaical compuaioal mehod has bee explored ad used for fiacial modelig. I his paper, four models amely Muliple Liear Regressio i Excel, Muliple Liear Regressio of Dedicaed Time Series Aalysis i Weka, Vecor Auoregressive Model i R ad Neural Nework Model usig NeuralWorks Predic are aalyzed usig real ime series daa. The daily exchages of currecy bewee he U.S. ad oher aios have a major effec o ieraioal rade. There are may advaages ad disadvaages associaed wih srog ad weak U.S dollars. This sudy will be helpful o learers ad expers alike as hey choose he bes approach o solvig basic daamiig problems. This will help i reducig he lead ime for geig he bes predicio possible. Refereces [1] Haga M.T. ad Behr S.M. (1987) IEEE Tras. Power Sys., PWRS-2(3). [2] Moghram Rahma S. (1989) IEEE Tras. Power Sys., 4, [3] Taylor J.W. ad McSharry P.E. (2007) IEEE Tras. Power Sys., 22(4). [4] Abu-El-Magd M.A. ad Fidlay R.D. (2003) Elecrical ad Compuer Egieerig, IEEE CCECE Caada Cof., 3, [5] Perwej Y. ad Perwej A. (2012) Ieraioal Joural of Compuer Sciece, Egieerig ad Applicaios, 2(2). [6] Kosorus H., H oigl J. ad K ug J. (2011) 22d Ieraioal Workshop o Daabase ad Exper Sysems Applicaios. [7] Brockwell P.J. ad Davis R.A (2006) Time Series: Theory ad Mehods, 2d ed., Spriger, New York. [8] Cowperwai P.S.P. ad Mecalfe A.V. (2009) Iroducory Time Series wih R, 1s ed., Spriger, New York. [9] Hall M., Frak E., Holmes G., Pfahriger B., Reuema P. ad Wie I.H. (2009) SIGKDD Exploraios, 11(1). [10] Shumway R.H. ad Soffer D.S. (2006) Time Series Aalysis ad Is Applicaios - Wih R Examples, 2d ed., Spriger, New York. [11] Suharoo ad Subaar, Gurio S. (2005) Jural Tekik Idusri, 7(1), [12] Povielli R.J. ad Feg X. (2003) IEEE Trasacios o Kowledge ad Daa Egieerig, 15(2), [13] Alves da Silva A.P. ad Mouli L.S. (2000) IEEE Trasacios o Power Sysems, 15(4), [14] Medeiros M.C., Veiga A. ad Pedreira C.E. (2001) IEEE Trasacios o Neural Neworks, 12(4), [15] Hipper H.S., Pedreira C.E. ad Souza R.C. (2001) IEEE Tras. Power Sys., 16(1), [16] Cărbureau M. (2011) Ecoomic Scieces Series, LXIII(3), [17] Weiss S.M. ad Idurkhya N. (1999) Predicive Daa Miig: A Pracical Guide, 1s ed., Morga Kauffma Publishers. [18] Ha J. ad Kamber M. (2006) Daa Miig: Coceps ad Techiques 2d ed., Elsevier Ic.: Morga Kaufma Publishers. The models have show favorable forecasig accuracy wih Liear Regressio of Dedicaed Time Series Aalysis i Weka ouperformig he oher hree models. Bioifo Publicaios 66