IWSSIP 1-17th Internatonal Conference on Systems, Sgnals and Image Processng Tme Delayed Independent Component Analyss for Data Qualty Montorng José Márco Faer Sgnal Processng Laboratory, COE/Pol Federal Unversty of Ro de Janero Ro de Janero, RJ, BRAZIL faer@lps.ufrj.br José Manoel de Seas Sgnal Processng Laboratory, COE/Pol Federal Unversty of Ro de Janero Ro de Janero, RJ, BRAZIL seas@lps.ufrj.br Abstract In the nformaton era, databases n companes and research centers are gettng larger, whch makes the qualty of data a key ssue. In ths paper, tme delayed ndependent component analyss s used for data qualty montorng of electrc load tme seres. The ndependent component analyss () was appled n the preprocessng phase, whch ncreased the data qualty system performance. The etracton of sgnal sources reduced the forecast error, revealed relevant nformaton and narrowed the valdaton corrdor wdth. Keywords- Data Qualty; Tme Seres; Independent Component Analyss; Neural Networks. I. INTRODUCTION In present days, the global development s due, n large part, to wde data dssemnaton, especally due to Internet. In fact, wth the enormous data volume ncrease, the attenton has turned to the ablty to absorb nformaton and respond approprately [1. Thus, data qualty ssues have become a key factor to the transformaton from data to relevant nformaton. Data qualty s the level of correctness, completeness, consstency, nterpretablty, securty, aggregated nformaton and other data characterstcs [. These data qualty dmensons must be specfed and montored n accordance to user specfcatons. In the electrc sector, data qualty study s mportant due the recent ncrease on electrc load demand, especally n emergng countres, such as Brazl. The demand ncreases has resulted companes fuson (data ntegraton from dfferent systems), decsons to avod blackout and other decsons to manager the electrc system. In ths work, a data qualty montorng system s developed to analyze electrc load tme seres wth respect to the peak energy. The methodology uses adjacent seres wth respect to the peak hour, the daly peak seres and temperature seres. These data contan fundamental patterns that mpact sgnfcantly a number of decson takng processes and they should not be corrupted. Thus, a data qualty montorng may dentfy problems and, eventually, correct mstakes and enrch the nformaton, n accordance to user specfcatons. To montor key data qualty dmensons n ths tme seres, a valdaton corrdor s proposed for evaluatng an ncomng sample ncluded n the database and correct for t, f necessary/requested. Here, the corrdor s bult dynamcally usng Independent Component Analyss [3, amng at dentfyng more structured data n the ncomng tme seres. Ths more structured nformaton may make the data qualty montorng system more effcent. Over the estmated ndependent sources, sgnal preprocessng s appled for removng seasonalty, cycles and tendency [. Neural network modelng [5 estmates the target applcaton from the resultng resdual sgnal. The valdaton corrdor center for data qualty evaluaton s the forecasted value for a gven sample and ts lmt s proportonal to the estmaton error. Ths method allows the correcton for outlers and mssng data [6. The Independent Component Analyss () s a statstcal technque to fnd hdden factors n observed sgnals. defnes a model generator from observed data, whch are assumed to be mtures of unknown ndependent varables (sources). has been used as an aulary tool n autoregressve processes for tme seres forecast [7. The paper s organzed as t follows. In the net secton, a more detaled eplanaton of the data qualty montorng system s made. Secton III presents the methodology used n the case study n data qualty montorng for electrc load tme seres, whch s conducted n Secton IV. Conclusons are derved n Secton V. II. TIME SERIES DATA QUALITY MONITORING The am of the data qualty montorng system s to evaluate the qualty of a new sample, whch s to be ncorporated nto the database, and correct for the ncomng sample, f necessary. The system s bult as a control system [6, where past samples are used to buld the tme seres model and produce a valdaton corrdor, wthn whch the ncomng sample should stay (see Fg. 1).
IWSSIP 1-17th Internatonal Conference on Systems, Sgnals and Image Processng Fg. 1. The valdaton corrdor concept The valdaton corrdor s defned dynamcally, at moment n, by the mean absolute error (µ error ) between estmated ( et ) and real ( ) sample values, adjusted by a constant to defne the mssng/fal probablty: n 1 1 est Corrdor( n). k. n 1. k.µ error The k parameter allows nclude the user role and determne a compromse between the contet and the user specfcatons. Typcally, k s adjusted to detect theoretcal presence of soft outlers n tranng set (one outler for 15 samples). The tme seres model (corrdor centers) s derved from preprocessed data estmatons (see Fg. ). The preprocessng stage etracts any modeled component [. The presence of seasonalty and cycles are analyzed n frequency-doman by the Fast Fourer Transform [8. To remove the dentfed components from data, t s verfed ther sgnfcance level. Net, the presence of heteroscedastcty s analyzed wth Goldfeld-Quandt test [9. In case of heteroscedastcty, an approprate acton, such as the applcaton of the logarthmc functon, should be consdered. Wth homoscedastc seres, the tendency s analyzed. For ths, a combnaton of the Dckey-Fuller (ADF) [1 and Phllps-Perron [11 tests s used. Such test combnaton checks for unt roots n tme seres. In case of fndng unt roots, the trend s stochastc and the frst dfference s appled m tmes (where m s the ntegraton order of the process). If the test does not detect unt roots, the trend s determnstc, and t s removed from a polynomal fttng. The estmator block s performed by a lnear model or a neural network. Neural estmators for tme seres forecastng have been wdely used [1. It has been au Seres Values Input sample Preprocessng Estmator (1) Output Fgure. Basc block dagram of the montorng system t-1 t-n y(t)b *(t) y Tme shown that neural systems are most effectve when nput data are preprocessed. In recent works, neural estmators have been fed from a resdue seres, whch s obtaned at the output of the preprocessng phase [[6. Ths resdual nformaton s the result of subtractng from the ncomng raw data the modeled tme seres components (tendency, seasonalty, cycles, etc), obtaned from the preprocessng block. Therefore, the estmator ams at forecastng what s unknown from data. In ths work, we proposed to nclude an block to the data preprocessng chan (see Fg. 3). The fnds the ndependent sources (y) derved from the observed sgnals (). In tme seres, fnds ndependent sgnals analyzng delayed correlatons to estmate the demng matr B [3: y B () If an ndependent component s assgned to nose, through correlatng components to the sources obtaned for modelng, deflaton may be appled. Net, after the preprocessng step ( block - Fg. 3), accordng to what was descrbed before, the system evaluates the correlaton ( block - Fg. 3) of delayed versons of the estmated (ndependent preprocessed sources) sources, n order to determne the relevant samples whch wll be fed nto estmator nput nodes. If correlatons are above a threshold, the qualfed samples feed the estmator ( block Fg. 3). The estmator desgn s based on parsmonous crteron [13. From smple models, the complety s gradually ncreased and evaluated. In the non-lnear case, nonrecurrent (mult-layer perceptron - MLP [5) and recurrent (Elman [1) networks are used. Thus, from a sngle hdden neuron, the number of hdden neurons s ncreased untl hypothess test rejecton. Besdes, early stop of network tranng s appled to avod over tranng [5. In the lnear case, the estmator s an auto-regressve movng average (ARMA [1) model. The forecasted tme seres s reconstructed from the modelng part of the sources (block - Fg. 3), resultng n the estmated sources (y est ). The process s reversed ( -1 block) and the forecasted values ( est ) are obtaned. From est, the corrdor s fnally constructed for the orgnal data space. For data qualty assessment, the data samples should reman wthn the corrdor lmts. Thus, the am s to obtan a corrdor as narrow as possble, for detectng errors and allowng ther correcton wth good accuracy, f necessary / requested. III. METHODOLOGY The data qualty montorng system was analyzed n the Fgure 3. Data Qualty Montorng System structure. y est -1 est
IWSSIP 1-17th Internatonal Conference on Systems, Sgnals and Image Processng framework of the electrc load tme seres from a European energy suppler (East-Slovaka Power Dstrbuton Company), whch was used n a competton n 1 by the European Network on Intellgent Technologes for Smart Adaptve Systems [15. Ths database comprses electrc load seres, n MW, collected every thrty mnutes from 1 January 1997 to 31 January 1999 and the daly temperature averaged n C, coverng the same tme perod. In the competton held n 1, the competton task was to develop models to forecast daly peak load for January 1999. Here, the 1997 perod was used for tme seres tranng, the year 1998 for tranng valdaton and January/1999 for testng (generalzaton). Besdes daly load peak, groups of seres near the mean peak tme (:) were also consdered. Thus, seven adjacent seres between 18:3 and 1:3 were used for seres modelng. The temperature was used as an aulary seres. The valdaton corrdor was estmated wth the k constant defned by fal/mssng probablty on tranng / valdaton set. The constant k was determned assumng one outler for 15 samples. For fndng ndependent sources (y) see (1) -, a specfc method for tme seres was employed. The method fnds a demng matr (B) by dagonalzaton of the Delayed-Auto-Cross-Covarance Matr: T { ( n) ( n τ } C E ) (3) τ Where, n s the tme sequence, τ s the tme delayed and the mng seres. We use the Second Order Blnd Identfcaton algorthm wth Robust Orthogonalzaton (SOBI-RO [16[17) for fnd ndependent components. Ths method dagonalzes a group of Delayed-Auto-Cross-Covarance Matres. The seres are presented to the Block n parallel. Target seres could also be presented delayed to Block - named here tme-delayed (TD 1 ) -, ncreasng the number of eplcatve seres. Analyss and forecasts are performed n the space and transformed back to the orgnal space, wth reversed. In the orgnal space, just the target seres no delayed s consdered. The neural network nput layer was constructed from sources consderng samples delayed tme-delayed neural networks (TDNN). We used a correlaton test, typcally.5 for threshold, to fnd relevant nput delays. For hdden layer, the hypothess testng of a model wth n neurons aganst the hypothess of n +1 neurons was computed for 5% sgnfcance level. The results were analyzed usng performance ndees evaluated over the test set: the normalzed mean square errors (NMSE). The nde NMSE 1 normalzes the MSE wth respect to the varance of the estmated seres - see () -, and NMSE uses the best random walk estmator as the normalzaton factor - see (5). 1 Delays for TD not confuse wth delays for Delayed-Auto-Cross- Covarance Matr. MSE NMSE1 σ E NMSE E E[ ( est ) ( ) µ [( est ) [( ) t 1 The corrdor center wth NMSE 1 smaller than 1 s better than a corrdor constructed wth the mean of the process (µ ). The same occurs when the NMSE s smaller than 1 and the corrdor s constructed usng the sample mmedately before ( t-1 ). The results wth and wthout were also compared usng three others performance ndees. The R ndcator s the rate between correlatons from orgnal seres and forecasted seres delayed one sample (Lag 1 ) and no delayed (Lag ) see (6). The MAC ndcator s the mean absolute corrdor wdth, gven n MW, and the MAPE s the mean absolute percentage error between forecasted and real sample - see (7) and (8). Lag () (5) t R (6) Lagt 1 MAC MAPE N 1 est. k. N (7) N est 1 N (8) 1 Here, and est are observed and forecasted samples, at tme nstant, respectvely, N s the number of samples, and k s the corrdor adjustment constant obtaned durng the tran phase. IV. ANALISYS AND RESULTS The best results wth are obtaned trough second order statstcs wth robust orthogonalzaton (SOBI- RO), dagonalzng the frst 5 delayed-crosscovarance matres. Better performance s acheved usng TD wth delays for target seres and wthout delays for temperature or other eplcatve seres. The preprocessng etracted frequency components above 6 standard devatons wth respect to the mean ampltude value. After the removal of cycles and seasonalty, the hypothess tests dd not detect stochastc tendency, to 5% sgnfcance level. Thus, a lnear trend was removed. Three standard devatons were used to defne the network nput samples from the correlaton functon. The data qualty montorng systems used both lnear (ARMA) and non-lnear estmators (MLP) to modelng the sources. For nonlnear case, the hypothess test defned mamum hdden neurons, at a 5% sgnfcance level. The sources #1 to #3 are modelng wth non-lnear estmators and the others used lnear models. Table I shows NMSE 1 and NMSE ndees computed from the testng seres wth. It s observed that only NMSE for Seres #7 s around 1 and the others are well below. Then, the data qualty montorng system performance ncreases when compared to mean and to the best random walk estmator.
3 5 1 5 1 5-5 -1-1 5 8 5 8 7 5 7 6 5 6 5 5 5 5 3 5 8 5 8 7 5 7 6 5 6 5 5 5 5 3 5 8 5 8 7 5 7 6 5 6 5 5 5 5 3 5 85 8 75 7 65 6 55 5 5 35 1 3 5 6 7 8 7 6 5 3 1 3 1-1 - -3 - -5-8 -1-1 -1-1 6-1 8 - - -8-1 -1-1 8 6 - - IWSSIP 1-17th Internatonal Conference on Systems, Sgnals and Image Processng TABLE II. TABLE I. NMSE 1 AND NMSE Seres NMSE 1 NMSE Seres #1 (18:3),,51 Seres # (19h),,6 Seres #3 (19:3),37,39 Seres # (h),39,8 Seres #5 (:3),,9 Seres #6 (1h),7,51 Seres #7 (1:3),5 1,1 Seres #8 (Peak),6,6 Wthout, for all seres, the best model was an ARMA wth mamum delays and no feedback, becomng a Movng Average (MA) model. The preprocessng was equal for both wth and wthout. Table II shows the performance for both usng or not. The best results for each case are epressed as boldfaced values. In general, performed better. For all seres modeled wth, R s above 1. Wthout, R ndcator s below 1 for seres #1, #6 and #7, ndcatng worse performance. Also, n general, the corrdor s narrower and the MAPE s smaller, when s used n the preprocessng chan. Fgure shows fve sources etracted from, ndcatng that temperature (frst seres and frst source) s one of the estmated sources. The second source suggests a semester dependency. The thrd source s from annual varaton and the others suggest a trmester dependency. The others sources dd not show an easy nterpretaton n the contet of the applcaton. Ths ablty to dentfyng orgnal and better structured nformaton proved here to facltate the work of the estmaton block. RESULTS FOR MAPE, MAC AND R Wth (SOBI-RO) Wthout Seres MAPE MAC R MAPE MAC R (%) (MW) (%) (MW) Seres #1 (18:3) 3.1 158 1,87.6 1,9 Seres # (19h).5 15 1,55.6 6 1,1 Seres #3 (19:3) 3.1 156 1,6.9 18 1, Seres # (h). 137 1,6. 151 1,8 Seres #5 (:3). 1 1,3. 136 1,6 Seres #6 (1h) 1.9 136 1,18 1.9 15,97 Seres #7 (1:3). 17 1, 3.1 158,9 Seres #8 (Peak). 153 1,85. 19 1,1 Fgure 6 shows the real values, the valdaton corrdor centers and the corrdor wdth for the peak seres, when s used n the seres preprocessng chan and MLP and ARMA are used to modelng the sources. Electrc Load (MW) 9 85 8 75 7 65 ) 6 5 1 15 5 3 35 Tme (days) Fgure 6. Real and forecasted peak seres and the valdaton corrdor for Jan/1999. V. CONCLUSIONS -- Real - Forecasted The data qualty montorng system proposed here uses a valdaton corrdor to evaluate ncomng samples of a target tme seres. The corrdor s bult around a forecasted value that s obtaned from preprocessed data. The dynamc corrdor adapts to the seres statstcal varatons and the system alerts the user when the ncomng sample s out of the corrdor lmts. In case a correcton s requred from the epert user or a mssng value s detected, forecasted sample may be used. In the proposed system, neural networks and lnear models were appled n combnaton wth Tme Delayed Independent Component Analyss and a pre-processng stage. The parsmonous models presented better results (lnear or non-lnear models wth few neurons). The mpact of was analyzed for a partcular case of electrc load tme seres. The algorthm used second-order statstcs wth tme sequence analyses (SOBI-RO) to etract the ndependent sources. The neural (recurrent and non-recurrent) or lnear model operated over preprocessed ndependent sources (analyzng and removng heteroscedastcty, trends, cycles and seasonalty). The montorng system proposed, usng, reduced the valdaton corrdors and mproved the forecast performance, whch have postve mpact on data qualty dmensons such as correctness, completeness, outdatng, and nterpretablty. ACKNOWLEDGMENT We are thankful to CNPq and FAPERJ (Brazl) for ther support to ths work. Fgure. On the left, the seres for temperature (top) and seres from #1to #. On the rght, ndependent sources obtaned through block. REFERENCES [1 Eckerson, W. W. (). Data Qualty and the Botton Lne, Report, The Data Warehousng Insttute. [ Chrsman, N. R. (1983). The Role of Qualty Informaton n the Long-Term Functonng of a GIS. In: Proceedngs of the AUTOCART6, v., pp. 33-31. [3 Hyvarnen, A., Karhunen, J. e Oja, E., (1). Independent Component Analyss, ISBN -71-5-X John Wley & Sons, Inc. [ DANTAS, A. C. H. ; DINIZ, F. C. da C. B. ; FERREIRA, T. N. ; SEIXAS, J. M. de. Statstcal and Sgnal Processng Based
IWSSIP 1-17th Internatonal Conference on Systems, Sgnals and Image Processng System for Data Qualty Management. In: IV Internatonal Conference on Data Mnng Includng Buldng Applcatons for CRM & Compettve Intellgence, 3, Ro de Janero. Proceedngs of the IV Internatonal Conference on Data Mnng Includng Buldng Applcatons for CRM & Compettve Intellgence, 3. p. 1-1. [5 Haykn, Smon. (8) Neural Networks and Learnng Machnes, da. Edton ISBN 1317139, Prentce Hall. [6 DANTAS, A. C. H. ; SEIXAS, J. M. de. Neural Networks for Data Qualty Montorng of Tme Seres. In: 9th Internatonal Conference on Enterprse Informaton Systems, 7, Funchal, Madera. Proceedngs of the 9th Internatonal Conference on Enterprse Informaton Systems, 7. p. 11-15. [7 Kvluoto, K., Oja, E. (1998). Independent Component Analyss for Parallel Fnancal Tme Seres. In Proc. Int. Conf. on Neural Informaton Processng (ICONIP 98), v., pp. 895-898, Tokyo, Japan. [8 Brgham, E.O. (), The Fast Fourer Transform, New York: Prentce-Hall. [9 S.M. Goldfeld and R.E. Quandt (1965), "Some Tests for Homoscedastcty". Journal of the Amercan Statstcal Assocaton 6, 539 57. [1 Dckey, D. A. and Fuller, W. A. (1979). Dstrbutons of the estmators for autoregressve tme seres wth a unt root. Journal of the Amercan Statstcal Assocaton, v. 75, pp. 7-31. [11 Phllps, P. C. B. (1987). Tme seres regresson wth a unt root. Econometrca, v. 55, n., pp. 77-31. [1 George Bo, Gwlym M. Jenkns, and Gregory C. Rensel. Tme Seres Analyss: Forecastng and Control, thrd edton. Prentce- Hall, 199. [13 Mederos, M. C., Teraasvrta, T., Rech, G. (6). Buldng Neural Network Tme Seres Models: A Statstcal Approach, Journal of Forecastng, v. 5, n. 1, pp. 9-75. [1 Elman, J. L. (199). Fndng structure n tme. Cogntve Scence, v. 1, pp. 179-11. [15 EUNITE, European Network on Intellgent Tech-nologes for Smart Adaptve Systems (1), http://neuron.tuke.sk/competton. [16 Belouchran, A., Abed-Meram, K., Cardoso, J., Moulnes, E. (1997). A Blnd Source Separaton Technque Usng Second Order Statstcs. IEEE Transactons on Sgnal Processng, 5 ():3-. [17 Belouchran, A., Cchock, A. (1). Robust whtenng procedure n blnd source separaton contet, Electroncs Letters, Vol. 36, No., pp. 5-53.