Kernelized Radial Basis Probabilistic Neural Network for Classification of River Water Quality

Transcription

1 Kernelzed Radal Bass Probablstc eural etwor for Classfcaton of Rver Water Qualty Lm Eng A a, Zarta Zanuddn b a Insttute of Engneerng Mathematc Unverst Malaysa Perls, 0000 Kangar, Perls Tel : E-mal : ealm@unmap.edu.my b School of Mathematcal Scence Unverst Sans Malaysa, Mnden, Penang E-mal : zarta@cs.usm.my ABSTRACT Radal Bass Probablstc eural etwor (RBP) demonstrates broader and much more generalzed capabltes whch have been successfully appled to dfferent felds. In ths paper, the RBP s extended by calculatng the Eucldean dstance of each data pont based on a ernel-nduced dstance nstead of the conventonal sum-of squares dstance. The ernel functon s a generalzaton of the dstance metrc that measures the dstance between two data ponts as the data ponts are mapped nto a hgh dmensonal space. Through comparng the four constructed classfcaton models wth Kernelzed RBP, Radal Bass Functon networs, RBP and Bac-Propagaton networs as ntended, results showed that, model classfcaton on Rver water qualty of Langat rver n Selangor, Malaysa by Kernelzed RBP exhbted excellent performance n ths regard. Keywords Kernel functon, Radal Bass Probablstc eural etwor, Water Qualty.0 ITRODUCTIO In recent years, water qualty management and assessment model s a very popular and mportant research topc. The socalled management and assessment means to use the related water nformaton to evaluate the level of water qualty of a rver. However, the water qualty status of a rver s usually what the publc concerns about and s usually the most mportant ey for the healthy water supply to household (Gnors et al, 007). Therefore, f the water qualty of a rver can be predcted much earler, the hazardous causes to the publc can be greatly reduced. In the past lterature, some researchers use conventonal statstcal model for model constructon, for example, the Dscrmnant Analyss of (Altman, 968), the Probt of (Ohlson, 980). However, n recent years, some researchers found that the model constructed by data mnng technque has better predcton accuracy than that of the conventonal statstcal model; some of them are, for example, the Bac- Propagaton etwors model of (Odom, 990) and the decson tree classfcaton model of (Breman, 996). Another recent approach done by (Zarta et al, 004) usng bacpropagaton method n classfcaton of rver water qualty. The approach provded a good result n classfcaton, but, t taes a long tmes for tranng the networ. In ths paper, Radal Bass Probablstc eural etwors (RBP) whch s nown to have good generalzaton propertes and s traned faster than the bacpropagaton networ (Ganchev, 003) s adopted for the constructon of water qualty level early warnng classfcaton predcton model. Our proposed method, used ernel-functon to ncrease the separablty of data by worng n a hgh dmensonal space. Thus, the proposed method s characterzed by hgher classfcaton accuracy than the orgnal RBP. The ernelbased classfcaton n the feature space not only preserves the nherent structure of groups n the nput space, but also smplfes the assocated structure of the data (Muller, 00). Snce Grolam frst developed the ernel -means clusterng algorthm for unsupervsed classfcaton (Grolam, 00), several studes have demonstrated the superorty of ernel classfcaton algorthms over other approaches to classfcaton (Zhang, 00; Wu & Xe, 003). In ths paper, we evaluate the performance of our proposed method, ernelzed RBP, wth a comparson to three wellnown classfcaton methods: the Bac-Propagaton (BP), Radal Bass Functon etwor (RBF), and RBP; meanwhle, comparson and analyss on the classfcaton power s done to these three methods. These methods performance are compared usng rver water qualty data sets from Langat Rver of Selangor, Malaysa.

2 .0 RADIAL BASIS PROBABILISTIC EURAL ETWORKS Radal Bass Probablstc eural etwor (RBP) archtecture s based on based on Bayesan decson theory and nonparametrc technque to estmate Probablty Densty Functon (PDF). The form of PDF s a Gaussan dstrbuton. (Specht, 990) had proposed ths functon (Yeh, 998): If X = X, then f ( X ) = f ( X ) = 0 X X ( ) σ σ π = f ( X ) = exp () m m Snce RBP s applcable to general classfcaton problem, and assume that the egenvector to be classfed must belong to one of the nown classfcatons, then the absolute probablstc value of each classfcaton s not mportant and only relatve value needs to be consdered, hence, n equaton (), m m ( π ) σ Can be neglected and equaton () can be smplfed as X X f ( X ) = exp () σ = In equaton (), σ s the smoothng parameter of RBP. After networ tranng s completed, the predcton accuracy can be enhanced through the adustment of the smoothng parameter σ, that s, the larger the value, the smoother the approachng functon. If the smoothng parameter σ s napproprately selected, t wll lead to an excessve or nsuffcent neural unts n the networ desgn, and over fttng or napproprate fttng wll be the result n the functon approachng attempt; fnally, the predcton power wll be reduced. At ths moment, RBP wll depend fully on the nonclassfed sample whch s closest to the classfed sample to decde ts classfcaton. When smoothng parameter σ approaches nfnty, f ( X ) = At ths moment, RBP s close to blnd classfcaton. Usually, the researchers need to try dfferent σ n certan range to obtan one that can reach the optmum accuracy. (Specht, 99) had proposed a method that can be used to adust smoothng parameter σ,that s, assgn each nput neural unt a sngle σ ; durng the test stage, σ that have the optmum classfcaton result are taen through the fne adustment of each σ. RBP s a three-layer feedforward neural networ (as n Fgure ). The frst layer s the nput layer and the number of neural unt s the number of ndependent varable and receves the nput data; the second hdden layer n the mddle s Pattern Layer, whch stores each tranng data; the data sent out by Pattern Layer wll pass through the neural unt of the thrd layer Summaton Layer to correspond to each possble category, n ths layer, the calculaton of equaton (3) wll be performed. The fourth layer s Compettve Layer; the compettve transfer functon of ths layer wll pc up from the output of the last layer the maxmum value from these probabltes and generate the output value. If the output value s, t means t s the category you want; but f the output value s 0, t means t s other unwanted category. Let d = X X Be the square of the Eucldean dstance of two ponts X and X n the sample space, and equaton () can be re-wrtten as d f ( X ) = exp (3) σ = In equaton (3), when smoothng parameter σ approaches zero, Fgure : RBP archtecture

3 3.0 KERELIZED RADIAL BASIS PROBABILISTIC EURAL ETWORKS 3. Kernel-Based Approach Gven an unlabeled data set X { x x } =, K, n the d- n dmensonal space R d d, let Φ : R ᆴ H be a non-lnear mappng functon from ths nput space to a hgh dmensonal feature space H. By applyng the non-lnear mappng functon Φ, the dot product x x n the nput space. The ey noton n ernelbased learnng s that the mappng functon Φ need not be explctly specfed. The dot product Φ( x ) Φ ( x ) n the hgh dmensonal feature space can be calculated through the ernel functon K(x, x ) n the nput space R d (Scholopf & Smola, 00). K ( x, x ) = Φ( x ) Φ ( x ) (4) Three commonly used ernel functons (Scholopf & Smola, 00) are the polynomal ernel functon, K( x, x ) = ( x x + c) d (5) where c ᄈ 0, d ᅫ ; the Gaussan ernel functon, x x K( x, x ) = exp σ where σ > 0; and the sgmodal ernel functon, where κ > 0 and γ < Formulaton (6) K( x, x ) = tanh( κ ( x, x ) + γ ) (7) Gven a data pont x ΣR d ( n) and a non-lnear mappng d Φ : R ᆴ H, the Probablty Densty Functon at data pont x s defned as Φ( x ) Φ( x ) f ( x) = exp (8) σ = where Φ( x ) Φ( x ) s the square of dstance between Φ( x ) and Φ( x ). Thus a hgher value of f ( x) ndcates that x has more data ponts x near to t n the feature space. The dstance n feature space s calculated through the ernel n the nput space as follows: ( ) ( ) Φ( x ) Φ ( x ) = Φ( x ) Φ( x ) Φ( x ) Φ( x ) = Φ( x ) Φ ( x ) Φ( x ) Φ ( x ) + Φ( x ) Φ ( x ) = K ( x, x ) K ( x, x ) + K( x, x ) (9) Therefore, Equaton (8) can be rewrtten as K ( x, x ) K ( x, x ) + K ( x, x ) f ( x) = exp σ = (0) The rest of the computaton procedure s smlar to that of the RBP method. 4.0 COSTRUCTIO OF CLASSIFICATIO PREDICTIO MODEL AD ROC CURVE AALYSIS A sequence of 33 rows of data conssts of Bochemcal Oxygen Demand (BOD), Chemcal Oxygen Demand (COD), Ammona-trate level (A), Suspended Substances level (SS), Dssolved Oxygen (DO), and Acdty (ph) that consttutes to the dataset. The data to ncluded n the database s collected by Department of Envronment of Malaysa and contans the records of Langat Rver water qualty level whch also reported n (Zarta et al, 004). All of data sets of the sx parameters have been Z-transformed for classfcaton purposes and the Gaussan ernel was used n tranng of Kernelzed RBP. The Z-transform s performed as a preprocessng step to mnmze the gap of mnmum and maxmum of data values. Thus, we dvded 33 rows of data nto two separate parts, 00 rows for tranng and 3 rows remanng for testng. So, the rato of tran to test s approxmately :. All tranng s done n a destop PC of Intel duo core GHz wth Gb RAM memory. In the Kernelzed RBP, MATLAB s used to self-wrte the program for the model constructon and we fxed value to the Smoothng Parameter σ to 0. accordng to (Sansone & Vento, 00). In the test phase, each classfer has been tested wth data sets mentoned above. Performance of dfferent classfers n the test phase can be observed n Table. The proposed classfer, whch referred as Kernelzed RBP, acheves the best result amongst all other methods. It can be concluded from Table that the performance of Kernelzed RBP s better than RBP, RBF and BP. In term of tme, Kernelzed

4 RBP s about 89 tmes faster than RBF and 99 tmes faster than GD. The classfcaton accuracy obtaned usng Kernelzed RBP s 6% hgher than RBP. It s also obvous that Kernelzed RBP consumed less tme n comparson to RBP, RBF and BP. RBP outperform RBF n classfcaton accuracy due to ts networ archtecture that mplemented Bayesan decson and nonparametrc technque whch mae t more applcable to general classfcaton problem, ths brng RBF has the worst performance among all models. Although the classfcaton accuracy of the well nown classfcaton method, BP, s hgh, but as we now, hgh processng tme of BP maes t undesrable for many on-lne recognton applcatons Kernelzed RBP RBP RBF GD Reference Lne Method AUC Kernelzed RBP RBP RBF GD 0.69 ROC curve The true postve rate n Fgure means the percentage of the number of class wth predcton result of 0 to the number of class wth real value of 0; false postve rate means the percentage of the number of class wth predcton result of to the number of class wth real value of. In ths paper, the mutual verfcaton of the data among the models s performed. The Kernelzed RBP s verfed as the best performed model and there are 3 rows of data n the test results. The result s drawn as Recever Operator Characterstc (ROC) curve as n Fgure. In the fgure, the farther the ROC curve above the reference lne, the larger the area under the ROC curve (AUC), whch also means the hgher the classfcaton predcton power of ths model (Bradley, 997). Table shows the mutual verfcaton results of the rver water qualty. Fgure shows that the area under the ROC curve of the Kernelzed RBP model of the rver water qualty data are larger than the other three models; from the observaton of Table, t seems that the specfcty of Kernelzed RBP model s less than other three models, but the AUC value are hgher than those of BP, RBP and RBF models; therefore, t can be concluded that Kernelzed RBP model has very good classfcaton predcton capablty. Table : Classfcaton Predcton accuracy of Langat rver water qualty. Method Classfcaton Accuracy (%) CPU Tme (s) Epoch Kernelzed RBP RBP RBF GD true postve rate false postve rate Fgure : The Mutual verfed ROC curve of test data of Langat rver water qualty 5.0 COCLUSIO In ths paper, a new RBP whch conssts of ernel-based approach has been proposed. ot le the conventonal RBP that only based on Eucldean dstance computaton, the Kernelzed RBP appled the ernel-nduced dstance computaton whch can mplctly map the nput data to a hgh dmensonal space n whch data classfcaton s easer. The networ s appled to rver water classfcaton problem and showed an ncreased n classfcaton accuracy n comparson wth other well-nown classfers. The metrcs for consderng performance of a classfer n ths wor were the classfcaton accuracy and processng tme of the classfer. In future wor, we plan to mprove the classfcaton rate by employng dfferent ernel functon such as wavelet functon. Table : Mutual Verfcaton wth AUC. REFERECES

5 Altman, E. I. (968). Fnancal ratos, dscrmnate analyss and the predcton of corporate banruptcy. Journal of Fnance, 3(4), Bradley, A. P. (997). The use of the area under ROC curve n the evaluaton of machne learnng algorthms. Pattern Recognton, 30 (7), Breman, L. (996). Baggng predctors. Machne Learnng, 4(), Zarta, Z., Sharfuddn, M. Z., Lm, E. A. & Hafzan, J. (004). Applcaton of neural networ n rver water qualty classfcaton. Ecologcal and Envronmental Modelng (ECOMOD 004), (), 4-3. Zhang, R. & Rudncy, A. I. (00). A large scale clusterng scheme for ernel -means. The 6 th Internatonal Conference on Pattern Recognton, Ganchev, T., Tasouls, D. K. & Vrahats, M.. (003). Locally Recurrent Probablstc eural etwor for Text- Independent Speaer Verfcaton. In Proc. Of the EuroSpeech, Vol 3, Gnors, Y. P., Amaral, A.L., colau, A., Coelho, M.A.Z., and Ferrera, E.C. (007). Development of an mage analyss procedure for dentfyng protozoa and metazoan typcal of actvated sludge system. Water Research, 4, Grolam, M. (00). Mercer ernel-based clusterng n feature space. IEEE Trans. eural etwors, 3(3), Muller, K. R. (00). An ntroducton to ernel-based learnng algorthms. IEEE Trans. eural etwors, (), 8-0. Ohlson, J. A. (980). Fnancal ratos and the probablstc predcton of banruptcy. Journal of Accountng Research, 8(), Odom, M. D. & Sharda, R. (990). A neural networ model for banruptcy predcton. IEEE IS IJC,, Specht, D. F. (990). Probablstc neural networ and the polynomal adalne as complementary technques for classfcaton. IEEE Trans. eural etwors, (), -. Scholopf, B. & Smola, A. J. (00). Learnng wth Kernels. Cambrdge: MIT Press. Sansone, C. & Vento, M. (00). A classfcaton relablty drve reect rule for mult-expert system. Internatonal Journal of Pattern Recognton and Artfcal Intellgent, 5(6), -9. Wu, Z. D. & Xe, W. X. (003). Fuzzy c-means clusterng algorthm based on ernel method. The Ffth Internatonal Conference on Computatonal Intellgent and Multmeda Applcatons, -6. Yeh, Y. C. (998). Applcaton of neural networ (Scholars Boo, Tawan Tape, 998). Scholars Boos. Tawan.