A Hybrid AANN-KPCA Approach to Sensor Data Validation

Transcription

1 Proceedngs of he 7h WSEAS Inernaonal Conference on Appled Informacs and Communcaons, Ahens, Greece, Augus 4-6, 7 85 A Hybrd AANN-KPCA Approach o Sensor Daa Valdaon REZA SHARIFI, REZA LANGARI Deparmen of echancal Engneerng College Saon T USA Sharf@amu.edu, Langar@amu.edu, hp:// Absrac: - In hs paper wo common mehods for nonlnear prncpal componen analyss are compared. These wo mehods are Auo-assocave Neural Nework (AANN) and Kernel PCA (KPCA). The performance of hese mehods n sensor daa valdaon are dscussed, fnally a mehodology whch akes advanage of boh of hese mehods s presened. The resul s a unque approach o nonlnear componen mappng of a gven se of daa obaned from a nonlnear quas-sac sysem. Ths mehod s fnally compared wh AANN and KPCA for sensor daa valdaon and shows a beer performance n erms of predcng/reconsrucng he mssng or corruped channels of daa. Key-Words: -Nonlnear PCA, Kernel PCA, Sensor Faul Deecon, Auo assocave Neural Nework. Inroducon Prncpal componen analyss (PCA) s a mahemacal echnque ha s used for compresson of lnearly correlaed daa. Inroduced by Pearson [] n 9 and ndvdually developed by Hoelng [] n 933, hs echnque fnds he drecons wh mamum varance n n-dmensonal se of daa, and ransforms he daa no new coordnae whch s a se of n orhonormal vecor ha have he mamum varance. When ransformed, he drecons wh leas varance whch are called as nose drecons are elmnaed and he remanng dmensons conan he mamum nformaon of orgnal daa wh mnmum possble dmenson. Because of he capably of compresson of daa, hs PCA s very common n daa processng mehods. I s also wdely been used for nose flerng and daa valdaon n lnear sysems. PCA based daa-drven faul dagnoss mehod, whch only depends on he npu and oupu daa of he monored process, have found broad applcaons snce 99 s, especally n process ndusry [3-6]. The core bass of all hese mehods s ha he daa obaned from sensors s nally compressed and subsequenly uncompressed or de-mapped no he orgnal se of daa. Snce PCA s a lnear echnque s performance n sensor faul dagnoss s accepable as long as we have adequae colneary beween dfferen sensor readngs. Bu, mos real sysems do no sasfy hs propery. Therefore, we need o fnd an alernave nonlnear mehod. Nonlnear prncpal componen analyss was nroduced by Kramer n 99 [7]. He used a specfc archecure of neural nework o ran a uny nework. The archecure of an Auoassocave neural nework (AANN) s shown n Fgure. AANN s a fve layer feed forward neural nework where he second and forh layers ncorporae nonlnear ransfer funcons whle he hrd layer as well as he oupu layer have lnear ransfer funcon. In hs archecure he hrd layer has he leas number of neurons. In prncple hs nework provdes and deny map [7];.e. oupu of he nework s he same as s npu. A key elemen n he operaon of he nework s he fac ha fnds he nonlnear correlaon beween dfferen channels of npu. Ths s accomplshed manly va he so called boleneck layer whch has fewer nodes han he npu and oupu layers. Ths layer n effec acs lke a feaure deecor. In oher word he orgnal daa are nonlnearly mapped no fewer dmensons han her orgnal form and subsequenly demapped o her orgnal form. Therefore we can

2 Proceedngs of he 7h WSEAS Inernaonal Conference on Appled Informacs and Communcaons, Ahens, Greece, Augus 4-6, 7 86 epec ha he daa n he reduced dmenson secon ac as a nonlnear prncpal componen of he orgnal daa. The nonlnear relaon for he frs prncpal componen s as follows. p n < > < > = F () = w σ( w j j) = j= < k > where w j Pc () are he elemens of he wegh mar of he k h layer n he neural nework, s he vecor of npus wh dmenson of n, p s he number of neurons n he mappng layer and σ s he acvaon funcon of he relevan neuron. Inpu layer appng layer Boleneck layer appng funcon F() De-appng layer Oupu layer De-mappng funcon G() Fgure - Archecure of Auoassocave Neural Nework Several successful applcaons of AANN n sensor faul dagnoss have been presened snce s brh n 99. Hnes, e al used AANN for sensor daa valdaon n nuclear power plan [8]. aern, e al Appled hs mehod for urbofan engne [9]. Anory, e al. have used hs mehod for ndusral process monorng []. In spe of hese, here are several dffcules n usng AANN for sensor daa valdaon. Frs of all, lke any oher neural neworks, we need o fnd he mnmum number of adjusable parameers n he nework ha can model our funcon whn a ceran error. Ths job s much more challengng n AANN because we have a large nework of fve layers. Anoher problem s o fnd he bes archecure of he neural nework. Kramer [7] n hs nroducory paper on AANN has presened some upper lms for he number of neurons n each layer bu n revewng he applcaon of AANNs n dfferen areas, s evden ha he bes archecure usually has far less parameers (neurons) han he proposed upper lm by Kramer. [- 3] Ths problem s even more consderable when comes o he number of neurons n boleneck. In fac, he number of neurons n boleneck s he number of prncpal componens and can be epressed as he number of ndependen varables n he observed parameers. Therefore selecng more neurons for he boleneck han he number of ndependen parameers quesons he raonaly of usng prncpal componens. In ne par a mehod for fndng he sze of boleneck n AANN s presened. The second problem s ha here s no specfc ranng mehodology for AANN. All of he suded papers n he leraure have used he regular backpropagaon algorhm for ranng. Havng a neural nework wh 5 layers a leas wo of whch have nonlnear ransfer funcons makes very dffcul o ran nework. In fac ranng he nework s an opmzaon problem wh a very complcaed and nonlnear objecve funcon. Wh he real observed varables whch usually are nosy, objecve funcons have mulple local mnmums around he mnmum. Ths problem lms he applcaon of AANNs for sensor faul dagnoss o a very narrow caegory of sysems whch have smple forms of nonlnear correlaon. The mos mporan problem wh AANNs s ha here s no unque soluon for he raned nework. In oher words each me we ran he nework wh dfferen nal condons, we ge dfferen fnal weghs for he nework. Therefore we have dfferen ses of nonlnear prncpal componens. Apparenly some of hese mappngs have beer performance of faul deecon han ohers. Therefore, he mmedae queson s ha how should we ran he nework o perform well n sensor faul dagnoss. Ths problem s also addressed n hs paper n he ne secon.. Kernel Prncpal Componen Analyss ehod of Kernel Prncpal Componen Analyss (KPCA) s nroduced by Scholkopf e al [4] n 998. The smplcy of mehod and s raonal mahemacal base has made hs mehod very favorable. Numerous applcaons of hs mehod have been presened Sne s publcaon,

3 Proceedngs of he 7h WSEAS Inernaonal Conference on Appled Informacs and Communcaons, Ahens, Greece, Augus 4-6, 7 87 especally n paern recognon [5-], bu few of hem are n sensor daa valdaon [],[]. In hs mehod, Insead of solvng prncpal componens n orgnal space,, we fnd he prncpal componen of a nonlnear ransformaon of hs space Φ () where Φ() usually have hgher dmenson han and even can have nfne dmenson. Bu as we wll see, we never need o do any calculaon n hs hgh dmensonal space. Assumng he mappng daa are cenered.e. Φ( =, we can wre he covarance k ) mar: T C = Φ( j ) Φ( j) () j= where s number of observed samples. In order o fnd he prncpal values we need o fnd he egenvalues and egenvecors of covarance mar: λ V = CV (3) Knowng he fac ha all soluons of V le n he span of Φ( ),..., Φ( ) []. We can wre he vecor V as follow: V = = α ( Φ( )) (4) Solvng hs equaon lead o he followng Egenvalue problem λ α = Kα (5) where K s called kernel mar and s defned as followng Kj : j = ( Φ( ) Φ( )) (6) and α s he vecor of α whch are coeffcens of egenvecors used n equaon 4. Afer normalzng he egenvecors, n order o fnd he prncpal componens we have o projec he daa no he normalzed egenvecors. Therefore, he k h prncpal value s k k Pc k = ( V Φ( )) = α ( Φ( ) Φ( )) (7) = One neresng characersc of Kernel PCA s ha we never need o calculae he nonlnear mappng Φ. In fac, we are always dealng wh he do produc of wo mappng funcon. Therefore, Insead of selecng funcon Φ, we can selec a kernel funcon n he followng form k(, j ) = Φ( ) Φ( j ) (8) k : R n R In oher word, he dmenson of nonlnear ransfer funcon can be nfnely large because we never acually work n ha space. Several kernel funcons are dscussed by Boser e al. [3] and Vapnc e al. [4], bu he mos kernel funcon used n KPCA applcaons s Radal Bass Funcon defed as y k (, y) = ep( ) (9) σ The proposed mehod of sensor faul dagnoss wh KPCA s o fnd he prncpal componens of observed daa usng a specfc kernel funcon, hen, fnd nverse ransform of hese prncpal componens back o he orgnal space. Ths process, however, has some shorcomngs. Frs of all we need o know wha s he bes kernel funcon for hs and wha are he parameers of ha funcon for eample f we selec Radal Bass Funcon, defned n equaon, The value of σ s mporan as well. Also, sze of kernel mar s square of he number of observed varables. Therefore, wh hgh number of ranng varables, we have a bg kernel mar and very hgh volume of calculaons. Anoher problem s ha usng KPCA n hs form s n fac a lazy mehod n whch we are usng all of he ranng daa durng he onlne classfcaon of daa. Evdenly, hese ypes of algorhms ake a lo of processng me and memory space and wh a hgh volume of ranng daa hs mehod fals n performance.

4 Proceedngs of he 7h WSEAS Inernaonal Conference on Appled Informacs and Communcaons, Ahens, Greece, Augus 4-6, Sensor Faul Dagnoss wh Kernel PCA If we fnd he nonlnear prncpal componens of he daa, wh KPCA algorhm, we can ran a smple nework o map he daa o he found prncpal componens. I s also very smple o ran anoher nework o reconsruc he orgnal daa ou of he prncpal values. Usng hese wo neworks serally works lke an Auoassocave Neural Nework bu he dfference s ha we have a predefned mappng funcon whch s found based on a raonal logc. Fgure shows hs procedure. Usng KPCA can also help n fndng he number of ndependen varables or he number of neurons n he boleneck layer of AANN, because he number of prncpal componens s acually he number of ndependen valuables. Therefore, hs mehod can be used as a supporng mehod for AANN. Ths procedure s eplaned n he ne secon as an eample. Anoher ndrec applance of KPCA s o use he number of prncpal componens found n hs algorhm as he number of neurons n boleneck layer n AANN. In oher word before we apply AANN, we have an nsgh abou he number of ndependen varables n he ranng daase.,,...,, 3 Y m l m KPCA NN NN y y, y,...,, 3 Y m l Y m l y l b b c Table Comparson of KPCA and AANN Kernel Prncpal componen Analyss The algorhm gves he number of prncpal componens The ranng s a smple lnear procedure prncpal componens are unque appng funcon s lmed o he seleced kernel funcon Slow algorhm wh large number of observaons Lazy algorhm Auoassocave Neural Nework We have o assume he number of prncpal componens The ranng needs nonlnear opmzaon Non unque soluon Capable of wde caegory of mappng funcons No problem wh large number of observaons The ranng daa s no needed when raned 4. Applcaon of ehod n a Sample Problem In order o compare hese mehods quanavely, a sample problem s solved usng AANN and KPCA. In hs problem we assume ha we have fve measuremens whch are funcons of wo ndependen varables. Defned as follows: = + + e = sn( ) + e 3 = cos( + ) + e3 4 = cos( ) + e4 5 = (sn) + e5 Where,..., are measuremens and 5 and are ndep enden varables. In order o provde daa for hs problem, random numbers n he range [, π / ] were generaed. Values of e,...,e5 are nor malzed Gaussan random numbers wh he mean of zero and sandard devaon defned as follows NN NN ˆ m n Fgure - Schemac dagram of sensor faul dagnoss usng KPCA Table compares brefly he wo mehods of AANN and KPCA. Phrases n he shaded able cells are dsadvanages of he mehod d σ =.5 Range( ) where Range( = a( ) n( ) ) In order o fnd he bes archecure of AANN, 6 dfferen archecures has been esed, and for each one process of ranng has been done 5 mes. The algorhm of ranng s Backpropagaon wh Levenberg-arquard opmzaon

5 Proceedngs of he 7h WSEAS Inernaonal Conference on Appled Informacs and Communcaons, Ahens, Greece, Augus 4-6, 7 89 algorhm. Two caegory of archecure are n forma of 5-J--J-5 and 5-J-3-J-5 for he values of J from 4 o nmum ranng error s shown n fgure 3. As you see for values of J>7 he amoun of ranng error does no change consderably wh furher ncreasng of he number of neurons. Therefore, he number of neurons n he second and forh layer s seleced as 7 for boh cases. In order o compare he performance of neworks wh dfferen sze of boleneck, s esed wh and 3 neurons n he boleneck. The npu values are generaed wh he followng values of and =, [,] = cos( ) Afer generaon of he correc values of all 5 sen- sors, one of hem s corruped by n dfferen regons n order o check he ably of he AANN o reconsruc he fauly daa. Tranng Error J-3-J-5 5-J--J J Fgure 3 -Value of ranng error for dfferen srucures The resuls are shown n Fgure 5 and 6 whch are he responses of a and nework respecvely. Comparng hese wo fgures, we see ha alhough he nework wh 3 neurons n he boleneck has less ranng error -see fgure 4-, has less ably o reconsruc error n he channels. I s also noceable ha n he process of reconsrucon for he nework wh 3 neurons n he boleneck, he values of oher channels are also more devaed whch s unfavorable and mgh mslead us o deecon of dfferen sensor as he fauly sensor Correc daa Fauly daa Regeneraed daa Fgure 4 Sensor value before and afer he neural nework n for a nework Correc daa easured.9 daa Regeneraed daa Fgure 6 Sensor value before and afer he neural nework n for a nework Snce we have generaed hese daa wh a known funcon, we know ha hese daa are orgnaed from wo ndependen sources bu hs s no he case for acual applcaon of sensor faul dagnoss. In hese cases, Nonlnear PCA wh KPCA algorhms can be used. In fac, he number of major prncpal componens represens he number of ndependen varables. In order o show hs fac n our sample daa, he algorhm of KPCA s done wh radal bass funcon as defned n Equaon. The parameer of radal bass funcon,σ, s changed dscreely from o and wh each value he algorhm of KPCA s appled and he frs 5 prncpal values are calculaed. Fgure 6

6 Proceedngs of he 7h WSEAS Inernaonal Conference on Appled Informacs and Communcaons, Ahens, Greece, Augus 4-6, 7 9 shows hese values. I s obvous from hs graph ha he frs wo componens have much more conrbuon o he varance of oal daa. Egenvalues of Kernel funcon Pc Pc Pc 3 Pc 4 Pc σ n Radal bass funcon Fgure 6 Nonlnear Prncpal values vs. σ n radal bass funcon Correc daa easured.9 daa Regeneraed daa Fgure 7 Sensor values before and afer he faul dagnoss algorhm wh KPCA 5. Concluson In summary he problems of AANN as a mehod of Nonlnear PCA and as a echnque for Sensor Faul Dagnoss are dscussed and some useful sraeges for fndng he bes archecure of AANN are suggesed. Specally for he number of neurons n boleneck layer whch represens he number of ndependen mbedded varables we showed ha f hs number s more han he number of ndependen varables he nework does no work effecvely for he purpose of sensor faul dagnoss. ehod of Kernel PCA s also dscussed and s pros and cons compared o AANN are suded. Snce n he mehod of KPCA s a lazy algorhm and he reconsrucon of he daa form embedded varables needs very complcaed compuaons and requres a nonlnear opmzaon, we suggesed ranng of wo neural nework, for mappng and reconsrucng respecvely. Usng hese wo neworks n seral emulaes an AANN and he numercal eamples shows he beer performance of hs mehod. References: [] Pearson K., On lnes and planes of closes f o sysems of pons n space Phl. ag., 9, [] Hoellng H., Analyss of a comple of sascal varables no prncpal componens J. Educ. Psychol., 933, [3] Baksh B. R., ulscale PCA wh applcaon o mulvarae sascal process monorng, AIChE J., Vo 44, 998, [4]Gerler G. W., L Y., Huang, and cavoy T. Isolaon enhanced prncpal componen analyss. AIChE J., 999, Vo.45, [5]Duna R., Qn S. J., Subspace approach o mul- dmensonal faul denfcaon and reconsrucon. AIChE J., 998, Vo.44, [6]Haqng W., Zhhnan S., Png L., Faul Deecon Behavor and Performance Analyss of PCA- based Process onorng ehods. Ind. Eng. Chem. Res.,, Vol. 4, [7] Kramer. A., Nonlnear Prncpal Componen Analyss usng Auoassocave Neural Neworks. AIChE J., 99, Vo.37, [8] Hnes J. W., Wres D.J. Uhrg R.E. Plan Wde Sensor Calbraon onorng. Proceedngs of he 996 IEEE Inernaonal Symposum on Inellgen Conrol, Dearborn, I Sepember 5-8, 996 [9] aern D.L., Jaw L.C., Guo T., Graham R. and ccoy W., Usng Neural Neworks for Sensor Valdaon, 34h IAA/ASE/SAE/ASEE Jon Propulson Conference & Ehb, July 3-5,998 / Cleveland, OH

7 Proceedngs of he 7h WSEAS Inernaonal Conference on Appled Informacs and Communcaons, Ahens, Greece, Augus 4-6, 7 9 []Anory D., Kruger U, Irwn G.W., ccullough G., Indusral process monorng usng nonlnear prncpal componen models, Inellgen Sysems, 4. Proceedngs. 4 nd Inernaonal IEEE Conference, Volume, -4 June 4 Page(s):93-98 Vol. [] aern D.L., Jaw L.C., Ten H.G., Graham R. and ccoy W., Smulaon of an engne sensor valdaon scheme usng an Auoassocave Neural Nework, AIAA/ASE/SAE/ASEE Jon Propulson Conference and Ehb, 33rd, Seale, WA, July 6-9, 997 []Huang J.; Shmzu H.; Shoya S. Daa preprocessng and oupu evaluaon of an Auoassocave neural nework model for onlne faul deecon n vrgnamycn producon Source: Journal of Boscence and Boengneerng, Volume 94, Number, July, pp. 7-77(8) [3] Tan S., ayrovounos.l., Reducng daa dmensonaly hrough opmzng neural nework npus, AIChE Journal Volume 4, Issue 6, Dae: June 995, Pages: [4]Scholkopf B. Smola A. uller K.L., Nonlnear Componen Analyss as Kernel Egenvalue Problem. Neural Compuaon, Volume, Number 5, July 998, pp () [5] L W. Gong W., Lang Y. Chen W., Feaure Selecon Based on KPCA, SV and GSFS for Face Recognon Lecure Noes n Compuer Scence, 5, no. 3687, pp [6] He Y. Zhao L. Zou C., Face Recognon Based on PCA/KPCA Plus CCA, Lecure Noes n Compuer Scence, 5, no. 36, pp [7] Zhao, Q; Lu, H GA-Drven LDA n KPCA Space for Facal Epresson Recognon Lecure Noes n Compuer Scence, 5, no. 36, pp [8] Lu B., B D., Tan J., A ehod of Image Denosng Based KPCA, Infrared Technology, 4, vol. 6, no. 6, pp [9] Kaeda K., Abe S., KPCA-based ranng of a kernel fuzzy classfer wh ellpsodal regons, Inernaonal Journal of Appromae Reasonng, 4, vol. 37, no. 3, pp [] Twnng C.J., Taylor C.J., The use of Kernel Prncpal Componen Analyss o model daa dsrbuons. Paern Recognon 36, 3, 7-7 [] Cho J., Lee J. Cho S. W., Lee D., Lee I.B., Sensor faul denfcaon based on Kernel Prncpal Componen Analyss., Proceedngs of he 4 IEEE Inernaonal Conference on Conrol Applcaons, Volume, -4 Sep., 4,3-8 Vol. [] Cho S. W., C. Lee and I.-B. Lee, Faul deecon and solaon of nonlnear processes based on kemel PCA, Chemomercs and nellgence Laboraory Sysems [3] Boser B. E., Guyon I.., Vapnk V. N., A Tranng Algorhm for Opmal argn Classfers., Proceedng of he 5 h Annual AC Workshop on Compuaonal Learnng Theory., 99, p44-5 [4] Vapnk V., Chervonenks A., The Naure of Sascal Learnng Theory., 995, New York Sprnger-Verlag.