Neural net Decision Support System for Hand-written Author Identifications

Transcription

1 Neural et Decso Support System for Had-wrtte Author Idetfcatos A Ermoleko Maagemet Academy of the Mstry of the Iteror of the Russa Federato, Moscow, Russa Abstract. Questos of Decso Support System o processg of had-wrtte detfcatos are cosdered. The author has developed a orgal allocato method of uque hadwrtg characterstcs o the bass of the le ad symbols aalyss. The formatoal cotet crtero s modfed ad parametrc base dvsblty s show.. Itroducto I practce, polce experts solve the problem of hgh-qualty authetcato of hadwrtg. Varous complexes of automato of maual work of the expert crmalst are developed. However, the operato of these systems s focused o the operato of arrow professoals ad s ot desged for hgh-speed processg of large amouts of data. Hadwrtg authetcty assessmet systems for large socal systems, whch ot oly automate the work of a expert, but also detfy the author, are poorly developed. Advatages of fuzzy logc ad eural etworks are ot fully used.. The cotrol system model Cosder the property state protecto system by polce uts. The ma place the tree of goals of polce protecto s the preveto of theft from protected obects ad apartmets (goal - A). I addto, t s mportat to receve come from securty servces to the state budget ( goal - A), the dsclosure of crmes (goal - A3), the search for crmals (goal - A4) ad the preveto of admstratve offeses (goal - A5). Formalze the model of maagemet () of the tactcal level protecto ut the form of []: * S o, T, Q, S, A, B, Y, f, K, Y, () where: S 0 problem stuato; T the tme for makg decsos; Q avalable resources for decsomakg; S =(S,, S ) set of alteratve stuatos that defe the problem stuato S 0 ; А =(А,, А К ) set of the goals requred for decso makg; B = (B,, B ) set of lmtatos; Y = (Y,, Y m ) set of alteratve decso; f a fucto of the prefereces of the decso makers; К soluto decso crtera Y * best decso []. The loss fucto () the developed model s represeted by: f ( x) (max K( x),m P( x),m R( x)) 0 T Tmax () 0 Q Qmax where K come from securty servces (correlated wth the umber of protected obects), P the rsk of rembursemet from the Federal budget damage by theft or fre o protected obects, correlated wth 7

2 the troducto of coeffcets wth the formalzed values of the equpmet of moder facltes by alarm systems (P ), by protecto of buldg structures (P ), by authetcty ad relablty of formato o the fuctog of the alarm system, securty alarm cotrol codes ad authorzed persos of protected obects (P 3 ) ad other parameters of P formg: P P (3), R Iteral costs such as fuel costs (R ), ammuto for the trag of rapd respose teams (R ) ad other parameters collapsble addtvely takg to accout the coeffcets by : m R R (4). Cosder mmzg the rsk of egatve cosequeces of poor qualty of securty servces, as well as reducg the teral costs of the orgazato []. Usg rsk maagemet techologes wth logc ad probablstc models (LPM) let's make L-scearo (fgure ) ad detfy 4 of the dagerous codtos: Ecoomc ad Socal losses S S Thefts from protected obects due to the alarm system dsable from protecto o the bass of a llegtmate applcato & Itetoal arso due to fre alarm loops dsable o the bass of a false statemet & S,S Thefts, fres due to o-complace wth the customer requremets for the qualty of securty or fre alarm, costructo safety the absece of a wrtte refusal of the st perso from the obect & S 4 -S 7 Wrte-off of goods ad materals at flated volumes Z 9 & S 3 Forgery of the appeal about alarm system shutdow Z Illegtmate add code to dsarm Peetrato o obect Z Z 3 И Loss of valuables Z 4 Exteral ope fre source Itetoal volato of wrg Cofrmato of securty requremets waver by ot the st perso Fake cofrmato of securty requremets waver (ow employee, thrd party) Z 5 Z 6 Z 7 Z8 Submsso of coset to the persoal data processg by oe perso Fgure. Logcal scearo of dagerous States whle esurg safety of obects. Gve the above scearo, the l-fucto of the dagerous state s represeted as: Z 0 Ispecto by Supervsory authortes у z... z zz3 z4 zz3z4 zz5 zz6 zz6 z3z4z7 (5) z3z4z8 z5z7 z5z8 z6z7 z6z8 z9 z0 z z0 z We have studed the problem lks of the formalzed cotrol model. The aalyss showed that the actual aspect s the preveto of decso-makg o forged hadwrtte documets. It s mportat to work wth hadwrtte documets, ot oly customers but also ther ow employees. The preseted Cause ad Effect Dagram as Fshboe Dagram by Kaoru Ishkawa [3] emphaszes that the cotrol process there s a eed for expert support of decsos by meas of artfcal tellgece (fgure ). These tools should be focused o the developmet of ew methods ad models of hadwrtg detfcato. Z clets persoal data leakage Z 7

3 Dshoesty of ts staff A colluso wth clet Neglgece Expertse equpmet lack Acceptace of appeals by Fax, mal, e-mal Clet oretato Acceptace of cosets from the st perso for all authorzed persos Departmet maagemet process Сroysm Decso Support Systems The problem of decso-makg whe cosderg hadwrtte appeals Low qualfcato of employees Lmted resources for scag ad processg formato Large documets array Iteral hadwrtte documet flow Usg tellget methods ad models to support decso-makg Fgure. Causalty of the decso-makg problem. 3. Developmet of Decso Support System The proposed decso support system (DSS) s based o fuzzy eural et detfcato (Fgure 3). Iput btmap hadwrtte documet mage Database of hadwrtg samples Determato of hadwrtg coarse (strg) parameters Determg of ma hadwrtg coarse (strg) parameters, cludg spaces ad les to hghlght the word Normalzato, equalzato by weght coeffcets Selecto of a text fragmet (words, letters) Fe (by symbols) characterstcs determato Normalzato, equalzato by weght coeffcets clet-server database Decso makg module by eural etwork Decso makg module by eural etwork Fal decso module o the bass fuzzy logc Decso makg Fgure 3. The proposed archtecture of DSS. 73

4 The aalyss carred out the course of the research showed that dfferet characterstcs of the hadwrtg emphasze the uqueess of ts author dfferet ways [4]. I order to assess the formatve characterstcs of hadwrtg ad ther balacg, a modfcato of the Shao crtero was developed (6): t _ per true _ maх false _ mп t _ per false _ maх true _ maх false _ mп true _ mп f _ false x dx f _ true x dx f _ false x dx f _ true x dx Idetfcato wth groups of hadwrtg parameters s realzed by meas of eural etworks. Whe creatg the detfcato module, the possbltes of usg varous eural etworks, cludg self-orgazg, radal-bass ad probablstc oes, were studed ad tested. At the same tme, the studes have show that the optmal decso model s the back propagato etwork. Formato of the trag sample wll produce formalzato of hadwrtg characterstcs of the orgal mage. The soluto classfcato problem of hadwrtg samples s solved the tadem "orgal" - "falsfcato". Image the trag ad the test sample F_O, T_O, F_T, T_T dfferetal form: x x x x x x x x3 x x3 x x x xn x xn x xn S x x3 x x3 x x 3 S O x... x4 x x4 x x4 (7) S N x xn x xn x x N xn xn xn xn xn x N ths wll crease the sample sze to NN dmo M М ММ, (8), where М coarse (by strgs) ad М fe (by symbols) set of parameters. To fd the optmal algorthmc complexty balace ad detfcato qualty, varous methods of backpropagato eural etwork trag (gradet method, Leweberg-Marquardt method, quas- Newto method) has bee vestgated. The best covergece rate ad mmze the mashg learg error EL G X Y ad geeralzato error E G F s bee quas-newto BFGS algorthm XX (Broyde Fletcher Goldfarb Shao algorthm) [5]. Algorthm fuctog s based o determato gradet vector at the k-th terato g k by multplcato Bk Pk gk that approxmates Hessa-matrx by matrx B k for ext drecto optmzato P k. BFGS-coverso formula s defed as Sk, gk Yk Yk, gk Sk Sk, gk Yk, Yk Pk gk Sk (9) Y, S Y, S Y, S k k k k k k Sk X k X k vector equal to the chage of Х the k-th terato, Yk gk gk the correspodg chage of gradet. I the BFGS formula uses the results of the prevous step. Thus, the st step should be carred out a dfferet way, for example, by the steepest descet method wth mmzg the error of the eural etwork: (6) 74

5 where p E W y d, (0) y value of -th output of eural etwork; d desred value of -th output; p umber of euros output layer. Chagg the weght wll produce the formula: E w h, () w where h - the parameter that determes the learg speed. The frst layer of the proposed eural etwork topology represets 5 euros wth tagecesgmodal actvato fucto. The secod layer cossts of euro wth lear actvato fucto. The umber of trag epochs was 000. Research of the detfer model have show (Fgure 4) uambguous separablty of both strg ad character parameters at the output of the eural etwork. Fgure 4. Neural etwork testg: left - for coarse detfcato, rght - for fe detfcato. Decso-makg usg fuzzy logc s realzed o the bass of the composte rule by Mamda ferece by meas of fuzzy mplcato: z ' ' z z. () C C C C C The put fuctos of the accessory are represeted by bell-shaped dstrbuto fuctos of characterstcs x x e xm. The lgustc output of the fuzzy model forms the aswer about the authetcty of the hadwrtte documet. It s possble to apply the cetrod method (ceter of gravty method) of output characterstc defuzzfcato: (3). Z Z Z Z CT С С 4. Idetfcato of had-wrtg characterstcs The coarse (strg) characterstcs determato of hadwrtg s realzed by mage segmetato, clusterg [6], vrtualzato of complete strgs clusters, two-level flterg ad approxmato of the referece traectory wrtg of strgs (fgure 5). 75

6 Level flter Least squares approxmato р, word δ й д fragmet determato for fe aalyss р level flter detfcato the δ-corrdor of a pecewse lear approxmato clusterg Fgure 5. The text settgs defto of coarse (row). After fltrato, t becomes possble to approxmate the referece traectory. Cosder the clusterg the mage. We use the flter to remove small clusters of puctuato marks ad large clusters of captal letters. Cut the par strgs oed oe cluster. The approxmato s feasble by the least squares method [7]. Dfferetatg the resdual fucto by ukow parameters b ad c ad equatg dervatves to zero, we obta a system of equatos: x y bx c x 0 b (4) x y bx c 0 b The approxmato of the referece traectory allows formg a orthogoal vrtual ruler: h av (5) where h av - the average clusters heght, - the parameter of the symbols devato, - the heght of the vrtual ruler. The aalyss of symbol pxel testes whe scag wth a vrtual ruler allows to determe 0 coarse strg parameters, as well as to select dvdual words for a th (by symbol) aalyss of the peculartes of wrtg letters ad ter-letter coectos by the author (fgure 6). h h 3 h Fgure 6. Character ecodg by trapezods ad determato of 3-beam coectos. The developed fe symbolc detfcato methods s basg o codg of the symbols cotour by trapezes [8] wth dyamcally chagg bases ad ther subsequet fltrato. The developed method formato of vectors begs wth the selecto of pots par (for example, M ad the ext P) of the frst studed curve of the hadwrtg strg. Havg the coordates xy, of the pot, drawg a le through t Ay Bx C 0 wth determstc parametersa, B, C. All 76

7 pots of cotour N x ; y the path betwee codto: cost, where M P Ax By C must meet the followg, Ay Bx C. (6) A B Determato s realzed by cosecutve descrpto of the symbol the graph represetato wth formato of statstcal matrx ad adacecy lst as well as auxlary lst (fgure 7). 0. X X. X. X X. X X. X X. X X. X X X X X X.. X X X. X X X X X X. X X.. X X. X X. X X. X X. X X. X X 8 X X. X X. X X X. 0:, 8, : 0,, :, 3, 3:, 4, 4: 3, 5, 5: 4, 6, 9, 6: 5, 6, 7, 7: 6, 9, 9, 8: 9,, 9: 7, 8, 0:,, : 8, 0, : 0, 3, 3:, 4, 4: 3, 5, 5: 4, 6, 6: 5, 7, 7: 6, 8, 9, 8: 0, 7, 9, 9: 5, 7, 7, 8, 5: 4, 6, 9, 6: 5, 6, 7, 7: 6, 9, 9, 7: 6, 8, 9, 8: 0, 7, 9, 9: 5, 7, 7, 8, Fgure 7. Matrx, the adacecy lst; auxlary lst. O the bass of the proposed model of decso-makg o eural etworks ad detfcato methods of 4 hadwrtg characterstcs we have developed software. The ma wdow of the developed software s show fgure 8. Tme clasterg Tme to determe the agle of clato Fgure 8. The ma Wdow of the developed software DSS. The proposed methods make t possble to formalze 4 hadwrtg characterstcs, wth ther subsequet cluso the DSS. The evaluato of formato cotet allows us to coclude that the most formatve character characterstcs are the agles of 3-beam coectos, ad from the umber of strg characterstcs - the relatve heght of the letters ad the agles of clato. The tme of operato ( t ) of the DSS cossts of the work of strg ( t ) ad character ( t ) eural etwork detfcato, as well as geeralzg fuzzy ferece ( )ad s ot more tha 7 secods: 77

8 t t t 3sec 3,5sec 0,5sec 7sec (7) The proposed algorthms were tested o a depedet sample of 000 samples. Cofdece terval a postve detfcato ths case was 0.9. The software has receved state regstrato. The software s mplemeted the clet-server database cocept ad ca be tegrated to law eforcemet databases. 5. Cocluso As a result of researches the decso support system allowg detfyg the author of the hadwrtte text s sytheszed. The decso model based o BFGS-eural etwork wth the method of trag o the backpropagato algorthm s selected. The developed fuzzy-eural etwork model of detfcato ad methods of formato processg ad determato of the uque characterstcs of hadwrtg are mplemeted ready-made software that allows detfyg the author of hadwrtg wth 7 secods of computer examato wth a cofdece terval of Refereces [] Soloetsev E 004 Scearo Logc ad Probablstc Maagemet of Rsk Busess ad Egeerg. Sprger. [] Soloetsev E 00 Logcal ad Probablstc Rsk Maagemet valdty flght tests of maches, processes ad systems // Problems of Rsk Aalyss, Volume 7, N4. pp 7 85 [3] Soloetsev E ad Lebedev Y 006 Logc-probablctc models of compay maagemet usuccess rsk / Mathematcal Ecoomcs, No 0, The Publshg House of the Wroclav Uversty of Ecoomcs. Wroclav, 3(40). [4] Vapk V 998 Statstcal Learg Theory. NY: Joh Wley [5] Avrel ad Mordeca 003 Nolear Programmg: Aalyss ad Methods. Dover Publshg [6] Castlema K 979 Dgtal lage Processg. Pretce-Hall: Eglewood Clffs. 407 p. [7] Gray, Lawrece F. Flaga, Fracs J. Kazda, Jerry L., Frak ad Davd H 990 Calculus two: lear ad olear fuctos, Berl: Sprger-Verlag, p 375 [8] Bartlett P ad Shawe-Taylor J 998 Geeralzato performace of support vector maches ad other patter classfers // Advaces Kerel Methods._ MIT Press, Cambrdge, USA. ( Ackowledgmets I Express my grattude to my supervsor, doctor of techcal Sceces, prof. Igor Goroshko (Maagemet Academy of the Mstry of the Iteror of the Russa Federato), my teacher Ph. D., assocate Professor. Korlyakova M. O. (Bauma Moscow State Techcal Uversty) ad my father's Ph. D., Assoc. Ermoleko V. A. (Bauma Moscow State Techcal Uversty) for hs valuable cotrbuto to the developmet of my scetfc potetal. 78