Using Cellular Automata for Improving KNN Based Spam Filtering

Transcription

1 The Inernaonal Arab Journal of Informaon Technology, Vol. 11, No. 4, July Usng Cellular Auomaa for Improvng NN Based Spam Flerng Faha Bargou, Bouzane Beldjlal, and Baghdad Aman Compuer Scence laboraory, Unversy of Oran, Algera Absrac: As rapd growh over he Inerne nowadays, elecronc mal (e-mals) has become a popular communcaon ool. However, junk mal also, known as spam has ncreasngly become a par of lfe for users as well as nerne servce provders. To address hs problem, many soluons have been proposed n he las decade. Currenly, conen-based an-spam flerng mehods are an mporan ssue; he spam flerng s consdered as a specal case of bnary ex caegorzaon. Many machne learnng echnques have been developed and appled o classfy emal as spam or non-spam. In hs paper, we proposed an enhanced -Neares Neghbours (NN) mehod called Cellular Auomaon Combned wh NN (CA-NN) for spam flerng. In our proposed mehod, a cellular auomaon s used o denfy whch nsances n ranng se should be seleced o classfy a new e-mal; CA-NN selecs he neares neghbours no from he whole ranng se, bu only from a reduced subse seleced by a cellular auomaon. eywords: Spam e-mal flerng, machne learnng, NN, cellular auomaa, nsance selecon. Receved Sepember 27, 2011; acceped Augus 18, 2012; publshed onlne Aprl 4, Inroducon The problem of undesred elecronc messages called spam s nowadays a serous ssue. Spam can be defned as unsolced (or junk) emal for a recpen or any emal ha he user does no wan o have n hs nbox. Spam s a bg problem because of he large amoun of shared resources consumes ncludng sorage space, bandwdh, processng me and he resulng loss n producvy. To solve he spam problem, several spam flerng sysems have been proposed by boh academc communy and he ndusry, rangng from smple blacklsng [7] o advanced ex classfcaon [3, 9, 15, 18]. Among hem, approaches whch use machne learnng algorhms o classfy e-mals have acheved more success [11]. Indeed, machne learnng echnques are wdely used n auomaed ex classfcaon, ncludng spam flerng (see [24] for a survey of echnques used). The -Neares Neghbours (NN) rule [12] s known o be one of bes sae of he ar classfers used for ex classfcaon. Many sudes ha have used he Reuers corpus [17, 26, 27] sugges ha NN and Suppor Vecor Machnes (SVM) ouperform oher mehods lke Lnear leas square f, Naïve Bayes (NB) and Neural neworks. The dea of NN can be explaned as follows: gven a es documen o be classfed, he algorhm searches for he k neares neghbours among he pre-classfed ranng documens based on some smlary measure, and ranks hose k neghbours based on her smlary scores, he caegores of he k neares neghbours are used o predc he caegory of he es documen by usng he ranked scores of each as he wegh of he canddae caegores, f more han one neghbour belong o he same caegory hen he sum of her scores s used as he wegh of ha caegory, he caegory wh he hghes score s assgned o he es documen. NN s robus and placed among he op algorhms. I was recommended prevously as a praccal approach. In 1999, Yang [26, 27] consdered NN as one of recommended approaches among more han en approaches o ex caegorzaon, and n 2002, Sebasan [23] recommended, snce s smple and comparable o he bes approach SVM. In addon, o s good performance, s very easy o undersand and o mplemen. However, he praccaly wll be los when he NN algorhm s appled o ex caegorzaon wh hgh dmenson; has ceran lmaons: Huge Memory: I requres sorng he whole ranng se. Hgh Compuaon Cos: I has o explore he enre ranng se n order o classfy a new documen. Low Tolerance o Nose: Because uses all daa as relevan even when he ranng se conans nose or unbalanced daa. Our am s o furher mprove he performance of NN algorhm by adopng a new sraegy called CA- NN. We propose he use of he cellular auomaon CASI [4] for wo purposes; frs, o encode ranng documens no a boolean represenaon, and secondly o search quckly a subse of relevan nsances from

2 346 The Inernaonal Arab Journal of Informaon Technology, Vol. 11, No. 4, July 2014 ranng se o be used by NN algorhm. Wh hs model we can face me and sorage requremens and unbalanced daa of NN algorhm. In hs paper, we examned CA-NN for he ask of e-mal spam flerng. The res of he paper s organzed as follows: Secon 2 wll descrbe he relaed work abou spam mal flerng. Secon 3 oulnes he prncple of he cellular auomaon CASI. Secon 4 s devoed o he descrpon of he proposed mehod. The expermenal sudy o evaluae hs approach s presened n secon 5 and he work s concluded n secon Conen-Based An Spam Flerng Numerous spam flerng sraeges wh varyng degrees of effcency have been proposed and developed (see [16, 24] for a survey of echnques used). However, despe hs ncreasng developmen, he number of spam messages connues o ncrease. The frs publshed work on sascal spam flerng was proposed by Saham e al. [19]. They proposed a Baysan neworks o fler spam emal usng bag of words represenaon and bnary weghng. However, he ranng and es daa were no large enough n he expermen and he daa are no publcly avalable. Androusopoulos e al. [2] consruced he Lngspam corpus and used o compare a mulnomal Naïve Bayes classfer and a varan of NN classfer. Boh mehods acheved very hgh classfcaon accuracy. Carreras and Marquez [8] used AdaBoos algorhm and mproved resuls on he same corpus. Compared wh wo learnng algorhms, he nducon Decson Trees (DT) and NB, he mehod clearly ouperformed han he above wo learnng algorhms. In expermens by Sakks e al. [20], a classfer combnaon mehod s proposed. They combned a NB and knn by sackng and found ha he ensemble mproved performance over any of he classfers separaely. Schneder [22] performed expermens wh wo sascal classfer models mulvarae Bernoull model and a mulnomal model. To selec he words whn he vocabulary, dfferen feaure selecon measures were used. Expermens obaned very hgh flerng raes hgher han 95 %. SVM was proposed for he frs me by Drucker e al. [13] o fler spam emals. Comparsons wh varous classfcaon echnques lke Boosng ree, Rpper and Roccho show ha SVM s he bes. Exensve ess have been performed comparng dfferen confguraons of dfferen classfers. For example, comparsons were made beween NB, maxmum enropy, NN, SVM and AdaBoos [28]. The resuls showed ha SVM, maxmum enropy and AdaBoos classfers were much more effecve han NN and even he popular NB classfer. El-halees [14] compared several supervsed machne learnng for flerng spam e-mal from mxed Arabc and Englsh messages. The expermens suggesed ha words n Arabc messages should be semmed before applyng classfer. Cormack and Lynam [10] esed he real-world spam flerng ools SpamAssassn, Bogofler, SpamProbe and CRM114 agans he LngSpam corpus. They found ha real spam flers were n general unable o classfy he Lngspam messages correcly. Bargou e al. [5] proposed a new symbolc nducon approach based on cellular auomaa o fler spam hey red o examne he mpac of varous feaures selecon algorhms, semmng and sop words removal on he performance of he cellular auomaon classfer. Expermens show a very hgh qualy of predcon when usng semmng and feaure selecon wh nformaon gan funcon. A performance mprovemen s observed over he repored resuls of NB and NN [2]. A combnaon of NB classfer and cellular auomaon s proposed n [6]. Majory vong provded beer resuls compared o ndvdual classfers. Sanos e al. [21] have explored he use of semancs n spam flerng by represenng e-mals wh a recenly nroduced Informaon Rereval model: he enhanced Topc-based vecor space model. Ths model used a semanc onology o deal wh synonymy. Based upon hs represenaon, hey apply several well-known machne-learnng models (NB, NN, SVM and DT) and show ha he proposed mehod can deec he nernal semancs of spam messages. Expermens on Lngspam corpora show ha hs approach provdes hgh percenages of spam deecon whls keepng he number of msclassfed legmae messages low. 3. Cellular Auomaon CASI The cellular auomaon CASI [4] s a boolean nference engne. I s organzed no cells where each cell s conneced only wh s neghbours. All cells obey n parallel o he same rule called local ranson funcon, whch resuls n an overall ransformaon of he sysem. I uses a knowledge base n he form of wo layers of fne auomaa. The frs layer, called CelFac, represens he fac base and he second layer, called CelRule, represens he rule base. In each layer, he conen of a cell deermnes wheher and how parcpaes n each nference sep: a every sep, a cell can be acve or passve, can ake par n he nference or no. The saes of cells are composed of hree pars: EF, IF, SF, and ER, IR, SR whch are he npu, nernal and oupu pars of he CelFac cells, and he CelRule cells, respecvely. Any cell n he CelFac layer wh npu EF () = 1 s regarded as represenng an esablshed fac. If EF

3 Usng Cellular Auomaa for Improvng NN Based Spam Flerng 347 () = 0, he represened fac has o be esablshed. Any cell j of he CelRule layer wh npu ER (j) = 0 s regarded as a canddae rule. When ER (j) = 1, he rule should no ake par n he nference.the neghbourhood of cells s defned by wo ncdence marces called R E, R S respecvely. They represen he npu respecvely oupu relaon of he facs and are used n forward channg. The npu relaon, noed R E j, s formulaed as follows: If (fac Premse of rule j) hen R E j = 1 else R E j = 0. The oupu relaon, noed R S j, s formulaed as follows: If (fac Concluson of rule j) hen R S j = 1 else R S j = 0. The cellular auomaon dynamcs mplemened as a cycle of an nference engne made up of wo local ranson funcons δfac and δrule, where δfac corresponds o he evaluaon, selecon and flerng phases and δrule corresponds o he execuon phase. The ranson funcon δfac s defned as: T ( EF, IF, SF, ER, IR, SR) ( EF, IF, EF, ER+ ( RE EF ), IR, SR) The ranson funcon δrule s defned as: ( EF, IF, SF, ER, IR, SR) ( EF + ( R S ER), IF, SF, ER, IR, ER) We have made changes a hs auomaon: 1. We have defned hree layers of fne auomaa nsead of wo o represen ranng se. 2. We have brough changes n he wo ranson funcons δfac and δrule n order o exrac relevan documens. The deals of hs model wll be represened n secon Proposed Approach In hs secon, we presen our approach o enhance NN classfcaon. Frs of all we gve he followng defnons and noaons: Defnon 1: Tranng documens se D = {(d, c ) = 1, 2,, N}, where d s a ranng documen and c s a label caegory 1 {0,1}. Each documen d D s assocaed wh a class label c, whch ndcaes wheher d, belongs o he arge class C (c = 1) or no (c = 0). Defnon 2: Vocabulary of erms, V = { j ; j = 1, 2,, M}, where j s a erm (n our case hs s he sem). Defnon 3: Documens represenaon as vecors each documen d s represened as a vecor = (w1,w 2,, w M ), where w j s he wegh of erm d j n documen d. As we have declared n nroducon, NN echnque s very smple, hghly effcen and effecve n he feld of classfcaon. Bu s compuaon cos s very expensve especally for ex caegorzaon where he number of documens and feaures s large. The algorhm s very slow because has o exhausvely mach all he ranng documens agans he es documen o fnd s k-neares neghbours. The mplemenaon requres sorng he complee ranng se, and classfcaon akes me proporonal o he sze of he ranng daa mes he dmenson of he feaure vecors. Also, for unbalanced corpora, he classfcaon performance ofen decreases. For all hese reasons, we propose a new approach named Cellular Auomaon combned wh -Neares Neghbours (CA-NN), whch s based on he noon of rerevng a subse of relevan documens from ranng se o classfy a new nsance. The selecon of hs subse wll be done by he cellular auomaon CASI. Our fndngs are: The number of erms n common beween he documen o classfy and he one of ranng se s an neresng parameer o locae relevan documens ha wll be nvolved n he calculaon of closes neghbors. Dscardng every documen n he ranng corpus no sharng any words wh he documen o classfy doesn ncrease error rae and should be used as a fler before measurng he smlary beween he ranng documen and he new one. We know ha he smlary beween documens s sensve o he number of erms n common. As hs number ncreases as he documens become more smlar. In hs research, we propose o keep only relevan documens from ranng se for classfyng new nsances. We show ha excludng rrelevan documens from ranng se when classfyng a new documen mproves sgnfcanly he NN classfer. The man challenge here s how o esmae ha a documen s relevan or non-relevan. We consder ha a ranng documen s rrelevan when he oal number of vocabulary erms n common wh he new documen s below a hreshold. Our hypohess s ha ype of documens represens nose; hey are no useful ranng nsances because hey decrease he classfcaon accuracy. We use cellular auomaon CASI [4] for wo purposes: frs o represen he ranng se and secondly o exrac relevan documens whch mus be nvolved n classfyng a new nsance. As shown n Fgure 1 bellow, he proposed approach consss of wo processes: Insance represenaon. Insance selecon and classfcaon. 1 We consder only bnary classfcaon.

4 348 The Inernaonal Arab Journal of Informaon Technology, Vol. 11, No. 4, July 2014 Tranng E-mals New Emal Preprocessng Feaure Selecon Insance selecon -NN Classfer Class label Fgure 1. CA-NN dagram. Boolean Represenaon Table 2. The hree layers used for documens encodng. CEL-TERM CEL-DOC Word ET IT ST Dfferenal Exrac Index Marx ID-Doc ED ID SD d d d d d d Insance Represenaon We propose an alernave sraegy of encodng documens; ranng documens are encoded no a Boolean represenaon (by usng he cellular auomaon CASI). Before ha, he ranng se s frs preprocessed; we exraced word okens from he daa, removed sop words and used a varan of he Porer 2 algorhm for semmng. Snce oo many erms are usually exraced, some of hem should be seleced as feaures. Many schemes of selecng feaures were already proposed [25]. In hs work, o reduce vocabulary sze we kep only feaures seleced by he nformaon gan funcon. Once he ndex s bul and reduced, we oban a documen-by-word marx A (N M) lke ha shown n Table1. Column M + 1 conans he class label of documens. The h documen d s represened by he characersc vecor d = (w1, w 2,, w M, c ). In hs paper, we deal wh a bnary weghng.(w j ) = 1 f he erm j V s presen one or more mes n d, 0 oherwse. Afer ha, we proceed wh he proposed sraegy of encodng documens. To llusrae hs encodng, le us consder he ranng se D represened n Table 1. D = {d 1, d 2, d 3, d 4, d 5, d 6 }; V = {dfferenal, exrac, ndex, marx} and C = {0, 1}. Table 1. Vecor represenaon. Dfferenal Exrac Index Marx Class = c d d d d d d The cellular auomaon CASI has been revewed and some changes have been esablshed n order o represen he ranng documens. We have defned hree layers nsead of wo. Table 2 shows he hree layers modellng he ranng se: 1. CEL-TERM layer s planned o hold all he erms of vocabulary V. Inally, all he cells npus are passve (ET = 0). CEL-RULE Rule ER IR SR R R R R CEL-DOC layer s nended o hold he denfers of ranng documens. Inally, all he cells npus are passve (ED = 0). The ID allows us o dsngush beween documens n he class c and hose who are no (ID = 1 f he documen s classfed c; 0 oherwse). 3. CEL-RULE layer represens he presence of he erm n he ranng documens. For each erm j V we assocae a rule R j whch ell us where he erm occurs. Accordng o example of Table 1, we have four rules: R 1 : If (erm = dfferenal ) hen d 4, d 5, d 6 R 2 : If (erm = exrac ) hen d 1, d 2, d 3 R 3 : If (erm = ndex ) hen d 1, d 3 R 4 : If (erm = marx ) hen d 1, d 4, d 6 For example he rule R 1 ndcaes ha he word dfferenal could be rereved from documens d 4, d 5 and d 6. The erms are lnked o her documens by Inpu Marx (IM) and Oupu Marx (OM): The IM marx s M M of dmenson, whle OM marx s of dmenson N M. For example, he erm dfferenal, whch s he npu of rule R 1 n IM marx s found n documens d 4, d 5 and d 6, whch are he oupu of R 1 n OM marx as shown n Table Inpu relaon: IM (, r) { j \ j CEL-TERM; j = 1... M} r { R j \R j CEL-RULE; j = 1... M} If ( s a premse of r) hen (IM (, r) =1). 2. Oupu relaon: OM (d, r) d { d \ d CEL-DOC; =1... N} ; r { R j \R j CEL-RULE; j = 1... M}: If (d s n concluson of r) hen (OM (d, r) =1) We can observe ha he concaenaon of OM marx wh ID vecor of CEL-DOC layer corresponds o he documen-by-word marx usng a bnary weghng. 2 hp://ararus.org/~marn/porersemmer/.

5 Usng Cellular Auomaa for Improvng NN Based Spam Flerng 349 IM OM Table 3. Inpu and oupu marces. Word R1 R2 R3 R4 Dfferenal Exrac Index Marx ID-Doc R1 R2 R3 R4 d d d d d d Insance Selecon and Classfcaon In hs secon we descrbe n deal our algorhm for nsance selecon and classfcaon. Before applyng our nsance selecon algorhm, we wll assume ha he Boolean represenaon of ranng documens has been creaed. We wll only operae on hs represenaon o deermne a subse of relevan documens usng algorhm 1. Le consder q he new unlabelled documen o classfy and TNT (q) he oal number of vocabulary erms ( V) and found n q. We defne a hreshold T (η, q) o be TNT( q ) + 1, whereη 2. η For a ranng documen d we defne TC (d ) o be s oal number of erms n common wh q. A documen d s relevan for classfcaon f sasfes he followng condon: TC (d ) T (η, q). δ fac s defned as follows: ( ET, IT, ST, ER, IR, SR) ( ET, IT, ET, ER+ ( IM T ET ), IR, SR) δ rule s defned as follows: ( ED, ID, SD, ER, IR, SR) ( ED+ ( OM ER), ID, SD, ER, IR, ER) Any ranng documens whose TC s lower han he hreshold T (η, q) s rejeced and can be used n he classfcaon process. Algorhm1: Insance Selecon Algorhm 1. Parameer η 2. Inpu: New unlabelled documen (q), CEL-TERM, CEL-RULE, CEL-DOC, IM and OM 3. Oupu: E D a subse of relevan documens 4. A =, E = 5. Calculae T (η, q) 6. Inalze CEL-DOC 7. For each erm j V q do 8. ET( j ) = 1 9. End for each 10. Apply δ fac δ rule 11. For each d n CEL-DOC do 12. If ED(d ) = 1 hen A = A {d } Calculae TC (d ) = And (OM (d ), ET T ) 13. End f 14. End for each 15. For each d A do 16. If TC (d ) T(η, q) hen E = E {d } 17. End f 18. End for each 19. Oupu he subse E To classfy a new nsance q, CA-NN operaes n hree seps. Durng he frs sep, CA-NN nalzes he CEL-TERM layer by acvang he ET sae for each cell correspondng o he erms found n q and sars he nference by applyng he global ranson δfac δrule o selec documens conanng a leas one erm n common wh q see equaon 1. We oban a reduced se of ranng documens called A: A = { d D ; where TC ( d ) 1} (1) In he second sep, and o furher reduce he ranng se, CA-NN searches whn A for he subse of documens sasfyng he condon TC (d ) T (η, q). These documens consue he se E see equaon 2: T E = { d A; where ( ET ) and OM ( d ) T ( n, q)} (2) The se E s obaned by calculang for each d n A, he oal number of acve cells of he oupu logc operaor AND of he (ET) T wh OM (d ). In he hrd sep, CA-NN uses NN algorhm wh subse E as ranng se o classfy he new documen. To llusrae hs algorhm, le us consder he es documen q wh he vocabulary erms: {ndex, exrac}. In CEL-TERM layer, only ET (ndex) and ET (exrac) saes wll be se o 1 as shows n Fgure 2-a. Afer, applyng δfac δrule he rereved documens are: d 1, d 2 and d 3 her ED sae s acvaed. Documens d 3, d 4 and d 6 are gnored as shows n Fgure 2-b. In he second sep, we fxed η=2 TNT ( q) and he hreshold wll be (n hs case s equal o 2 ) he classfcaon wll be done only wh d 1 and d 3. Documen d 2 s also, dscarded from he se of ranng documens as shows n Fgure 2-c. a. Inalzaon Term 1 Dfferenal 2 Exrac 3 Index 4 Marx b. Inference (δ fac δ rule): Documen 1 d 1 2 d 2 3 d 3 4 d 4 5 d 5 6 d 6 ET IT ST CEL-TERM ED ID SD CEL-DOC A={d 1, d 2, d 3} c. From A keep only hose sasfyng TC(d ) 2: T C ( d 1 ) = OM ( d 1 ) A N D ( ET ) = 2 T C ( d 2 ) = OM ( d 2 ) A N D ( ET ) = 1 T C ( d 3 ) = OM ( d 3 ) A N D ( ET ) = 2 E={d1, d3} Fgure 2. Insance selecon example.

6 350 The Inernaonal Arab Journal of Informaon Technology, Vol. 11, No. 4, July Expermenal Evaluaon To evaluae he proposed approach for spam flerng, we have underaken expermens on he Lngspam corpus. Lngspam s freely avalable and has been used n many sudes [1, 2, 5, 6, 20, 21]. I consss of 2412 legmae emals and 481 spam n he oal daase; he specaly of hs corpus s larger legmae examples han spam examples. Spam e-mals represen only 16% of he whole daase. So, he class of legmae e-mals s larger han he class of spam e-mal Performance Measures We measured several ndcaors of classfcaon performance: he recall of class spam (SR), precson class spam (SP), he F-measure of class spam (F1) and fnally he accuracy (A) shown n equaons 3 o 6. Le TN he number of legmae emals classfed as legmae (rue negaves), TP he number of spam emals classfed as spam (rue posves), FP he number of legmae emals classfed as Spam (False Posves) and FN he number of spam emals classfed as legmae (false negaves), hen we have: D q d w q w d 2 E (, ) = ( ( ) ( )) ; V w ( q ) s he wegh of erm V n he new nsa an ce q ; w ( d ) : s he wegh of erm V n documen d D c ( q ) = A rgm ax y c d y ( c k, d ) = (, ) d ( ) k k N N q c k { spam, legm ae } 1 d c k 0 d c k (10) (11) Where knn(q) denoe he se of neares neghbours of new nsance q. The resuls manly dscuss he effecveness of he proposed approach; Fgures 3, 4, 5 and 6 show he resuls of he precson, recall, F-measure and accuracy when applyng CA-NN on Lngspam daa. Precson TP SP= TP + FP TP SR= TP + FN (3) (4) Fgure 3. CA-NN s precson as a funcon of number of seleced arbues and hreshold. 2 SP SR F1= SP+ SR TP+ TN A= TP + TN + FP + FN (5) (6) Furhermore, we measured he True Posve Rao (TPR) shown n equaon 7, he False Posve Rao (FPR) shown n equaon 8 and he oal cos rao (TCR) 3, shown n equaon 9. Greaer TCR values ndcae beer performance: TP TPR = TP + FN FP FPR = FP+ TN TP+ FN TCR= FP + FN 5.2. Expermenal Resuls and Dscusson (7) (8) (9) We performed a k-fold cross valdaon wh k = 10. The daase was spl 10 mes no 10 dfferen ses of learnng ses (90% of he oal daase) and esng ses (10% of he oal daa). We used he Eucldean dsance for searchng k neghbours see equaon 10 and majory vong o deermne he class of he new emal see equaon 11: Recall Fgure 4. CA-NN s recall as a funcon of number of seleced arbues and hreshold. F1 Fgure 5. CA-NN s F1-measure as a funcon of number of seleced arbues and hreshold. The resuls of performance measures depend on he number of seleced erm-arbues and he hreshold k. As we can see, excep precson whch has s bes values when he number of seleced erm-feaures s 100 and k = 23, recall, accuracy and F1 measures have he bes resuls wh lower values of k (beween 3 o 7) and hgher values of erm-arbues. When k ncreases, here s a decrease on recall, accuracy and F1. 3 We consder TCR wh only λ =1. Accuracy wh varyng vocabulary sze

7 Usng Cellular Auomaa for Improvng NN Based Spam Flerng 351 Accuracy From he resuls, s shown ha our proposed mehod ouperforms he radonal NN wh greaer accuracy. Table 4. Performance of NN and CA-NN wh he bes confguraon on Lngspam daa. Fgure 6. CA-NN s accuracy measure as a funcon of number of seleced arbues and hreshold. Fgures 5 and 6 show a resul of 98,55% score for accuracy and 96,0% score of F-measure. These values are acheved when k akes 5 and arbues number akes 500. The reason for hese resuls s he way how CA-NN works; a reduced number of neghbors s suffcen o make decsons because CA-NN searches he k neghbors from a subse of relevan documens. We do no need o ncrease he number of neghbors o mprove decson; on he conrary, usng hgh values of k can harm he effecveness of classfcaon because more documens wh low smlary scores are nvolved, low smlary score ndcaes ha es and ran documens are probably of dfferen caegores, and hey may confuse he classfer decson. We have also, mplemened he radonal NN approach o compare wh CA-NN (Fgures 7-9). F1-Measure Fgure 7. The F1-Measure by varyng he value from 1 o 33. Accuracy Fgure 8. The accuracy of CA-NN and NN by varyng he value from 1 o 33. Toal cos rao Fgure 9. The oal cos rao of CA-NN and NN by varyng he value from 1 o 33. Table 4 depcs he bes confguraons for performance accuracy of CA-NN compared wh radonal NN. M SR(%) SP(%) F1(%) A(%) TCR NN ,39 98,88 84,25 95,05 3,64 CA-NN ,1 94,9 95,99 98,55 12, Comparson wh Prevous Work To evaluae he conrbuon of he proposed algorhm o spam flerng, we compare our approach wh he bes repored resuls of boh real-world soluons and academc approaches ced n secon 2 usng he Lngspam corpora see Table 5 below. Table 5. Comparson wh publshed works. Model A (%) TPR FPR [1] λ = 1 λ = 9 λ = 99 [20] = 5, λ = 1 and m = 100 = 3, λ = 9 and m = 200 = 7, λ = 1 and m = 300 = 3, λ = 9 and m = 100 [22] Bernoull mv-mi mn-mi dmn-mi f-mi df-mi [10] SpamAssassn BogoFler SpamProb CRM 114 [21] Bayesan Nework DT: Random fores N = 10 CA-NN = 5 m = ,06 96,33 94,19 98,06 97,20 84,89 97,30 98,00 98,86 98,06 85,52 98,79 98,48 84,1 90,1 94,8 81,5 99,26 98,72 0,83 0,78 0,65 0,92 0,84 0,90 0,85 0,89 0,96 0,93 0,17 0,96 0,95 0,04 0,40 0,69 88,8 0,97 0,94 0,08 0,08 0 0, ,45 0,00 0,00 98,55 0,97 0,00 Table 5 shows he bes resuls obaned usng he CA-NN echnque alongsde hose prevously publshed [1, 10, 20, 21, 22] and repored n [21]. The parameer k s he neghbourhood sze for he NN, λ deermnes he srcness of he crera for classfyng an e-mal as spam, m s he number of arbues. The resuls ndcae mproved performance when classfyng wh CA-NN. 6. Conclusons In hs paper a new mehod called CA-NN s proposed o mprove he NN s classfcaon performance. CA-NN needs no searchng he neares neghbours from all ranng se based on cellular auomaon CASI. Therefore, he searchng scope s reduced and only a subse of relevan

8 352 The Inernaonal Arab Journal of Informaon Technology, Vol. 11, No. 4, July 2014 documens s seleced o parcpae n he classfcaon of a new nsance. By usng a cellular model o represen ranng documens and rereve relevan documens for classfcaon, we have shown ha our proposed mehod no only mproves he classfcaon accuracy bu also, a performance mprovemen s observed over radonal NN and oher publshed works. The man conrbuons n hs paper are summarzed as follows: We propose and develop an mproved NN algorhm whch s beer han classcal NN algorhm whle mprovng he sorage memory and classfcaon performance. Based on he cellular auomaon CASI [4], we propose for he frs me a cellular nference echnque whch allows us o search very quckly from ranng se only relevan documens and skp he ohers whch are no mporan for classfcaon. We mprove sorage memory by organzng he ranng se no a cellular srucure. We mprove performance classfcaon and face he problem of nosy or unbalance daa by selecng only relevan documen from ranng se o classfy he new documen. Fnally, we evaluae hs algorhm for he e-mal spam flerng. Alhough, he fndngs are neresng and encouragng, many ssues may be suded n fuure work. We mus also conduc a dealed comparson of hs approach wh oher learnng algorhms used n spam flerng and consderng oher corpus such as Spam Assassn and evaluaon crera such as weghed accuracy and weghed error. References [1] Androusopoulos I. and ousas J., An Evaluaon of Nave Bayesan Neworks, n Proceedngs of Machne Learnng n he New Informaon Age, Barcelona, Span, pp. 9-17, [2] Androusopoulos I., Palouras G., arkaless V., Sakks G., Spyropoulos C., and Samaopoulos P., Learnng o Fler Spam E-Mal: A Comparson of a Nave Bayesan and a Memory- Based Approach, n Proceedngs of he 4 h European Conference on Prncples and Pracce of nowledge Dscovery n Daabases, Lyon, France, pp. 1-13, [3] Apé G., Damerau F., and Wess S., Auomaed Learnng of Decson Rules for Tex Caegorzaon, ACM Transacons on Informaon Sysems, vol. 12, no. 3, pp , [4] Aman B. and Beldjlal B., nowledge Dscovery n Daabase: Inducon Graph and Cellular Auomaon, Compung and Informacs Journal, vol. 26, no. 2, pp , [5] Bargou N. and Bargou F., A Boolean Model for Spam Deecon, n Proceedngs of he Inernaonal Conference on Communcaon, Compung and Conrol Applcaon, Tunsa, pp , [6] Bargou F., Bargou N., and Aman B., Combnng Classfers for Spam Deecon, Inernaonal Conference on Neworked Dgal Technologes, vol. 293, no. 3, pp.78-89, Duba, UAE, [7] Carpner J. and Hun R., Tghenng he Ne: A Revew of Curren and Nex Generaon Spam Flerng Tools, Compuers and Secury, vol. 25, no. 8, pp , [8] Carreras X. and Marquez L., Boosng Trees for An-Spam Emal Flerng, n Proceedngs of he 4 h Inernaonal Conference on Recen Advances n Naural Language Processng, Bulgara, pp , [9] Cohen W., Learnng Rules Tha Classfy E- Mal, n Proceedngs of AAAI Sprng Symposum on Machne Learnng n Informaon Access, Sanford, Calforna, USA, pp , [10] Cormack G. and Lynam T., Onlne Supervsed Spam Fler Evaluaon, ACM Transacons on Informaon Sysems, vol. 25, no. 3, pp. 1-31, [11] Cormack G., Emal Spam Flerng: A Sysemac Revew, Foundaons and Trends n Informaon Rereval, vol. 1, no. 4, pp , [12] Cover T. and Har P., Neares Neghbor Paern Classfcaon, IEEE Transacons Informaon Theory, vol. 13, no. 1, pp , [13] Drucker H., Wu D., and Vapnk V., Suppor Vecor Machnes for Spam Caegorzaon IEEE Transacons on Neural Neworks, vol. 10, no. 5, pp , [14] El-halees A., Flerng Spam E-Mal from Mxed Arabc and Englsh Messages: A Comparson of Machne Learnng Technques, he Inernaonal Arab Journal of Informaon Technology, vol. 6, no. 1, pp , [15] Gee., Usng Laen Semanc Indexng o Fler Spam, n Proceedngs of ACM Symposum on Appled Compung, Daa Mnng Track, Melbourne, Florda, USA, pp , [16] Guzella T. and Camnhas W., A Revew of Machne Learnng Approaches o Spam Flerng, Exper Sysems wh Applcaons, vol. 36, no.7, pp , 2009.

9 Usng Cellular Auomaa for Improvng NN Based Spam Flerng 353 [17] Joachms T., Tex Caegorzaon wh Suppor Vecor Machnes: Learnng wh Many Relevan Feaures, n Proceedngs of he 10 h European Conference on Machne Learnng, Berln, Germany, vol. 1398, pp , [18] ara H., Flerng Junk E-Mal: A Performance Comparson beween Genec Programmng and Nave Bayes, Techncal Repor, Unversy of Waerloo, [19] Saham M., Dumas S., Heckerman D., and Horvz E., A Bayesan Approach o Flerng Junk E-Mal, Techncal Repor, Learnng for Tex Caegorzaon Workshop, Sanford Unversy, U, [20] Sakks G., Androusopoulos I., Palouras G., and arkaless V., Sackng Classfers for An- Spam Flerng of E-mal, n Proceedngs of he 6 h Conference on Emprcal Mehods n Naural Language Processng, Psburgh, USA, pp , [21] Sanos I., Laorden C., Sanz B., and Brngas P., Enhanced Topc-Based Vecor Space Model for Semancs-Aware Spam Flerng, Exper Sysems Wh Applcaons, vol. 39, no. 1, pp , [22] Schneder., A Comparson of Even Models for Nave Bayes An-Spam E-Mal Flerng, n Proceedngs of he 10 h Conference of he European Chaper of he Assocaon for Compuaonal Lnguscs, Sroudsburg, USA, pp , [23] Sebasan F., Machne Learnng n Auomaed Tex Caegorzaon, ACM Compung Surveys, vol. 34, no. 1, pp. 1-47, [24] Upasana P. and Chakravery S., A Revew of Tex Classfcaon Approaches for E-Mal Managemen, Journal of Engneerng and Technology, vol. 3, no. 2, pp , [25] Yang Y. and Pedersen J., A Comparave Sudy on Feaure Selecon n Tex Caegorzaon, n Proceedngs of he 14 h Inernaonal Conference on Machne Learnng, Nashvlle, USA, pp , [26] Yang Y., An Evaluaon of Sascal Approaches o Tex Caegorzaon, Informaon Rereval, vol. 1, no. 1-2, pp , [27] Yang Y. and Lu X., A Re-Examnaon of Tex Caegorzaon Mehods, n Proceedngs of he 22 nd Annual Inernaonal ACM SIGIR Conference on Research and Developmen n IR, New York, USA, pp , [28] Zhang L., Zhu J., and Yao T., An Evaluaon of Sascal Spam Flerng Technques, ACM Transacons on Asan Language Informaon Processng, vol. 3, no. 4, pp , Faha Bargou s a compuer scence eacher n he Deparmen of Compuer Scence a Oran Unversy (Algera). She earned her Maser of Scence Degree n 1998 from Oran Unversy. She s currenly a PhD canddae n he Compuer Scence Deparmen a he same unversy. Her research neress focus on ex mnng, nformaon exracon, and nformaon rereval areas. Bouzane Beldjlal receved hs PhD degree n compuer scence from he Unversy of Oran (Algera) n He s a professor n he Compuer Scence Deparmen a he Unversy of Oran. Hs research neress nclude formal specfcaons, knowledge managemen, daabases, arfcal nellgence, and auomac learnng. Baghdad Aman receved hs Maser of Scence Degree n 1996 from he Deparmen of Compuer Scence n Oran (Algera). He s currenly a PhD canddae n he Compuer Scence Deparmen a he Unversy of Oran. Hs research neress nclude knowledge dscovery n daabases, daa mnng, feaure selecon, neural neworks, and cellular auomaa.