A Fuzzy Clustering Approach to Filter Spam

Transcription

1 , July 6-8, 2011, London, U.K. A Fuzzy Clustering Approch to Filter Spm E-Mil N.T.Mohmmd Abstrct Spm emil, is the prctice of frequently sending unwnted emil messges, usully with commercil content, in lrge quntities to set of indiscriminte emil ccounts. However, since spmmers continuously improve their techniques in order to compromise the spm filters, building spm filter tht cn be incrementlly lerned nd dpted becme n ctive reserch field. Reserches employed mchine lerning techniques which hve been widely used in solving similr problems like document clssifiction nd pttern recognition, such s Nïve Byesin, nd Support Vector Mchine. In this Pper, we exmine the use of the fuzzy clustering lgorithm (Fuzzy C-Mens) to build spm filter. The proposed use of the Fuzzy hs been tested on different dt set sizes collected from Spm ssssin corpor by rel user s emils. After testing Fuzzy C-Mens using Heterogeneous Vlue Difference Metric with vrible percentges of spm nd using stndrd model of ssessment for the spm problem, we demonstrte the potentil vlue of our pproch. Index Terms Spm filtering, Fuzzy clustering, Fuzzy C- Mens. I. INTRODUCTION Spm, or unwnted commercil emil, hs become n incresing problem in recent yers. Estimtes suggest tht perhps 70% of ll emil trffic is spm. As spm clutters inboxes, time nd effort must be devoted to either deleting it fter it is received, or preventing it from even reching the user [9]. The problem of spm multiplies dily, nd is n nnoynce to every user of emil. Some estimtes suggest tht the verge per person is 10 working dys per yer spent solely deling with spm[10]. Commercilly vilble spm filters must judge n emil to decide whether it is spm or legitimte, colloquilly clled hm. These rely mostly on pttern mtching rules tht re mnully constructed [2].The construction of such rules is not trivil tsk nd requires expertise. Fig. 1. Spm s percentge of ll emil trffic N.T.Mohmmd is with Deprtment of Computer Informtion Systems, University of Jordn. (emil: nehy.mohmmd@yhoo.com) As shown in Figure 1 [9], the proportion of spm to legitimte emil chnges over time, which puts the spm filtering in the ctegory of skewed clss distribution problems [6]. Spm to hm emil proportion vries from person to person s well s over time. Fwcett lso suggests tht the proportion of spm is influenced by the emil recipient s domin, how esy it is to cquire tht emil ddress, nd how long n ddress hs been in existence. A misclssified spm tht rrives in user s inbox is nnoying. A misclssified hm tht the user never sees my result in loss of business, productivity, opportunity, or time. Spmmers ctively ttempt to defet spm filters by substituting look-like chrcters for letters, hiding rndom text in n emil, misspelling words, including pictures tht show the dvertisement, or embedding links into deceptively-phrsed emils. Their techniques chnge dily. Therefore ny nti-spm technology must be ble to dpt quickly. Automted methods of spm filtering tht cn lern how to distinguish spm emils from hm emils nd cn be trined lern in n updtble fshion - re of vitl importnce. A good nti-spm technique will hve three chrcteristics: it will ccurtely clssify spm nd hm, it will be esily dptble, nd it will be esily sclble. Most of the current reserch in spm filtering concentrtes on using dt mining pproches to solve the spm filtering. According to dt mining, the spm is clssifiction problem where the filtering system ims t distinguishing spm from legitimte (hm) emils. Thus, clssifiction lgorithms tht re widely used for pttern recognition cn be used to solve the spm problem. In this pper, we study the spm filtering s dt mining nd n AI problem. We im to evlute the current stte of reserch nd propose our own solution to the problem. We present n implemented system tht is bsed on using strong distinguishing fetures of spm emil nd clssifiction technique tht suits the problem. The rest of the pper is orgnized s follows. Section 2 overviews spm filtering work tht look t common methods of fighting spm, including rtificil intelligence influenced techniques. The fuzzy spm filtering technique is given in section 3. Results re given in section 4. Finlly, conclusions nd future work re given in section 5.

2 , July 6-8, 2011, London, U.K. II. RELATED WORK Vrious techniques exist for filtering spm. These methods cn be generlly ctegorized into techniques tht hve been influenced by rtificil intelligence nd mchine lerning, nd other techniques. These other techniques tend to be older nd less robust. For exmple, use of white lists, blck lists, nd gry lists is strightforwrd; if the emil is sent from known spmmer, it is mrked s spm; if it is sent from user-pproved ddress, it is llowed through to the inbox. Anything else is gry listed to folder where the user cn pprove it s vlid or mrk it s spm. The difficulty with this pproch is tht the burden on the user cn be considerble. Rules-bsed spm filters pply prewritten rules to spm, such s if subject contins Vigr, emil is spm. These my ccidentlly result in misclssifiction of rel emil s spm, clssifiction of spm s vlid emil, nd must be updted frequently to sty brest of spmmers techniques. Both of these techniques hve their plce; however, they should not be relied upon s the only filter. Content-bsed filters re founded on the premise tht it is possible to crete set of rules, exemplrs or fetures tht represent the degree to which n emil is to be considered s spm, nd tht if this is over some threshold, is considered to be spm. Such filters hve been the focus of considerble interest, with work on rule-bsed filters, nerest neighbor clssifiers [12], decision trees [5] nd Byesin clssifiers [11]. Initil implementtions of these filters were centrlized, but with spm comprising 50% of ll emils trffic. As the knowledge bse is now in the hnds of the system dministrtors, it cn be customized to suit the chrcteristic emil nd spm tht individul domins receive. Users cn feed informtion bck bout flse positives nd flse negtives tht enbles the filter to be retrined. Spm Assssin given in [13] is perhps the most known exmple of this pproch. Thus the huge contentbsed filters hve been developed towrds higher degree of collbortion s they hve become decentrlized Clutters. sclble nd esy to updte pproch. If ech emil tht comes in is used s prt of the dt pool to mke decisions bout future emils, spm trends will be detected nd dpted to utomticlly. There is not the lrge cost of reclcultion tht would occur with decision trees, or the mnul mintennce of rules-bsed filters. Even if there ws one optiml solution to the spm problem currently, with thousnds of spmmers looking for new wys to defet it, the pyoff of reserch into lternte methods is pprent. In this pper, we evlute the use of text mining nd fuzzy clustering s n nti-spm technique. III. FUZZY SPAM FILTER MODEL Figure 2 shows the min steps of the spm filter developed in this pper. The filter consists of three min stges: feture extrction, trining nd testing. In the coming discussion, ech of these stges is discussed. Fig. 2. Spm Filter Model A. Feture Extrction Before we could extrct fetures, we first hd to find the dt we would use for testing nd trining. We sought ctul emils, both spm nd vlid, nd s up to dte s possible. There re severl corpuses of emil vilble on the web; we eventully used two of the collections from the SpmAssssin Spm Corpus [13]. These contined over 3,000 emils submitted by vrious people, ll lbeled s either spm or hm, in text-only formt. Mny other projects nd experiments hve been performed upon SpmAssssin corpuses, nd they re esily vilble. Therefore, mking the experiments we performed esy to replicte nd to compre with other pproches. Figure 3 shows how the text mining works in our pproch to finlly provide the clssifiction of the emils. Mchine lerning techniques re more vried nd flexible. Decision trees [5] clssify emil s spm or hm bsed on previous dt. They re costly to clculte nd reclculte s spmmers chnge techniques. Byesin networks [11] re the most populr nti-spm technique currently, but they cn be difficult to scle up nd rely on mny fetures to mke their judgments. In this pper, we evlute the use of fuzzy clustering nd text mining for spm filtering. Fuzzy clustering is Fig. 3. Text mining system min phse

3 , July 6-8, 2011, London, U.K. The next step in the procedure ws to define nd extrct fetures. The min difficulty we encountered here ws in choosing fetures to extrct. Though most of the previous works emphsized the importnce of feture selection for ccurte spm detection, they only give brod description of the fetures they used. We cme up with preliminry list of fetures tht included emil length in chrcters, percent of non lphnumeric chrcters in the emil, percent of white spce chrcters, verge word length, nd number of emil ddresses in the heder. After running the first few experiments, we dded word list of suspected spm words such s mortgge nd kept count of how mny of those words ppered in the emil, s well s counting HTML tgs nd whether font or bckground colors hd been set. Ech emil ws run through our Jv progrm to extrct these fetures, which were sved to n individul file s vlues seprted by blnk spces. The files were nmed with n s for spm or h for hm, followed by the emil id number, to mke our record keeping esier. Dt file nme ws not feture we used in clculting the clusters. Our Jv code took folder nd processed every file in it, producing one feture file per initil dt file. Once this informtion is extrcted, it fed it to the fuzzy clustering lgorithm. As our experiments progressed, fetures were refined, or expnded. B. Fuzzy Clustering Fuzzy clustering hs been successfully pplied to vriety of problems rnging from vector quntiztion coding nd neurl networks trining, to more specific fields s diverse s food clssifiction, wter qulity nlysis, nd wether forecst. However, the two min fields where fuzzy clustering excels re pttern recognition nd imge processing. Fuzzy clustering for pttern recognition cn be pplied to text clssifiction problem such s spm, where the ptterns to be clssified re texts. While mny clustering lgorithms hve been introduced, the Fuzzy C-Men (FCM) lgorithm, first presented by Bezdek [4] is the most populr one. FCM ssumes the number of clusters is known or if not known, then t lest some fixed number. Ech of the c clusters is represented by prototype v. These prototypes re chosen rndomly t the i beginning nd ech trining vector is ssigned degree of membership to clusters with respect to the vector s distnce from the cluster prototype. The cluster prototypes re then replced by the center of grvity of the vectors tht belong to ech cluster. The lgorithm repetedly lters ssignment of ptterns to their nerest cluster nd updtes prototypes until the lgorithm converges. Convergence is reched when chnges re less thn specified threshold. Figure 4 describes the steps performed by the FCM. FCM ims to minimize the objective function given in eqution 1: f 1u d ij ij 1 c i n j (1) Under the constrint in eqution 2: c u ij i 1 f 1,for j =1,,n (2) Where: u ij {0,1} indictes the degree of membership of vector v j in cluster c i. Step 1: Select the number of clusters c, initil prtition mtrix u, the termintion criterion ε. Also, set the itertion index l to 0. Set the fuzzifier [1,2]. Choose Distnce Metric. Step 2: Clculte the fuzzy cluster centers Step 3: Clculte the new prtition mtrix Fig. 4. Fuzzy C-Men Clustering Algorithm (FCM),,, Step 4: if mximum chnge in u > ε, return to step 2. When FCM is trined, given vector is declred to belong to the cluster for which it hs the mximum membership. A modified pproch constrints the degree of membership to exceed predefined threshold [16]. The level of success of the FCM lgorithm relies on three min fctors. First,setting the Correct number of clusters is vitl. Second, the fuzzifier m in formul 4 must be set. Third, suitble distnce metric must be selected. Choosing these settings is non-trivil tsk. Mny pproches hve been proposed for predicting the number of clusters nd for setting the fuzzifier [16]. A discussion of such pproches is beyond the scope of this pper. For our work, we dopted the following two simple pproches:

4 , July 6-8, 2011, London, U.K. 1. Experimentlly setting the vlue of the fuzzifier. 2. Experimentlly finding the minimum number of the clusters tht gives the best results using the sme environmentl setting. Clusters tht contin less thn predefined number of vectors re discrded. Vectors which re members of these discrded clusters re lso discrded s outliers C. Distnce Metric A lerning lgorithm must hve bis in order to generlize. A bis is rule or method tht cuses n lgorithm to choose one generlized output over nother (Mitchell 1980). Wilson [14] showed tht no lerning lgorithm cn generlize more ccurtely thn ny other when summed over ll possible problems. The bis of the lerning lgorithm depends on the distnce metric used. This gives the conclusion tht no distnce metric cn be better thn ny other in terms of generliztion bility, when considering ll possible problems [14]. A distnce metric cn be suitble to prticulr problem or set of problems if it cn improve the generliztion bility of the lerning lgorithm by being ble to ctch the chrcteristics of the problem, normlly presented by vector in clustering problems. Wilson [14] presented distnce metric tht hndles problems involving nominl nd continuous fetures. The distnce metric improved performnce on collection of 48 pplictions. Since our fetures vector consists of nominl nd continuous fetures. The Heterogeneous Vlue Difference Metric (HVDM) is used. The HVDM is defined s: HVDM(x, y) 2 d ( x, y ) (5) Where m is the number of ttributes, function d (x,y) returns the distnce between two vlues x nd y for ttribute nd is defined s: 1, if x or y is unknown f normlized_vdm(x, y), if f is nominl (6) normlized_diff(x, y), if is liner The function normlized_diff is defined in eqution (7) nd function normlized_vdm hve three lterntives, the one used in this pper is lbeled s N1 in [14] nd is defined in eqution (8). x y normlized _ diff ( x, y) (7) 4 N, x, c N, y, c normlized _ vdm (8) N, x N, y IV. RESULTS As stted in the introduction, the proportion of spm emils to hm emils vries from person to person nd over time. As result, it is not esy to prove the performnce of ny spm filter since it will be bised to its trining nd testing sets. Fwcett [6] suggested using the Probbility Cost Function (PCF), which is the x-xis of cost curve s non-bised mesure of spm filter s performnce. The filter is viewed s spm detector where spm emil is positive input nd hm emil is negtive one. A spm PCF cn be defined s: PCF spm p( spm) * cos t( FlseNegtive) p( spm) * cos t( FlseNegtive) p( hm) * cos t( FlsePositive) Where p(spm) nd p(hm) is the probbility of spm emil nd hm emil respectively. Flse Positive is clssifying hm emil s spm while flse negtive is considering spm emil s hm. Clerly, flse positive error hs greter cost thn flse negtive. As suggested in [6] vlue of 10 times cost of the negtive flse cn be used, ccordingly the PCF for spm nd hm cn be reduced to: PCF spm PCF hm p(spm) p(spm) p(hm) *10 (9) p(hm) *10 p(spm) p(hm) *10 (10) Equtions (9) nd (10) will be used to evlute the performnce of the proposed spm filter. A. Experiments Experiment 1 In order to vlidte the presented pproch, severl experiments were conducted. The gol is to mesure the performnce of the vrious enhncements s well s coming out of model tht gives the best performnce using the proposed pproch, nmely, the FCM lgorithm. As stted in the feture selection discussion, in order to reduce the dimensionlity of the feture spce nd to provide the clssifier with domin-knowledge fetures, feture selection is performed. In experiment 1, we evluted the performnce of the FCM lgorithm s spm filter using Euclidin distnce s distnce metric nd without performing ny normliztion

5 , July 6-8, 2011, London, U.K. on the feture s vlues. Tble 1 summrizes the vrious settings used to conduct the experiment. Note tht the initil number of clusters nd the fuzzifier vlues re experimentlly set. FCM Prmeters Trining set TABLE 1: SETTINGS FOR EXPERIMENT 1 Initil number of 12 clusters fuzzifier 1.9 Distnce Metric Euclidin distnce Initil setting of Rndom weights Stopping criteri Mx chnge in Uij< E=.0001 Spm proportion 70% Hm proportion 30% Size 2000 Normliztion None Experiment 1 results re given in tble 2. The performnce ws below the desired vlues of the PCF for both the spm nd the hm emils. To understnd why tht might hppen, consider the smple vlues of some of the fetures presented in tble 3, which indictes wide vriety in the feture vlue rnges. Fetures with lrge vlues nd wider rnges will be dominnt; for instnce, the totl number of chrcters vries more between emils thn does the number of HTML tgs. In ddition, Euclidin distnce is difficult to clculte properly when used with discrete vlues. Clerly normliztion nd more suitble distnce metric re required to hve better performnce. TABLE 2: RESULTS OF EXPERIMENT 1 Error Rtes Number of misclssified emils Totl Error Rte 66% 851 Flse Positive 34.1% 133 Flse Negtive 52% 473 PCF(Spm) 19% PCF(Hm) 81% Experiment 2 Here, we pplied the HVDM described in the distnce metric section on our set of fetures before feeding then into the clustering lgorithm, using the sme sitting s in experiment 1. Tble 4 gives summry of the experiment. Clerly, the Performnce is drmticlly better thn the previous one nd exceeds the PCF for both spm nd hm emils. This grees with the results obtined by [Wilson] regrding the suitbility of the HVDM metric to the text clssifiction problems set of which spm filtering is prt. To verify the performnce of our pproch, we run severl tests with vrible proportions of spm to hm emils strting from 10% spm up to 90% spm. The results cn be seen in figures 3 nd 4. Vrying the proportion of the spm emils incresed the flse negtive error when the proportion of spm emils ws lower, but the error rte ws still quite low. Since the totl mount of spm is smll, the ctul mount of misclssified spm should not be problem. In comprison, the flse positive error rte shows lmost stble behvior, even when the proportion of hm ws only 10%. The flse positive error did not exceed 1.5%. Since the repercussions of flse positives re much higher thn tht of flse negtives, our results re promising. The flse positive rte is too high compred to commercil spm filters but we believe it could be lowered with dditionl tweking of fetures extrcted from the smple emils, lrge smple popultion, or perhps modifiction of the distnce metric. TABLE 3: SAMPLE FEATURE VALUES FOR A HAM . Attribute Nme Vlue Totl number of chrcters in the messge 2424 Number of white spce chrcters 244 Number of specil chrcters 558 Number of words 240 Averge word length 7.6 Percent of specil chrcters 23.2 Percent of non-lphnumeric chrcters 32.9 Number of URLs 3 Number of suspected Spm words 11 Number of HTML tgs 0 Number of colors 1 Averge number of chrcters per word 7.1 Percent of totl chrcters to specil chrcters 4.9 Percent of totl chrcters to Alphnumeric 13.3 chrcters Percent of cpitl word to simple word 0.01 Percent of cpitl chrcters to totl chrcters Number style formt used in the html code such 0 tht (hed, body, font,...,etc) Number of smile fce symbols 0 TABLE 4: RESULTS OF EXPERIMENT 2 Error Rtes Number of misclssified emils Totl Error Rte 4.5% 59 Flse Positive 0.7% 3 Flse Negtive 6.1% 56 PCF(Spm) 19% PCF(Hm) 81% Fig. 4. Flse Negtive Error Rte. Note number on x-xis is multiplied by 10, so 9 stnds for 90%. The higher flse negtive rte cn be explined by the type of extrcted fetures, which seem to be bised towrd clssifying emil s hm. Consider the number of spm

6 , July 6-8, 2011, London, U.K. words ttribute, very strong feture in most spm filtering pproches. In pproches like the one used in [11], predefined set of words is given to the clssifier. If the size of the set is n, nd number of spm words is m, then m/n of the given words re spm which decreses the flse negtive error if the emil is rel spm, but increses the flse negtives if the emil is hm. However, in our pproch, this will only strengthen one feture, which is combined with other fetures to decide whether the emil is rel spm. [2] I. Androutsopoulos, J. Koutsis, K. V. Chndrinos, nd C. D. Spyropoulos, An experimentl comprison of nive byesin nd keyword-bsed nti-spm Filtering with personl e-mil messges, In Proc. Of SIGIR-2000, ACM, [3] I. Androutsopoulos, J. Koutsis, K. V. Chndrinos, G. Pliours, nd C. Spyropoulos, 2000, An evlution of nive byesin ntispm Filtering, In Proc. of the Workshop on Mchine Lerning in the New Informtion Age, 9-17, [4] J. C. Bezdek, Pttern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, New York, [5] Crrers, X., nd Mrquez, L., 2000, Boosting trees for nti-spm emil filtering, Proc. of SIGIR-2000, ACM, [6] T. Fwcett, "In vivo" spm filtering: chllenge problem for KDD, ACM SIGKDD Explortions Newsletter, 5(2): , [7] R. Feldmn, Y. Aumnn, M. Fresko, O. Liphstt, B. Rosenfeld, nd Y. Schler, Text Mining vi Informtion Extrction, In Principles of Dt Mining nd Knowledge Discovery, Chpter in Book. Pges , Fig 5. Flse Positive Error Rte. Note numbers x-xis is multiplied by 10, so 9 stnds for 90%. V. CONCLUSION AND FUTURE WORK Spm Filtering is problem of gret importnce nd hs gined gret ttention in the lst decde. The problem s difficulty nd interestingness rises from the chnging nture of spm. The high ccurcy required from ny useful spm filter mkes the problem even more demnding. In this pper, the Fuzzy C-Men Clustering lgorithm is evluted s tool of building spm filter. The lgorithm ws tested with set of fetures normlized using the HDVM function. The pproch hs been testing using vrint proportion of spm emils which reflects nture of the problem. The pproch is evluted using stndrd model suggested by [6] for evluting spm filters. The results gined were promising. The flse positive error rte did not exceed 1.5% nd stbled round 0.7% when hm emils proportion is more thn 50%. We chieved between 16% to 4% for the flse negtive error rte. These results support our hypothesis regrding the suitbility of the combined pproches used. Our pproch tkes dvntge of the generliztion bility of the FCM lgorithm, extrcts representtive fetures from the dt, nd uses suitble distnce metric. Finlly, Our future work includes Optimizing Prmeters; which re the fuzzifier vlue nd number of clusters tht give the best clssifiction success rte compred with the spm filter techniques., REFERENCES [1] S. Aksoy, nd R. M. Hrlick, Feture normliztion nd likelihood-bsed similrity mesures for imge retrievl. Pttern Recognition Letters, 22(5): , [8] M. A. Herst, Untngling text dt mining. In Proceedings of the 37th Annul Meeting of the Assocition For Computtionl Linguistics on Computtionl Linguistics (College Prk, Mrylnd, June 20-26, 1999). Annul Meeting of the ACL. Assocition for Computtionl Linguistics, Morristown. [9] Messge Lbs Spm Intercepts dt, 2006, ch_dotcom_en/thret_sttistics/spm_intercepts/da_ chp. html [10] N. Nie, A. Simpser, I. Stepnikov, nd L. Zheng, Ten yers fter the birth of the internet, how do mericns use the internet in their dily lives? Technicl report, Stnford University, [11] M. Shmi, S. Dumsi, D. Heckermn, nd E. Horvitz, A byesin pproch to filtering junk e-mil. In Lerning for Text Ctegoriztion, Ppers from the 1998 Workshop, Mdison, Wisconsin, [12] G. Skkis, I. Androutsopoulos, G. Pliours, V. Krkletsis, C. Spyropoulos, nd C. Stmtopoulos, A Memory-Bsed Approch to Anti-Spm Filtering for Miling Lists, Informtion Retrievl, 6:49 73, [13] SpmAssssin Public Corpus, 2006, [14] D. R. Wilson, nd T. R. Mrtinez, Improved Heterogeneous Distnce Functions, J. Artificil Intelligence, Res. 6: , [15] E. P. Xing, A. Y. Ng, M. I. Jordn, nd S. Russell, Distnce metric lerning, with ppliction to clustering with side-informtion, In Advnces in Neurl Informtion Processing Systems 15, pges , Cmbridge, MA, MIT Press. [16] S. Nscimento, Fuzzy Clustering Vi Proportionl Membership Model, IOS Press, 2005.