Spam Filter: VSM based Intelligent Fuzzy Decision Maker

IJCST Vo l. 1, Is s u e 1, Se p te m b e r 2010 ISSN : 0976-8491(Online Spam Filter: VSM based Intelligent Fuzzy Decision Maker Dr. Sonia YMCA University of Science and Technology, Faridabad, India E-mail : soniabansal2@yahoo.com Abstract : Over the internet, email becomes a most popular means for communication. The received unsolicited and undesired emails are called spam and junk mails, which arises day by day. To filter the spam from the legitimate emails, classification approach using context based techniques are proposed. In this talk, an efficient and effective classification approach to detect the spam email from the legitimate will be discussed. In the first phase, A Vector Space Model (VSM based classification method is developed accordingly. In which the input mail is converted into matrix and on the basis of term frequency. Then similarity coefficient has been computed. In the second phase, the intelligent fuzzy decision maker has been developed that categorized the email for user decision. The real legitimate and real spam can be filtered by using the fuzzy decision maker. Keywords : Spam, Spam Filter, Vector Space Model, Fuzzy Decision Maker. I. Introduction The junk emails are received daily in inbox and they are really a headache to most of the people. The precious time of the employee is wasted to browsing through the spam emails. Spam filtering is done to filter those junk emails from your inbox so that you save a lot of time [15,18,19]. With spam filtering your receive only the genuine emails that are intended for your reading spam is used here to mean receiving unsolicited electronic mail, usually advertising some product, service, business, scheme, website, etc. spamming via other means is a different problem which we do not tackle here [4,6,8,15]. The spam problem is complex system and should be dealt with developing strategies to holistically interact with it. Such Strategies must embrace both technical and legal realities simultaneously in order to be successful [3,9]. A spam filter is a program that is used to detect unsolicited and unwanted email and prevent those messages from getting to a user s inbox. Like other types of filtering programs, a spam filter looks for certain criteria on which it bases judgments [1]. For example, the simplest and earliest versions (such as the one available with Microsoft s Hotmail can be set to watch for particular words in the subject line of messages and to exclude these from the user s inbox [7,10,11,13]. This method is not especially effective, too often omitting perfectly legitimate messages. Spam Filtering somewhat similar to Information Retrieval (IR where the spam are retrieved from the mail box of user [4,17]. One of the information retrieval methods have been used to filter the spam is Vector Space Model (VSM.The vector-space models for information retrieval are just one subclass of retrieval techniques that have been studied in recent years. To efficiently satisfy the user query requirements in information retrieval, a query optimization algorithm for spam retrieval is proposed. In this thesis we propose Vector Space Model (VSM Spam Filtering technique using text based vector space model. This method constructs the spam detection model by contents of various kind of mail and finds spam more efficiently. The algorithm has been proposed that can be applied on bulk of the messages to detect the spam mails. The architecture of VSM Spam Filter is also proposed. Moreover an efficient classification engine is defined to explain the filtering process. The vector space model for information retrieval is just one subclass of retrieval techniques. The taxonomy provided in labels the class of techniques that resemble vector-space models formal, feature-based, individual, partial match retrieval techniques since they typically rely on an underlying, formal mathematical model for retrieval model, the mail documents as sets of terms that can be individually weighted and manipulated, perform queries by comparing the representation of the query to the representation of each mail document in the space, and can retrieve spam mail documents that don t necessarily contain one of the search terms [35,36]. In the first phase of this paper Vector Space Model (VSM method on the basis of similarity has been defined. In which the input mail is converted into matrix and on the basis of term frequency the similarity coefficient has been computed. In the second phase, the intelligent fuzzy decision maker has been developed that categorized the email for user decision. The real legitimate and real spam can be filtered by using the fuzzy decision maker. II. Literature Survey Spam mails vary significantly in content and they roughly belong to the following categories: money making scams, fat loss, improve business, sexually explicit, make friends, service provider advertisement, etc., [5,14]. Among the proposed methods, much interest has focused on the machine learning techniques in spam filtering. They include rule learning [23,34,38], Naive Bayes [24,37], decision trees [39], support vector machines [2,21,28] or combinations of different learners [12,22]. The common concept of these approaches is that they do not require specifying any rules explicitly to filter out spam mails. Instead, a set of training samples (preclassified e-mails is needed. Sahami et al. [30,37] employed Bayesian classification technique to filter junk e-mails. By making use of the extensible framework of Bayesian modeling, they can not only employ traditional document classification techniques based on the text of e-mail, but they can also easily incorporate domain knowledge to aim at filtering spam e-mails. Androutsopoulos et al. [23 25] presented a series of papers that extended the Naı ve Bayes (NB filter proposed by Sahami et al. [37], by investigating the effect of different number of features and training-set sizes on the filter s performance. Drucker et al. [21] used support vector machine (SVM for classifying e-mails according to their contents and compared its performance with Ripper, Rocchio, and boosting decision trees. Sakkis et.al. [20] proposed a memory based approach to spam filtering for mailing list. Zhang and Yao [32] presented a maximum entropy based approach to junk mail filtering. They showed that comparing two Naïve Bayes, the maximum entropy method reduced comparable or better results and domainspecific features provided by Spam Assassin [16].The fuzzy process can be include in order to detect the spam in which fuzzification, membership function and inference rule includes [27,33]. 48 International Journal of Computer Science and Technology

ISSN : 0976-8491(Online III. Vector Space Model Filter Technique The vector space model procedure can be divided in to three stages. The first stage is the document indexing where content bearing terms are extracted from the document text [26]. The second stage is the weighting of the indexed terms to enhance retrieval of document relevant to the user. The last stage ranks the document with respect to the query according to a similarity measure. E-mail Collection IJCST Vo l. 1, Is s u e 1, Se p te m b e r 2010 the spam mails. A. VSM based Spam Filter Algorithm wi 1 wi4 w1 j w i j Split e-mail into matrix. A collection of n message can be represented in the vector space model by a term-message matrix. An entry in the matrix corresponds to the weight of a term in the message; zero means the term has no significance in the message or it simply doesn t exist in the message. Term Weights: Term Frequency More frequent terms in a message are more important, i.e. more indicative of the topic. f ij = frequency of term i in document j May want to normalize term frequency (tf by dividing by the frequency of the most common term in the message: tf ij = f ij / maxi{f ij } Term Weights: Inverse Message Frequency Terms that appear in many different messages are less indicative of overall topic. m fi = message frequency of term i tf = number of messages containing term i imfi = inverse message frequency of term i, im fi = log2 (N/ m fi (N: total number of mail documents An indication of a term s discrimination power. Log used to dampen the effect relative to tf. TF-IDF Weighting A typical combined term importance indicator is tf-imf weighting: w ij = t fi im fi = tf ij log2 (N / m fi A term occurring frequently in the mail document but rarely in the rest of the collection is given high weight Many other ways of determining term weights have been proposed. Experimentally, tf-imf has been found to work well. Similarity Measure The Similarity Coefficient calculation can be measure by using the formula SC(Q,M = imf tf ij IV. Proposed Work To efficiently satisfy the user query requirements in information retrieval, a query optimization algorithm for spam retrieval is proposed. Vector Space Model (VSM Spam Filtering technique is using text based vector space model. This method constructs the spam detection model by contents of various kind of mail and finds spam more efficiently. The algorithm has been proposed that can be applied on bulk of the messages to detect See which one exist in the inverted index and calculate the similarity coefficient. o Need number of messages for spam detection o Need term frequency of each word in a message TF(m,t = 0 if n(m,t=0 = 1+log(1+(n(m,t otherwise o Calculate inverse message frequency M 1 + M t IMF = log where M is message collection and M t is the set of message containing t term. Sort it alphabetically o Combine TF and IMF into complete vector space model. The coordinate of message m in axis t is given by mt = TF(m,t. IMF(t o Calculation of Similarity Coefficient SC(Q,M = j= 1 t w m q i j where M( mi1 ----------m it collection of messages with t term and Q(wq1,wq2---------wqm terms found on the query o Parse each weight into fuzzy decision maker Using these weights make the rule sets for intelligent fuzzy decision o Result on the basis of decision maker Choice of user for proper action Do it for n number Certain optimizations have been implemented in addition to this algorithm. They are as follows: o Using Rocchio s approach used vector space model to find the more relevance message spam document and used as International Journal of Computer Science and Technology 49

IJCST Vo l. 1, Is s u e 1, Se p te m b e r 2010 relevant feedback. o Using a stop word list of most common word hence speeding up the process of classification and learning as used in Goggle search engine[3] B. VSM based Spam Filter Example Consider a case of query and message collection consisting of three mail Q: Free ticket click M1: Guaranty of free shirt in a mall M2: Delivery of ticket movie in a ticket click M3: Guaranty of free movie in a click In the collection there are three mail documents n=3. If the term appears in only one of the three mail document, its imf is log(n/mfj=log(3/1=0.477. If the term appears in two of the three mail document, its imf is log(2/1=0.176 and it appears in all the three documents it has an imf= log(3/3=0. The imf for the terms in the three mail documents is given below 1 imf a = 0 2 imf click 3 imf delievry = 0.477 4 imf free 5 imf guaranty 6 imf in = 0 7 imf mall = 0.477 8 imf movie 9 imf of = 0 10 imf shirt = 0.477 11 imf ticket Mail document vectors now be constructed. Since eleven terms appear in the mail document collection, an eleven-dimensional mail document vector is constructed. The alphabetical ordering given above is used to construct the mail document vector. The weight for the term i in the vector j is computed as the imf tf ij. Mail document vectors now be constructed. Since eleven terms appear in the mail document collection, an eleven-dimensional mail document vector is constructed. The alphabetical ordering given above is used to construct the mail document vector. The weight for the term i in the vector j is computed as the imf tf ij. The mail document vectors are SC(Q,M1 = (0(0 + (0(0.176 + (0(0 + (0.176(0.176 + (0 (0.477 + (0(0 + (0(0 + (0(0.477 = (0.176 2 = 0.031 Similarly, SC(Q,M2 = (0.352(0.477 + (0.176 2 = 0.519 SC(Q,M3 = (0.176 2 + (0.176 2 = 0.062 Hence the mail document M2 having weight more than M1 and M3. On the basis of similarity coefficient the fuzzy decision maker differentiate the real legitimate. Table 1. Terms Appear in the Collection C. The Architecture VSM based Spam Filter The architecture of the VSM Spam Filter is defined in Fig 1. Fig. 1. The Deployment Diagram of VSM Spam Filter Bulk of e-mails are received Pass from the pop server Go into the VSM Spam Filter The result of the spam filter is sent to the intelligent decision maker User can choose the option Fig. 2. VSM Spam Filter Architecture 1. VSM Spam Filter is shown in Fig. 2. a. Letter is received. b. It is used as plain text including body and header. c. The letter is represented in to matrix d. Term frequency calculated e. The inverse message frequency is calculated. f. The similarity coefficient (SC calculation is done using the term frequency and inverse message frequency.. Fig. 3. Intelligent Decision Maker Architecture ISSN : 0976-8491(Online 2. The Intelligent Fuzzy Decision Maker is described in Fig. 3. a. The result of SC calculation used as input. b. The fuzzification is done. c. The email is categorized. 50 International Journal of Computer Science and Technology

ISSN : 0976-8491(Online d. The rule set is constructed to take final decision about the email. e. Final out put for user choice. D. The VSM based Filter Classification Engine The classification engine used to filter the spam by using the vector space model is defined in Fig. 4.The various part of classification engine is explained in detail. Emails Data Source: The email data source represents the input. The source should be a source of text such as an article, or an email for that matter Matrix Conversion: The matrix conversion tokenizes the text, into words, passes those words onto the inverse document calculation engine to further process. Inverse Message Frequency Calculation: The inverse term frequency calculation to be done by the following equation: imf = log(m/m fj Where m is the total number of mail documents and m fj number of documents which contain term or word( tj Sorting of Inverse Message Frequency: Message vectors can now be constructed and all the terms appear in the mail document collection. The alphabetical order sorting is done to construct the mail document vector. Similarity Coefficient (weight Calculation: Calculation of the weighting factor (m for a term in the document is defined as a combination of term frequency (tf, and inverse mail document frequency ( imf. To compute the value of the j th entry in the vector corresponding to mail document i, the following equation is used: m ij = tf imf A simple similarity coefficient (SC between the query Q and a mail Mi is define by the product of the two vectors. Since a query vector is similar in length to the mail vector, this same measure is often used to compute the similarity between the two documents. IJCST Vo l. 1, Is s u e 1, Se p te m b e r 2010 assigned a weight, it is time to classify. The deployment of fuzzy decision maker is shown in Fig. 5. Fuzzy decision maker works on the soft computing and by taking decision using fuzzy rule set is very real. The fuzzy logic provides a convenient way of converting existing results into fuzzy logic rules and email is categorized in the form of legitimate, unlike, like and spam. The user can take decision corresponds to these emails in the form of no check, essential check, rigorous check and discard respectively. The input values related to each message are translated into linguistic concept. As in the previous work, emails are defined into two-form spam and legitimate. In this work the emails are categorized on the basis of weight. The fuzzy decision maker categorized email into four different forms like legitimate, unlike, like and spam according to their weight. The legitimate is defined in the range of 0 to 0.4, unlike spam having the range from 0.2 to 0.7, while like spam is in the 0.5 to 0.7 and the definitely spam is defined 0.8 onwards. On the basis of these observation, the user can select no check, essential check, rigorous check and discard respectively. Fig. 5. Deployment of Fuzzy Decision Maker Fig. 4. VSM Classification Engine wqj 1 SC(Q,Mi= t j= 1 m i Where a vector M(mi 1,mi 2..,m it of size t for each mail. The vectors are filled with the term weights. Similarly, a vector Q(wq 1, wq 2,.,wq t is constructed for terms found in the query. Intelligent Fuzzy Decision Maker: Once all token have been V. Concluding Remarks As the email becomes a popular means for communication over the internet, the problem receiving unsolicited and undesired emails, called spam or junk. To filter spam from emails, automatic classification approaches using text mining technique has been proposed. Vector space model is popular retrieval model and used in the detection of spam. The main advantages of vector space spam filter are, its term weighting scheme improves filtration performance, its partial matching strategies allows retrieval of messages that approximate the query conditions, its weight scheme sorts the messages according to their degree of similarity of the query. Vector space model is fairly cheap to compute and yields decent effectiveness. Moreover it is very popular and smart enough so it is most commonly used. We proposed Vector Space Model (VSM Spam Filtering technique using text based vector space model. This method constructs the spam detection model by contents of various kind of mail and finds spam more efficiently. This chapter deals with the calculation of term weight calculation with an example. The algorithm has been proposed that can be applied on bulk of the messages to detect the spam mails. The architecture of VSM Spam Filter is also proposed. Moreover an efficient classification engine is defined to explain the filtering process. International Journal of Computer Science and Technology 51

IJCST Vo l. 1, Is s u e 1, Se p te m b e r 2010 The main purpose of considering the vector space model in the detection of spam is that VSM spam filter, it is assumed that terms are independent. The measure is important as it is used by a retrieval system to identify which mail documents are displayed to the user as spam and not spam. More over vector space spam filtering model can be applied to all featured i.e. header, subject and body. This will give the better performance than any other technique. The email is categories into four parts like legitimate, unlikely, likely and spam. The legitimate is useful mail, which the user will not check it for spam. The unlikely mail are those email which needs essential check, not for spam but in case of likely, the rigorous check has been done to take decision that this mail may be spam or not. Finally when the weight is very high the discard the mail indicates that the mail is spam. For this a intelligent fuzzy decision maker is created and achieve higher accuracy by using both the model together. First crisp value is converted into fuzzy by fuzzification. The fuzzification gives the possibility to define e mail in the form of legitimate, unlikely, likely and spam. This fuzzy value is used as input to get the final result for the user choice by using the decision maker. With this method not only the emails are categorized but the characteristics of the output are used for user choice. This method is used to differentiate the real legitimate and real spam by combining the logic model with vector space model through use of decision maker, which is easily usable and give higher accuracy. VI. Acknowledgement This work was supported by Dr. A. K. Sharma and Dr. C.K. Nagpal. The author is very much thankful to Mr. Niranjan Kumar for his support. The author would like to thank anonymous referees for their valuable comments and suggestions. References [1]. Graham P., A Plan for Spam, [Online] Available: http:// www.paulgraham.com/spam.html. [2]. Kołcz A., Alspector J., SVM-based filtering of e-mail spam with content-specific misclassification costs, Proc. of TextDM 01 Workshop on Text Mining, 2001. [3]. Berger A., Pietra S.D., and Pietra VD. A maximum entropy approach to Natural Language. Computational Linguistics,Vol. 22(2, pp.39-71(1996. [4]. Dudani A. The distance-weighted k-nearest neighbor rule. IEEE Transactions on Systems, Man and Cybernetics, Vol. 6(4, pp.325-327(1976. [5]. Schwatz A.and S. Garfinbel, Stopping Spam Stamping out Unwanted emails and news posting, publisher, O Reilly, (1998. [6]. Paul N.C. and Monitor C.S., New strategies aimed at blocking spam e-mail, [online] Available: http://newsobserver.com/24hour-/ technology/ story/655215p-4921708c.html. [7]. Mueller S.H., Ant-Spam Abuse site, [Online] Available: http:/spam.abuse.net/. [8]. McGlynnaria T. [online] Available: arizona.edu/courses/ tutorials/class/html/class.html. [9]. Clement T., CAUCE, How do you define Spam?, [Online] Available: http://www.cauce.org/about/faq.shtml [10]. Williams C., D. Ferris, The cost of spam false positives, private communication, August 2003. [11]. Wolpert D.. Stacked Generalization. Neural Networks, Vol. 5 (2, pp.241-260(1992. ISSN : 0976-8491(Online [12]. Espiner T., Demand for Anti-Spam Products to increase. UK. Zdnet, June 2005. [13]..net basic describer of the way Spam is handled by this organization- http:/ www.everyone.net/spam.html. [14]. Smadja F., Tumblin H., Automatic spam detection as a text classification task, in: Proc. of Workshop on Operational Text Classification Systems, 2002. [15]. Falk J.D., Fight Spam on Internet What is Spam. [Online] Available: http://spam.abuse.net/overview/ whatsspam.shtml. [16]. Fuzzy Logic Toolbox User s Guide, The Math Works Inc. (2004. [17]. lir G.J. and Yuan B. Fuzzy Sets and Fuzzy Logic, Prentice Hall P T R, Upper Saddle River, New Jersey, USA (1995 [18]. Sakkis G., Androutsopoulos I., Paliouras G., Karkaletsis V., Spyropoulous C., and Stamatopoulos P. Stacking Classifiers for Anti-Spam Filtering of E-mail. Proceedings of the 6th Conference on Empirical Methods in Natural Language Processing, (2001, pp. 44-50. [19]. Iooannidis J.. Fighting Spam by Encapsulating Policy in Email Addressed, 10th Network and Distributed System Security Symposium (2003 [20]. Sakkis G., Androutsopoulos I., Paliouras G., Karkaletsis V., Spyropoulous C., and Stamatopoulos C.. A memorybased approach to Anti-Spam Filtering for mailing lists. Information Retrieval, 6th, pp.49-73(2003. [21]. Drucker H, Wu D., Vapnik V.N., Support vector machines for spam categorization, IEEE Trans. Neural Netw. Vol. 10 (No. 5, pp.1048 1054 (1999. [22]. Tony Clement, http://www.cauce.org/about/problem. shtml. [23]. Androutsopoulos I., Paliouras G., Karkaletsis V., Sakkis G., Spyropoulos C.D., Stamatopoulos P., Learning to filter spam e-mail: a comparison of a Naı ve Bayesian and a memory-based approach, in: Proc. of the workshop: Machine Learning and Textual Information Access,, 1 13(2000. [24]. Androutsopoulos I., Koutsias J., Chandrinos K., Paliouras G., Spyropoulos C.D., An evaluation of naive bayesian anti-spam filtering, in: Proc. of the Workshop on Machine Learning in the New Information Age: 11th European Conference on Machine Learning,, pp. 9 17(2000. [25]. Androutsopoulos I., Koutsias J., Chandrinos K., Paliouras G., Spyropoulos C.D., An experimental comparison of naive bayesian and keyword-based anti-spam filtering with encrypted personal e-mail messages, in: Proc. of the 23rd Annual International ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 160 167 (2000. [26]. Information Retrieval by Mitra S. [27]. Baldwin J. F., Fuzzy logic and fuzzy reasoning, in Fuzzy Reasoning and Its Applications, Mamdani E.H. and Gaines B.R. (eds., London: Academic Press (1981. [28]. Provost J., Naïve-Bayes vs. rule-learning in classification of email, Technical-report, University of Texas at Austin, 1999. [29]. Carpinter J., Hunt R., Tighting the Net: A Review of Current and Next Generation Spam Filtering Tools, Computer & Security Vol. 25 (2006 566-578. [30]. Praed J., Latest trends in the Legal Fighter Against Spammers, Spam Conference 2004. [31]. Kojima H., Chung C., Westen C., Strategy on the landslide 52 International Journal of Computer Science and Technology

ISSN : 0976-8491(Online IJCST Vo l. 1, Is s u e 1, Se p te m b e r 2010 type analysis based on the expert knowledge and the quantitative prediction model, ISPRS, Amsterdam 2000. [32]. Zhang L. and T-Yao. Filtering Junk Mail with a maximum Entropy Model. In Proceeding of the 20th International Conferences on Computer Processing of Oriental Languages, 446-453(2003. [33]. Zadeh L.A., Fuzzy algorithms, Info. & Ctl., Vol. 12,, pp. 94-102,(1968. [34]. Zadeh L.A., Making computers think like people, IEEE. Spectrum, 8, pp.26-32, (1984. [35]. Zadeh L.A., Fuzzy Sets, Information and Control, 1965. [36]. Zadeh L.A., Outline of A New Approach to the Analysis of Complex Systems and Decision Processes, 1973. [37]. Sahami M., Dumais S., Heckerman D., Horvitz E., A Bayesian approach to filtering junk e-mail, in: Learning for Text Categorization Papers from the AAAI Workshop, pp. 55 62(1998. [38]. Cohen W.W., Learning rules that classify e-mail, in: Proc. of AAAI Spring Symposium on Machine Learning in Information Access, pp. 18 25 (1996. [39]. Carreras X., L. Ma rquez, Boosting trees for anti-spam email filtering, in: Proc. of fourth Int l Conf. on Recent Advances in Natural Language Processing, pp. 58 64(2001. Dr. Sonia is presently working as an Assistant Professor in YMCA University of Science and Technology, Faridabad, Haryana, India. She has done her M.Sc. and PhD from Jamia Millia Islamia, New Delhi, India. She has also completed her M.C.A. and M.Tech. (Computer Engineering degrees from MDU, Rohtak, Haryana, India. Her research interest includes Web Mining, Data Mining and Spam Filtering. International Journal of Computer Science and Technology 53