Analysis of Email Fraud Detection Using WEKA Tool



Similar documents
Authentication. Authorization. Access Control. Cloud Security Concerns. Trust. Data Integrity. Unsecure Communication

ASSOCIATION RULE MINING ON WEB LOGS FOR EXTRACTING INTERESTING PATTERNS THROUGH WEKA TOOL

Data Mining in Web Search Engine Optimization and User Assisted Rank Results

K-means Clustering Technique on Search Engine Dataset using Data Mining Tool

What is Data Mining? Data Mining (Knowledge discovery in database) Data mining: Basic steps. Mining tasks. Classification: YES, NO

An Introduction to Data Mining

Spam Detection Using Customized SimHash Function

Data Mining Solutions for the Business Environment

Comparative Analysis of EM Clustering Algorithm and Density Based Clustering Algorithm Using WEKA tool.

Data Mining & Data Stream Mining Open Source Tools

Pentaho Data Mining Last Modified on January 22, 2007

Kaspersky Anti-Spam 3.0

Introduction Predictive Analytics Tools: Weka

How to Analyze Company Using Social Network?

Introduction. A. Bellaachia Page: 1

DATA MINING TOOL FOR INTEGRATED COMPLAINT MANAGEMENT SYSTEM WEKA 3.6.7

An Introduction to WEKA. As presented by PACE

Hexaware E-book on Predictive Analytics

A Proposed Algorithm for Spam Filtering s by Hash Table Approach

INVESTIGATIONS INTO EFFECTIVENESS OF GAUSSIAN AND NEAREST MEAN CLASSIFIERS FOR SPAM DETECTION

Viewpoint ediscovery Services

International Journal of Computer Science Trends and Technology (IJCST) Volume 2 Issue 3, May-Jun 2014

Clustering Technique in Data Mining for Text Documents

Web Document Clustering

RAPIDMINER FREE SOFTWARE FOR DATA MINING, ANALYTICS AND BUSINESS INTELLIGENCE. Luigi Grimaudo Database And Data Mining Research Group

DKIM Enabled Two Factor Authenticated Secure Mail Client

PSG College of Technology, Coimbatore Department of Computer & Information Sciences BSc (CT) G1 & G2 Sixth Semester PROJECT DETAILS.

Release System Administrator s Guide

Robin Sharma M.Tech, Computer Sci. Engg, RIMT-IET, Mandi Gobindgarh, Punjab, India. Principal Dr. Sushil Garg RIMT-MAEC, Mandigobindgarh Punjab, India

Problem Solving Hands-on Labware for Teaching Big Data Cybersecurity Analysis

A Security Integrated Data Storage Model for Cloud Environment

Manjeet Kaur Bhullar, Kiranbir Kaur Department of CSE, GNDU, Amritsar, Punjab, India

An intelligent Analysis of a City Crime Data Using Data Mining

Exploitation of Server Log Files of User Behavior in Order to Inform Administrator

Product Comparison List

Technical Report. The KNIME Text Processing Feature:

Anti-Spam Filter Based on Naïve Bayes, SVM, and KNN model

A QoS-Aware Web Service Selection Based on Clustering

Improving data integrity on cloud storage services

International Journal of Advanced Engineering Research and Applications (IJAERA) ISSN: Vol. 1, Issue 6, October Big Data and Hadoop

Data Mining Algorithms Part 1. Dejan Sarka

Data Mining in Personal Management

IBM SPSS Direct Marketing

Comparison of K-means and Backpropagation Data Mining Algorithms

A Comparative Study of Different Log Analyzer Tools to Analyze User Behaviors

Modeling and Design of Intelligent Agent System

Identifying Peer-to-Peer Traffic Based on Traffic Characteristics

A Survey on Web Mining From Web Server Log

not possible or was possible at a high cost for collecting the data.

N TH THIRD PARTY AUDITING FOR DATA INTEGRITY IN CLOUD. R.K.Ramesh 1, P.Vinoth Kumar 2 and R.Jegadeesan 3 ABSTRACT

Research of Postal Data mining system based on big data

How To Restore From A Backup In Sharepoint

Importance of Domain Knowledge in Web Recommender Systems

Microsoft Outlook 2010 contains a Junk Filter designed to reduce unwanted messages in your

Understanding Web personalization with Web Usage Mining and its Application: Recommender System

Dr. Anuradha et al. / International Journal on Computer Science and Engineering (IJCSE)

ManageEngine Exchange Reporter Plus :: Help Documentation WELCOME TO EXCHANGE REPORTER PLUS... 4 GETTING STARTED... 7 DASHBOARD VIEW...

Students Behavioural Analysis in an Online Learning Environment Using Data Mining

Web Log Based Analysis of User s Browsing Behavior

Web Mining. Margherita Berardi LACAM. Dipartimento di Informatica Università degli Studi di Bari

A Study of Web Log Analysis Using Clustering Techniques

International Journal of Engineering Research ISSN: & Management Technology November-2015 Volume 2, Issue-6

GFI Product Guide. GFI Archiver Evaluation Guide

Search Result Optimization using Annotators

Web Archiving and Scholarly Use of Web Archives

An Overview of Knowledge Discovery Database and Data mining Techniques

Distributed Framework for Data Mining As a Service on Private Cloud

Arti Tyagi Sunita Choudhary

In this presentation, you will be introduced to data mining and the relationship with meaningful use.

Botnet Detection Based on Degree Distributions of Node Using Data Mining Scheme

Improving Decision Making and Managing Knowledge

The online environment

Analytics on Big Data

2/24/2010 ClassApps.com

Web Advertising Personalization using Web Content Mining and Web Usage Mining Combination

Workshop 5051A: Monitoring and Troubleshooting Microsoft Exchange Server 2007

Prof. Pietro Ducange Students Tutor and Practical Classes Course of Business Intelligence

Social Media Mining. Data Mining Essentials

Association rules for improving website effectiveness: case analysis

Improving spam mail filtering using classification algorithms with discretization Filter

Quick Start Policy Patrol Mail Security 9

Just EnCase. Presented By Larry Russell CalCPA State Technology Committee May 18, 2012

Adaption of Statistical Filtering Techniques

SPATIAL DATA CLASSIFICATION AND DATA MINING

Harnessing the power of advanced analytics with IBM Netezza

A Survey of Customer Relationship Management

What is Customer Relationship Management? Customer Relationship Management Analytics. Customer Life Cycle. Objectives of CRM. Three Types of CRM

Clustering on Large Numeric Data Sets Using Hierarchical Approach Birch

Management CSCU9B2 CSCU9B2 1

Predictive time series analysis of stock prices using neural network classifier

Analyzing survey text: a brief overview

Automated distribution of SAS results Jacques Pagé, Les Services Conseils HARDY, Quebec, Qc

SPAM UNDERSTANDING & AVOIDING

Keywords Big Data; OODBMS; RDBMS; hadoop; EDM; learning analytics, data abundance.

User Experience Enhancements...2 New Mobile and Social...3. Click to Cloud Connectors...3 Media Gallery...4 Mobile...5

Transcription:

Analysis of Email Fraud Detection Using WEKA Tool Author:Tarushi Sharma, M-Tech(Information Technology), CGC Landran Mohali, Punjab,India, Co-Author:Mrs.Amanpreet Kaur (Assistant Professor), CGC Landran Mohali, Punjab,India Abstract Data mining is also being useful to give solutions for invasion finding and auditing. While data mining has several applications in protection, there are also serious privacy fears. Because of email mining, even inexperienced users can connect data and make responsive associations. Therefore we must to implement the privacy of persons while working on practical data mining. Using K-mean clustering algorithm and weka tool we implemented the technique of Email-mining. The WEKA tool calls the.eml file format into text converter and then processed the whole data into preprocessor output in form of.csv file format. The preprocessor output shows the graphical results of the processed email data. The goal of this implementation is to detect or filter the email addresses from which we get maximum emails. Index Terms Data mining, Email mining, Weka tool, K-mean clustering algorithm, Preprocessor This paper focuses on the work done to classify the textual spam E-mails using data mining techniques. Our purpose is not only to filter messages into spam and not spam, but still to divide spam messages into thematically similar groups and to analyze them, in order to define the social networks of spammers [2]. In this paper we proposed a dynamic clustering of data with simple k-means algorithm. The algorithm takes number of clusters (K) as the input from the user and the user has to mention whether the number of clusters is fixed or not. If the number of clusters fixed then it works same as K-means algorithm. II. METHODOLOGY Initially historical method of research will be used to collect the relevant data through authentic literature, books journals etc, in this research I will study a past published research thesis related to Email Mining [3]. I. INTRODUCTION Every day E-mail users receive hundreds of spam messages with a new content, from new addresses which are automatically generated by robot software. To filter spam with traditional methods as black-white lists (domains, IP addresses, mailing addresses) is almost impossible. Application of text mining methods to an E-mail can raise efficiency of a filtration of spam. Also classifying spam messages will be possible to establish thematic dependence from geographical [1]. Fig.1: Data analysis process structure ISSN: 2231-2803 http://www.ijcttjournal.org Page203

After that evaluative and analytical method will be used through structured questionnaires of two patterns: A. Primary Data: Information based on the results of analyzing data of logged emails and the research scholar will form the primary data. B. Secondary data: Information based on the literature, books, journals will form the secondary data. C. Survey method will be used for the study. D. Tool and techniques for analysis would be Reliability and validity test. Some of the data which was extracted as a single main from incoming mailbox different organizations or and personal mail boxes for sample I have collected data available for an organization in outlook express data formatted this data was converted to simple delaminated cvs format for easy data conversion [4]. III. PROPOSED WORK A. K-mean Algorithm The k-means algorithm is an evolutionary algorithm that gains its name from its method of operation. The algorithm clusters observations into k groups, where k is provided as an input parameter [5, 11]. Input: o k: the number of clusters. o D: a data set containing n objects. : A set of k clusters. Method: 1. Arbitrarily choose k objects from D as the initial cluster centres. 2. Repeat. 3. Re-assign each object to the cluster to which the object is most similar using Eq. 1, based on the mean value of the objects in the cluster. 4. Update the cluster means, i.e. calculate the mean value of the objects for each cluster. 5. Until no change. B. WEKA The workflow of WEKA would be as follows [6]: Data Pre-processing Data Mining Knowledge The supported data formats are ARFF, CSV, C4.5 and binary. Alternatively you could also import from URL or an SQL database. After loading the data, preprocessing filters could be used for adding/removing attributes, discretization, Sampling, randomizing etc. C. OutLook Express Microsoft Outlook and Microsoft Exchange use a proprietary email attachment format called Transport Neutral Encapsulation Format (TNEF) to handle formatting and other features specific to Outlook such as meeting requests [7]. An open-source project called UnDBX was also created, which seems to be successful in recovering corrupt databases. Microsoft has also released documentation which may be able to correct some non-severe problems and restore access to email messages, without resorting to third-party solutions. However, with the latest updates applied, Outlook Express now makes backup copies of DBX files prior to compaction. They are stored in the Recycle Bin. If an error occurs during compaction and messages are lost, the DBX files can be copied from the recycle bin [8, 14]. Opening or previewing the email could cause code to run without the user's knowledge or consent. Outlook Express does not correctly handle MIME, and will not display the body of signed messages inline. Users get a filled email and one ISSN: 2231-2803 http://www.ijcttjournal.org Page204

attachment (one of the message text and one of the signatures) and therefore need to open an attachment to see the email [9]. IV. RESULTS AND DISCUSSION Process of Email mining through weka works as given below with results: Fig.4 Weka explorer to select K-mean algorithm Fig.2: Weka explorer EML to text converter The eml to text converter is the first dialog box of the implementation work which consist that how many emails are to be refine [9]. Fig.5: Weka explorer showing Cluster output Fig.3: Weka explorer showing Cluster output Open dialog box appeared to select the extract emails in form of.csv form which are located in Final directory. The WEKA tool calls the.eml file format into text converter and then processed the whole data into preprocessor output in form of.csv file format [10]. The preprocessor output shows the graphical results of the processed email data. Instances Attributes No. of Iterations 7-number of live matches 6-Selected 6 attributes are Date, MessageId, CC, From, Subject, HTML 2 ISSN: 2231-2803 http://www.ijcttjournal.org Page205

Mails 3 Attributes 3 out of 7 Fig.6: Weka explorer showing Cluster output Instances 4 different and 3 identical Instances 6 (86%), means 6 out of 7 instances are identical Instances 1(14%) means 1 out of 7 instance No. of Iterations 2 The graphical results shows the count and percentage values of instances on the basis of attributes selected. Attributes are the objects of email we select to compare [13]. The percentage values show the percentage amount of instances out of total. These instances shows the frequency of email data which helps in detecting the email locations that are sending maximum number of spam mails or unwanted mails. These are used to do email mining and filtering of emails [12]. Figure 4.9: Weka explorer showing Preprocessor output Mails 3 Attributes 6 out of 7 Instances 2 different and 5 identical In second case we selected all 6 attributes and then processed the data of three mails. It resulted into 2 different instances and 5 similar instances of,. And all those are shown in graphs of different colors. Fig.7: Weka explorer showing preprocessor output ISSN: 2231-2803 http://www.ijcttjournal.org Page206

V. CONCLUSION AND FUTURE WORK Coming to conclusion the main theme of this work is to detect or filter the emails to analyze the particular email address from which the number of receiving emails is maximized. A new idea that seems promising is Semantic Email has been proposed. The technique used is resultant to more secured feature to detect spam mails and their source address. The software implemented could be used to detect those methods and integrate them into useful and accurate email-mining which will let people take back control of their mailboxes. Future recommendations have multiple choices likewise the present work is done for single email-id and the data processed is done for single email-id. But the work could be done by taking multiple email-ids. Moreover this filter evaluation technique is processed by taking just six attributes. But we can take much more number of attributes to improve the filtered results. The efficiency of text converter which converts the outlook express file format into WEKA tool file formats. As in the present work the eclipse code generated perform the text conversion from.eml file format to.csv file format. But in further recommendations the file conversion could be perform for multiple file formats of WEKA tool like.arss. In this work we have chosen K-mean algorithm to process the clusters of data in WEKA tool. But in future work we can use more efficient existing or proposed algorithm to process the email data. [4] Ding Zhou et al and Ya Zhang, "Towards Discovering Organizational Structure from Email Corpus". (2005) Fourth International Conference on Machine Learning and Application. [5] Giuseppe Carenini, Raymond T. Ng and Xiaodong Zhou, "Scalable Discovery of Hidden Emails from Large Folders". Department of Computer Science, University of British Columbia, Canada. [6] Hung-Ching Chen el al, "Discover The Power of Social and Hidden Curriculum to Decision Making: Experiments with Enron Email and Movie Newsgroups". Sixth International Conference on Machine Learning and Applications. [7] Christian Bird, Alex Gourley, Prem Devanbu, Michael Gertz and Anand Swaminathan, "Mining Email Social Networks". (May 22-23, 2006). Dept. of Computer Science, University of California, Davis. [8] Rong Qian, Wei Zhang, Bingru Yang, "Detect community structure from the Enron Email Corpus Based on Link Mining".(2006) Sixth International Conference on Intelligent Systems Design and Applications. [9] Deepak P, Dinesh Garg and Virendra K Varshney, "Analysis of Enron Email Threads and Quantification of Employee Responsiveness". IBM India Research Lab, Bangalore - 560 071, India. [10] Ziv Bar-Yossef et al, "Cluster Ranking with an Application to Mining Mailbox Networks". (2006) Sixth International Conference on Data Mining. [11] Hua Li et al, "Adding Semantics to Email Clustering". (2006) Sixth International Conference on Data Mining. [12] http://libguides.sjsu.edu/a-z - The SJPL library database [13] Shlomo Hershkop et al, "Email Mining Toolkit Technical Manual". Version 3.6.8 -June 2006 [14] http://www.apachefriends.org/en/xampp-windows.html, XAMPP source. VI. REFERENCES [1] A. Anderson, M. Corney, O. de Vel, and G. Mohay."Identifying the Authors of Suspect E-mail". Communications of the ACM, 2001. [2] Shlomo Hershkop, Ke Wang, Weijen Lee, Olivier Nimeskern, German Creamer, and Ryan Rowe, "Email Mining Toolkit Technical Manual". (June 2006) Department of Computer Science Columbia University. [3] Bron, C. and J. Kerbosch. "Algorithm 457: Finding all cliques of an undirected graph." (1973). I am Tarushi Sharma persuing M-Tech from CGC Landran,Mohali(2011-13) in Information Technology.I did my B-Tech from CCS University Campus,Meerut(2007-11) in Information Technology. ISSN: 2231-2803 http://www.ijcttjournal.org Page207