A DATA MINING APPROACH FOR CLASSIFICATION OF HEART DISEASE DATASET USING NEURAL NETWORK

Transcription

1 A DATA MINING APPROACH FOR CLASSIFICATION OF HEART DISEASE DATASET USING NEURAL NETWORK 1 Miss. Manjusha B. Wadhonkar, Prof. P. A. Tijare 2, Prof. S. N. Sawalkar 3 1 M.E Computer Engineering,Computer Science and Engineering Department. Sipna College of Engineering & Technology, Amaravati 2 Associate Professor, Computer science and Engineering Department Sipna College of Engineering & Technology, Amaravati 3 Assistant Professor, Computer Science and Engineering Department. Sipna College of Engineering & technology, Amaravati ABSTRACT Data mining is the process of automating information discovery. ANN is widely used data mining method to extract pattern. Classification is one of the important data mining techniques for classifying given set of input data. In this experiment classification of heart disease dataset is done with the use of Cleveland Heart Disease Dataset. Classification is carried out using neural network classifier MLP.In this experiment performance measures are compared with chosen optimal parameter of MLP neural network, when it is trained and tested over cross validation, the training percentage of 98±0.5 %, testing percentage of 98±1.5% and 97± 1.2% overall accuracy, sensitivity 95±0.5%,specificity 100% are achieved. It shows the consistent performance of MLP neural network as compare to other models. In this work heart disease dataset is classified using 13 input attributes as well as by using 16 inputs attributes. The accuracy difference between 13 attributes and 16 attributes in training dada is 1.67 % and in testing data is 3.7% and in overall accuracy is.1.47%.the results obtained in this experiment shows the efficiency and accuracy of MLP NN. Keywords: Heart Disease dataset, MLP, Neural Network, Back-Propagation Algorithm, Classification, PE, Knowledge Data Discovery 1. INTRODUCTION Data mining is an important step in discovery of knowledge from large dataset. In recent years data mining has found its significance in every field including healthcare [1]. A major challenge for healthcare organizations is the provision of quality services at affordable costs. Services imply diagnosing patients correctly and administrating effective treatment [2]. Medical data comprises number of tests essential to diagnose to a particular disease. Integration of clinical decision support with computer-based patient records could reduce medical errors, increase patient safety, decrease unwanted practice variation, and improve patient outcome [15]. Global burden of disease estimates for 2001 by World Bank Country Groups shows severity statistics indicated in year 2001 is 25.2 % for India and from literature survey now it has increased to 46% [3]. Effective and efficient automated heart disease classification systems can be beneficial in healthcare sector for heart disease classification. The aim of our work is to introduce a classification approach using Multilayer Perceptron (MLP) with Back Propagation learning algorithm with heart disease dataset. Classification is one of the important techniques of data mining [5]. Classification is the processing of finding a set of models or functions which describe and distinguish data classes or concepts. In classification, inputs are given a set of data, called training set, where each record consists of several fields or attributes. One of the attributes, called the classifying attribute, indicate the class to which each dataset belong. The aim of the classification is to build a model of the classifying attribute based upon the other attributes which are not from the training dataset [4].Artificial neural network is widely used technique for extraction of patterns in data mining. ANN has some advantages such as it automatically allow arbitrary nonlinear relations between the independent and dependent variables, and allows all possible interactions between the dependent variables, due to above said advantages of ANN the use of neural network technique is adopted for the classification of dataset [5]. Parallel processing approach is implemented at each node to increase the efficiency of classification. Volume 4, Issue 5, May 2015 Page 426

2 2. LITERATURE SURVE AND RELATED WORK Integration of clinical decision support with computer-based patient records could reduce medical errors, increase patient safety, decrease unwanted practice variation, and improve patient outcome [15]. Global burden of disease estimates for 2001 by World Bank Country Groups shows severity statistics indicated in year 2001 is 25.2 % for India and from literature survey now it has increased to 46% [6]. In spite of the rapid development of pathological research more than 60,000 people die suddenly each year in India due to cardiovascular diseases [5]. A number of techniques have been used for identification or prediction of heart diseases such as waveform analysis, time frequency analysis, complexity measures, Neuro Fuzzy RBF NN, but it has been observed that classification accuracies were only up to 79 % [5]. Classification of heart disease dataset using ANN with feature selection gives only 80% result with these techniques and still have enough scope in improving it by choosing appropriate NN model [7]. The above analysis shows that Neural Network with 8 input attribute and 13 input attributes have shown the approximate accuracy of 81% so far [8]. 3. DATASET FOR THE CLASSIFUCATION OF HEART DISEASE Data for the classification of heart disease dataset is obtained from four different datasets of UCI [5], centre for machine learning and intelligent system.this database contains total 76 attributes, but for classification,a subset of 17 of them namely Age(in years),sex, Chest Pain type, Resting blood Pressure, Serum cholesterol, fasting blood sugar, Resting ECG, Maximum heart rate achieved, exercise induced angina, ST depression induced by exercise relative to rest, The slope of the peak exercise ST segment, Number of major vessels,number of cigarettes per day, years as a smoker and fam_hisory and last feature is output based on classification of heart disease. Table 1 shows name of datasets and number of their instances [9]. Table 1: Database Names and their Instances [5] Name of Database Number instances Cleveland 303 Hungarian 294 Switzerland 123 Long Beach V.A 200 of The goal or output field is a five bit value which will represent five different classes as class 0-normal person, class 1- first stroke, class 2- second stroke and class 3- third stroke and class 4-end of life. 4. DESIGN OF CLASSIFICATION SYSTEM The design of the neural network mainly consist of topology(i.e. arrangement of PE s, connections, and patterns in the neural network) and architecture of the network[10].for the classification of Heart disease dataset using 13 input attributes and one output testing results gives maximum 90.6% accuracy for single layer and 94% for multilayer feed forward network [11]. To increase the accuracy of classification of heart disease dataset, in our system three other input attributes as number of cigarettes per day, years as a smoker and fam_history are used which increases the risk of cardiovascular disease. Thus this system is an attempt to introduce a classification approach using multilayer Perceptron (MLP) with back-propagation algorithm which includes 16 input attributes and an output attribute. An output attribute is a resultant class to which patient belong and is displayed as a combination of five bits such as [ ] represent class 0, [ ] for class 1 and so on. Attribute values for cig-per-day, yrs and family-history are tabulated in below table 2. Table 2: Proposed attribute values Name Description Range Cig-per-day Value Continuous Yrs Value Continuous Family history 1=True,0=Fals 1,0 e Volume 4, Issue 5, May 2015 Page 427

3 4.1 Multilayer Perceptron Neural Network For more complex decision function the inputs each with its own set of weights and threshold are fed into a number of perceptions nodes [5].The output of these input nodes are given as an output to another layer of nodes. Output of final layer of nodes is the output of the network. Such type of network is termed as MLP [7].The layers of nodes whose inputs and outputs are seen only by other nodes are termed hidden [8]. Back_propagation learning algorithm with supervised learning methods is used to compute the connection weights. There are different variants of back_propagation algorithm in the literature [12]-[13]. 4.2 Data Flow for the Classification of Heart Disease Dataset The workflow for the classification of dataset is as shown in figure 1(a) and 1(b) which provides brief description of fundamental steps that should be followed to apply ANNs for the classification of heart disease dataset Start Collect patient Data from Database Preprocessing Start New Patient Data Acquisition ANN Training using Back Propagation Algorithm Preprocessing No Are Test Result OK Yes Trained and Verified ANN Processing for Prediction of Class Predicted Result Stop Stop Figure 1 (a): Training Procedure for Classification of Heart Disease Dataset Using MLP Network [5]. Figure 1 (b): Testing of ANN based Classification of heart Disease dataset for new patient data [5] Data Collection Neural network is trained using Cleveland dataset of example cases. This dataset is nothing but records of patient s stored in a database. Database contains number of reliable examples to be given as an input to the training network Pre-processing Data in the training dataset must be pre-processed before the evaluation by the neural network. Data to be given as input are scaled within the interval (0, 1) because the interference function used is logistic one. During pre-processing Cleveland dataset, 11 records contain missing attribute values that should be removed from the dataset to improve the classification performance. Thus total 272 records are given as an input to the neural network. Volume 4, Issue 5, May 2015 Page 428

4 4.2.3 Training & verification using ANN The neural network is trained with Heart Diseases database by using feed forward neural network model and backpropagation learning algorithm with momentum and variable learning rate. The input layer of the network consists of 16 neurons to represent each attribute as the database consists of 16 attributes. Several neural networks are constructed with single hidden layers network and trained with heart disease dataset. A selection of maximum number of epochs is provided prior to training within which the training is expected to converge. The convergence is said to be achieved when the error between the output generated by the trained network and the actual output from the database matches within a certain error limit preset before the training. If a convergence is not achieved then training with new network configuration (i.e. hidden neuron count) is carried out. Below figure 2(a) and 2(b) shows training graph and the error graph which depicts the actual output, predicted output by the trained neural network and the absolute error difference between actual and predicted output. Figure 2(a): Training graph converged at 704 epochs. X axis=number of epoch, Y axis=error difference Figure 2(b): Error graph converged at 704 epochs. X axis=number of instances, Y axis=scaled output value and Error difference Verification Once a convergence is achieved the ANN is declared to be trained and its verification is initiated which normally is similar to the verification carried out during training by comparing the predicted outputs of the ANN with the actual ones, only difference being the dataset used this time is different from the one used in training. Once this verification results match then the ANN is declared as trained and verified for application purpose. Periodic verification of ANN and retraining if verification fails is a normal process with the ANNs Testing Once an ANN is declared to be trained and verified it is usable for application to the classification problem. In this phase it is provided with new user s heart disease data and asked to classify. The results are used as correctly generated. 4.3 Architecture for the classification of heart Disease Dataset The architecture for the classification of heart disease dataset is as shown in figure 3. Initially Cleveland database (76 attributes) and its subset of (16) have been acquired and a Database structure for the system is being set into place for the loading of the Database as well as Help presentation on the database. A scalable approach is used with the use of Database module which uses two scripts labeled as Database Info and Database Load. The first one provides the information about the Database features/attributes and their naming, the second one is provided for loading the Volume 4, Issue 5, May 2015 Page 429

5 database in memory for processing. GUI for the Classification of Heart Disease Dataset Using MLP with Back Propagation Algorithm Database Modules (Load& Info.) Preprocessing Modules Training Module Verification Module Application Module Heart Database Training result Verification Classification Result Figure 3: Architecture of the System for classification of Heart Disease dataset using MLP [5] Training Module:-ANN is trained by using MLP with Back-propagation learning algorithm. Training Result: - Are predicted results, obtained by summing the results of inputs with adjusted weights. Verification Module:-In this module predicted output of ANN is compared with actual output. Verification Result: - Once this verification results match then the ANN is declared as trained and verified for application purpose. Application Module: - Once an ANN is declared to be trained and verified weights from input to hidden layer and hidden to output layer are stored and reloaded for application to the classification problem. In this phase it is provided with new patient s heart disease data as an input and display result as a class to which patient belong. Classification Result: - For inputs of any new patient s heart disease dataset, it provides results such as whether the patient is a healthy person or if not then to which class it belongs. If input is given in the form of file containing patient records then classification result is the form of confusion matrix. 5. EXPERIMENTAL RESULTS AND DISCUSSION OF MLP NN CLASSIFIER The network is trained several times with different random initialization of connection weights to ensure the true learning. Termination is when training gets convergence i.e. the error difference between the actual and predicted output is less than or equal to error limit. It is also established from the results that, the 90 % training and 10 % testing data partition gives best results. It is clear that transfer function of neurons in hidden layer as well as output layer should be tanh (hyperbolic tangent). Details about the training algorithm and its parameter are as given in table 3. The MLP neural network should be trained using back propagation algorithm with supervised learning rule. The designed classifier is evaluated on cross validation with regard to percent classification accuracy, specificity and sensitivity. Table 3: Variable Parameters of MLP NN ( ) Parameter Range Optimal Values Exemplars for training 10% to 90 % 90% Exemplars for cross 10% to 90 % 10% validation Number of epochs 1000 to Class Class Class Class Class Number of Hidden 1 to 3 1 layer Number of hidden 2 to Volume 4, Issue 5, May 2015 Page 430

6 neurons Transfer function of the neurons in hidden layer Transfer function of the neuron in the output layer Supervised Learning Rule Step Size at hidden and output Layer Tanh, sigmoid, Linear tanh, Tanh Log sigmoid, Bias axon Linear Axon, Axon Tanh,sigmoid,Linear tanh,log Tanh sigmoid, Bias axon Linear Axon, Axon Step, Momemtum,conjugate Step Gradiant(CG) 0 to Error limit 0 to Selection of Error Criteria Normally Euclidian or L2 norm is used. When problem incorporates very high degree of nonlinearity different error norms could be examined for their suitability in computation of error between output of NN model and the desired output. For MLP NN L2 norm provides the highest classification accuracy on training, testing and cross validation. 5.2 Performance Measures of MLP NN Proposed neural network is trained using back propagation algorithm and confusion matrix for cross validation so as to ensure that its performance does not depend on any specific data partitioning scheme. In this, rows are selected randomly by factor n which depends upon the data partitioning percentage of train and cross validation. Table 4 shows the performance measures for the MLP NN classifier with different dataset with respect to normal and diseased heart instances. Table 4: Performance Measures for MLP NN Classifiers Data Sets % Classification Accuracy 13:60:05 MLP 90% training data 90% Trainin g Data 10% Testing Data % Sensitivit y % Specificit y Error Limit 96.69% 96.29% 92.56% 100% :60:05 MLP 90 % training data % 100% 95.86% 100% 0.1 From above table it implies that MLP NN as a classifier in this work possesses more learning ability compare to previously implemented techniques. The most important criterion in this work is to what extent the MLP NN classifier is able to correctly classify the exemplars [16]. To confirm whether the proposed model is really consistently capable of more accurate classification, different data partition sets are used to train the network. As per the confusion matrices it was found that the MLP Neural classifier has the advantage of reducing misclassifications among the neighborhood classes as compare to other NN classifiers [13]. 6. RESULTS AND DISCUSSION Table 5: Performance comparison of other technique with others on same dataset Previous Technique Performance % Accuracy, References % Sensitivity, % Specificity, error limit 13input 2 output Accuracy 94%,Error Rate 0.1 [11] 13:60:05 MLP 90% train data Accuracy 96.69%,Sensitivity 92,56%, Specificity 100%,Error Rate , Volume 4, Issue 5, May 2015 Page 431

7 16:60:05 MLP 90 % train data Accuracy %,Sensitivity 96.86%, Specificity 100%,Error Rate 0.1, From the performance comparison of proposed technique with others on same dataset as shown in above table 5, it is proved that the proposed MLP NN classifier with 16 input attributes clearly outperforms earlier researcher s techniques. Previous related research analysis for heart diseases dataset shows report 94 % accuracy.with selected parameters of MLP NN, when it is trained several times and tested over cross validation, then overall accuracy 98.16%, sensitivity 95.86% and 100 % specificity are achieved which shows consistent performance than other neural network. 7. CONCLUSION Proposed neural network method proved to be reliable for diagnosis of angina in patients with heart disease. Additional studies with larger number of patients are required to improve accuracy and usefulness of artificial neural network. It is observed that MLP NN is fastest, simple in design, lowest MSE and highest accuracy. As per wide range of applicability of ANN, neural networks are well suited to solve complex problems due to their ability to learn complex and nonlinear relationships including noisy or less precise information. From the design of neural networks, it is evident that MLP NNs required a compact architecture as compared to other NNs, in terms of number of hidden nodes required for the classification. The number parameters such as weights and biases required for the designing of MLP NN is sufficiently lower than other. This simple and compact structure indicates the feasibility of MLP NN for online implementation and the hardware implementation [14]. Whenever new dataset findings are listed, this classification system can be retrained to accommodate new knowledge. This MLP NN classifier can be used to assist physicians to detect heart disease class for preliminary diagnosis, thus they can attempt perfection in the diagnosis of heart disease. REFERENCES [1] Anamica Gupta, Naveen Kumar and Vasuda Bhatnagar, Analysis of Medical Data using Data Mining and formal Concept Analysis, Proceedings of World Academy Of Science, Engineering and Technology, Vol. 6, June [2] Bonow, Libby, Mann, Zipes, Heart Disease: a textbook of Cardiovascular Medicine, Eight edition, Saunders, Elsevier, [3] Mathers C.D., Lopez A., Stein D., Deaths and disease burden by cause: Global burden of Disease estimates by World Bank Country Group, [4] John Shafer, Rakesh Agarwal, and Manish Mehta, SPRINT: A Scalable parallel classifier for Data Mining, In Proceedings of the VLDB Conference, Bombay, India, [5] Manjusha B. wadhonkar, P.A.Tijare,S.N Sawalkar, Artificial Neural Network Approach for Classification of Heart Disease Dataset, International Journal of Application or Innovation in Engineering & Management(IJAIEM),Vol.3,Issue 4,pp ,April [6] R.Rojas, Neural Networks: a systematic introduction, Springer-Verleg, [7] R.P.Lippmann, Pattern Classification using Neural Networks, IEEE commun.mag.pp.47-64, [8] Simon Haykin, Neural Network: A Comprehensive foundation, Pearson Prentice Hall, New Delhi, [9] Murphy P.M. and Aha D. W., UCI Machine Learning Databases Repository Irvine C.A: University of California, Department of Information and Computer Science,ftp://ftp.ics.uci.edu/pub/machine-learningdatabases/heart/,2004. [10] Bose, N.K. and Liang, P. Neural Network Fundamentals with graphs, algorithms and applications: Tata McGraw- Hill publishing company Ltd., New Delhi, [11] Dr. K Usha Rani, Analysis of Heart Disease Dataset using Neural Network Approach, International journal of Data Mining & Knowledge Management(IJDKP),Vol.1,No.5,pp. 1-6,September [12] Hagan, M.T, Demuth H.B, Beale M.H., Neural Network Design, PWS Publishing, Boston, MA [13] Ranjana Raut, Dr. S.V. Dudul, Intelligent Diagnosis of Heart Diseases using Neural Network Approach, International Journal of Computer Applications( ),Vol.1-No.2,pp ,2010. [14] Reyneri, L.M., Implementation Issues of Neuro Fuzzy Hardware: going towards HW/SW co design, IEEE Trans. On Neural Networks, Vol.14, no.1, pp , [15] Sahana Devanathan,Ambika R, Heart Disease Prediction System using Bayes Theorem, International Journal of scientific Engineering Research, Vol. 4,Issue 4,pp ,Apr [16] Nadir N.Chamiya, Sanjay V. Dudul, Classification of material type and its surface properties using Digital signal Processing techniques and neural network, Applied Soft Computing, ELSEVIER, Vol. 11,Issue 1,pp ,Jan Volume 4, Issue 5, May 2015 Page 432

8 AUTHOR Manjusha B. Wadhonkar M.E (Computer Engineeing) Second Year. Computer Science and Engineering department. Sipna College of Engineering and Technology, Amaravati( M.S). Volume 4, Issue 5, May 2015 Page 433