# Improved Fuzzy C-means Clustering Algorithm Based on Cluster Density

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Journal of Computational Information Systems 8: 2 (2012) Available at Improved Fuzzy C-means Clustering Algorithm Based on Cluster Density Xiaojun LOU, Junying LI, Haitao LIU Shanghai Institute of Micro-system and Information Technology, Chinese Academy of Sciences, Shanghai , China Abstract The fuzzy C-means algorithm (FCM) is one of the most popular techniques used for clustering. However, the conventional FCM uses the Euclidean distance as the similarity criterion of data points, which leads to limitation of equal partition trend for data sets. And the clustering performance is highly affected by the data structure including cluster shape and cluster density. To solve this problem, a distance regulatory factor is proposed to correct the similarity measurement. The factor is based on cluster density, which represents the global distribution information of points in a cluster. It is then applied to the conventional FCM for distance correction. Two sets of experiments using artificial data and UCI data are operated, and the results show that the proposed algorithm is superior to the other three clustering algorithms. Keywords: Clustering; Fuzzy C-means; Regulatory Factor; Cluster Density; Distance Correction 1 Introduction Cluster analysis is a branch in statistical multivariate analysis and unsupervised machine learning, which has extensive applications in various domains, including financial fraud, image processing, medical diagnosis, and text categorization [1-4]. Clustering is a process of grouping a set of objects into clusters so that the objects in the same cluster have high similarity but are dissimilar with objects in other clusters. The similarity criterion for distinguishing the difference between objects is generally measured by distance. Obviously, we put two data points in the same group if they are close to each other. And data points are from different groups if the distance between them is large. So, the performance of a clustering algorithm is highly affected by the similarity criterion. The fuzzy C-means (FCM) algorithm is one of the most popular techniques used for clustering [5]. It is a kind of partitional clustering algorithm, which aims at partitioning a given data set into disjoint subsets so that specific clustering criteria are optimized. The conventional FCM method uses Euclidean distance as the similarity measurement of the data points. It has a Corresponding author. address: (Xiaojun LOU) / Copyright 2012 Binary Information Press January 2012

2 728 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) limitation of equal partition trend for data sets and is only suitable for clustering data groups with hyperspherical shape [6]. In order to improve the performance of conventional FCM, a lot of researches have been done. Wu and Yang [7] replaced the Euclidean norm with a normalized distance function 1 exp( β x i v c 2 ), where β is a positive constant. Yang and Lin [8, 9] separately added a penalty term w ln ξ i and a compensated term τ tanh ξ i to distance function. Graves and Pedrycz [10] used 1 K(x i, v c ) as the distance measure, where K(x i, v c ) is a kernel function. Kernel function can be defined in many forms, which could bring different performance. In [11], Mahalanobis distance is used by Xiang. It differs from Euclidean distance in that it takes into account the correlations of the data set and is scale-invariant. A feature-weighted distance [12, 13] was proposed to improve the performance of FCM, as the fact that the features differ from each other for contribution to clustering. Tsai [14] presented a new distance metric that incorporates the distance variation in a cluster to regularize the distance measure. And a hybrid distance metric [15] represented as the linear combination of several know distance metrics is proposed. That approach could benefit from advantages of different distance. Although there is a lot of study on distance measure, the approaches mentioned above not fully take into account the data distribution, and the shape character of data set is not well used. In this paper, we study on the distribution of the data set, and introduce a definition of cluster density as the representation of the inherent character of the data set. A regulatory factor based on cluster density is proposed to correct the distance measure in the conventional FCM. It differs from other approaches in that the regulator uses both the shape of the data set and the middle result of iteration operation. And the distance measure function is dynamically corrected by the regulatory factor until the objective criterion is achieved. Two sets of experiments using artificial data and UCI data are operated. Comparing with some existing methods, the proposed algorithm shows the better performance. The remaining content of this paper is organized as follows: Section 2 introduces the conventional FCM and some improved algorithms in related literature. Section 3 demonstrates the proposed algorithm in detail. The performance of the algorithm is evaluated in Section 4. And in Section 5 concludes this paper. 2 Related Work In this section, we review the conventional fuzzy c-means clustering algorithm, and give short introduction of two existing improved approaches. 2.1 FCM The most widely used fuzzy clustering method is the fuzzy c-means clustering, FCM for short [16]. The objective function of the FCM is to obtain a fuzzy c-partition for the given data set by minimizing the objective function J F CM, seeing the following Eq. (1). The constraints related to the fuzzy membership degree are given, as expressed in Eq. (2). J F CM (X, U, V ) = c u m ij d 2 ij (1) i=1

3 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) s.t. u ij [0, 1], u ij > 0, c u ij = 1 (2) The optimal minimum value of J F CM is normally solved by the Alternative Optimization (AO) method. It relates to two updated equation, fuzzy membership degrees and cluster prototypes. The two update equations are given in Eqs. (3) and (4). i=1 u ij = c 1 ( d ij d kj ) 2 m 1 k=1 (3) v i = u m ij x j u m ij (4) The distance between data points is defined as bellows: d 2 ij = x j v i 2 A = (x j v i )A(x j v i ) T (5) Where A is a symmetric positive definite matrix. In the conventional FCM, the distance measurement is usually Euclidean distance, which means A = I in Eq. (5). The FCM uses the probabilistic constraint that the membership degree of a data point across all clusters sums to 1. The constraint is used to generate the membership degree updated equations for an iterative algorithm. The membership degrees resulting from the FCM and its derivations, however, do not always correspond to the intuitive concept of degree of belonging. Moreover, FCM is only suitable for hyperspherical clusters and has a limitation of equal partition trend for data sets. 2.2 FCM-M In [17], Cai proposed an improved new algorithm called Fuzzy C-Means based on Mahalanobis distance function (FCM-M), and added a regulating factor of covariance matrix to each class in objective function. The Mahalanobis distance between a data point and a cluster centroid is defined as bellows: ˆd 2 ij = (x j v i ) T Σ 1 ij (x j v i ) (6) Where 1 ij is the covariance matrix of x j and v i. The Mahalanobis distance is better adapted than the usual Euclidian distance to setting involving non spherically symmetric distributions. It is more particularly useful when multinormal distributions are involved, although its definition does not require the distributions to be normal. Involving the regulatory factor based on Mahalanobis, the objective function is given by J F CM M (X, U, V, Σ) = c u m ij [(x j v i ) T Σ 1 ij (x j v i ) ln Σ 1 ij ] (7) i=1

4 730 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) FCM-σ In [14], Tsai and Lin proposed a new metric that takes the distance variation in each clusters as the regularization of the Euclidean distance. The new distance metric is defined as Where σ i is the weighted mean distance in cluster i, and is given by ˆd ij = x j v i σ i (8) u m ij x j v i 2 σ i = u m ij Different from the Mahalanobis distance, the new distance measure normalizes the distance based on the spread of data points from the centroid in a cluster. It is not normalized with respect to the covariance between features. The FCM-σ searches for C clusters by minimizing the objective: c J F CM σ (X, U, V ) = u m x j v i 2 ij (10) σ i 3 Proposed Approach The current existing fuzzy c-means based clustering algorithms either consider only hyperspherical clusters in data space or describe only weighted features. However the density of data points in a cluster could be distinctly different from other clusters in a data set. The conventional distance measure only evaluates the difference between two individual data points. It ignores the global view of the data distribution. Considering the shape of the data set, a distance regulatory factor based on cluster density is proposed to correct the similarity measurement. i=1 1/2 (9) 3.1 Cluster density Given a data set X = {x 1, x 2,, x n }, for every data point x i, the dot density is usually defined as: 1 z i =, d ij e, 1 i n (11) d ij,j i Where e is the effective radius for density evaluating. The value of e should be set appropriately according to the application scenarios, because it affects the result directly. The dot density is relatively higher when e is set bigger. To avoid this uncertainty, we can simplify the Eq. (11) when the distribution of data set is uniform and these are no outliers. The new definition is: z i = 1/ min({d ij }), 1 i n (12) We use the inverse of the minimum distance value among the neighbors as the dot density approximately.

5 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) However, dot density considers only the local distribution of the data set. Cluster density is proposed to solve this problem. Cluster density is the weighted linear combination of dot densities as expressed in Eq. (13). α ij w ij z j ẑ i =, 1 i c (13) α ij w ij Where α ij is the category label of data point x j, and w ij is the weight of x j. α ij = 1 when x j most likely belongs to the cluster i, otherwise α ij = 0. And w ij is a positive constant which can be adjusted by users. Cluster density ẑ i considers the global shape of the data set in a cluster and uses the dynamical membership degrees in the iteration process. Furthermore, using cluster density instead of dot density can reduce the computation consumption during clustering. 3.2 FCM based on cluster density (FCM-CD) Using the cluster density, the distance measure is corrected as Eq. (14). ˆd 2 ij = x j v i 2 ẑ i, 1 i c, 1 j n (14) Thus, the optimization expression can be written as follows base on Eqs. ((1),(13) and (14)), as seen in Eq. (15): J F CM CD (U, V, X) = c u m ij x j v i 2 i=1 α ik w ik k=1 (15) α ik w ik z k k=1 Applying Lagrange Multiplying Method to Eq. (15), we can obtain the two update equations given in Eqs. (16) and (17). u m ij x j v i =, 1 i c (16) u m ij u ij = c k=1 ˆd 2/(m 1) ij ˆd 2/(m 1) kj (17) Comparing the conventional FCM method, the proposed algorithm (FCM-CD) brings in the cluster density based regulatory factor as the distance correction in similarity measurement. The process of stepwise regression involves of these steps as follows: 1) Choose the number of clusters c, fuzziness index m, iteration error ε, maximum iterations T, and initialize the membership degree matrix U (0).

6 732 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) ) Get the initial centroids using Eq. (17). 3) Calculate the dot density of every data point using Eq. (12). 4) And when the iteration index is t(t = 1, 2,, T ), updating the membership degree matrix U (t) and cluster centroids V (t) using Eqs. (17) and (18). 5) Calculate the value of the objective function J (t) using Eq. (15). 6) If U (t) U (t 1) < ε or t = T, then stop the iteration and get the membership degree matrix U and the cluster centroids V, otherwise set and return to step (4). 4 Numerical Experiments This section evaluates the performance of the proposed FCM-CD algorithm through numerical experiments. The experiments include two groups: group one using artificial data sets to analysis the advantage of the FCM-CD and group two using the well-known public UCI data sets to check the ability of the FCM-CD in clustering. Four clustering methods including the conventional FCM, FCM-M, FCM-σ, FCM-CD, are evaluated and compared with each other in both groups experiments. And we choose the same parameters by setting m = 2, ε = 10 5, and T = Group one: artificial data In order to show the differences of clustering methods visually, artificial data sets are created in a two-dimensional plane for experiments. First, two circular clusters are simulated, as seen in Fig. 1. They are both normally distributed in a circle. The centroids are (0, 0) and (5.5, 0); the radiuses are 1 and 5. The two circles have a small overlapping area. Each cluster has 100 data points. (a) FCM (b) FCM-M The clustering results in Fig. 1 reveal that FCM and FCM-M have poor performance when the densities of the clusters are different and FCM-σ and FCM-CD get the higher clustering accuracy

7 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) (c) FCM-σ (d) FCM-CD Fig. 1: Two circular clusters with different density (Points in the same color are in the same cluster; the connecting lines represent the clustering results; the black dots are centroids after clustering) due to the regulatory factor. The clustering errors and absolute deviations of centroids of the four methods are shown in Table 1. Table 1: Clustering result with two circular clusters Result Clustering method FCM FCM-M FCM-σ FCM-CD Clustering error 10.0% 9.5% 1.0% 0.5% Centroids deviation Then a circular cluster and an elliptic cluster are simulated, as seen in Fig. 2. The centroids are (0, 0) and (5.5, 0); the radius of the circle is 1. The major radius and minor radius of the ellipse are 5 and 2. Each cluster has 100 data points. (a) FCM (b) FCM-M Fig. 2 shows that the conventional FCM method has the biggest clustering error when the data set is not spherical distributed. And the proposed FCM-CD method performs best. The errors

8 734 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) (c) FCM-σ (d) FCM-CD Fig. 2: A circular cluster and a elliptic cluster (Points in the same color are in the same cluster; the connecting lines represent the clustering results; the black dots are centroids after clustering) of the four clustering methods (FCM, FCM-M, FCM-σ and FCM-CD) are 4.5%, 4.0%, 2.0% and 1.5%, as shown in Table 2. The absolute deviations of centroids are also given in Table 2. Table 2: Clustering result with a circular cluster and a elliptic cluster Result Clustering method FCM FCM-M FCM-σ FCM-CD Clustering error 4.5% 4.0% 2.0% 1.5% Centroids deviation A good clustering algorithm should be tolerant to different densities and various cluster shapes. The comparing experiments above show that FCM-CD is superior to the FCM and the other two existing FCM based methods. 4.2 Group two: UCI data In this group of the experiments, the well-known public real data sets are applied. We choose the Iris, Wine and SPECT Heart data sets, which are widely used for classifications and clustering. Table 3 gives the description of the data sets. Data set Table 3: Data sets description Number of Number of Number of clusters instances attributes Iris Wine SPECT Heart

9 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) Four clustering methods are operated with these data sets and the results are shown in Table 4-6. We give out the number of misclustering in each cluster and the total error rate. We can see that the proposed FCM-CD outperforms the other algorithms in the real data sets, although FCM-M has nearly the same performance. Taking advantages of the mahalanobis distance, FCM- M has some more intrinsic strengths than Euclidean based methods. And the error rates of the Wine data and the SPECT Heart data are relatively higher due to the nature of data sets. These two data sets have high-dimensional attributes, which will easily cause misclusterings without applying suitable feature selection method. And none of the four methods mentioned in this paper have consideration of the feature weight. That s why the error rates reach nearly fifty percent. However, the experiments still can prove that the FCM-CD is superior. Table 4: Clustering result of Iris data Clustering Misclustering Misclustering Misclustering method number in cluster 1 number in cluster 2 number in cluster 3 Error rate FCM % FCM-M % FCM-σ % FCM-CD % Table 5: Clustering result of Wine data Clustering Misclustering Misclustering Misclustering method number in cluster 1 number in cluster 2 number in cluster 3 Error rate FCM % FCM-M % FCM-σ % FCM-CD % Table 6: Clustering result of SPECT heart data Clustering Misclustering number Misclustering number method in cluster 1 in cluster 2 Error rate FCM % FCM-M % FCM-σ % FCM-CD % 5 Conclusions As the fact that the conventional fuzzy c-means clustering algorithm is only suitable for hyperspherical clusters, a improved clustering approach based on cluster density (FCM-CD) is proposed.

10 736 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) Considering of the global dot density in a cluster, a distance correction regulatory factor is built and applied to FCM. Two groups of experiments using artificial data and UCI data are operated. Two other FCM based clustering methods are introduced to compare with the proposed FCM-CD algorithm. The experiment results reveal that FCM-CD has a good tolerance to different densities and various cluster shapes. And FCM-CD shows a higher performance in clustering accuracy. Acknowledgement This work is supported by the National Science and Technology Major Projects of China and the Major State Basic Research Development Program of China. References [1] R. Filipovych, S. M. Resnick, C. Davatzikos. Semi-supervised cluster analysis of imaging data. NeuroImage, 2011, 54 (3): [2] J. Jiang, W. Li, X. Cai. Cluster behavior of a simple model in financial markets. Physica A: Statistical Mechanics and its Applications, 2008, 387 (2 3): [3] Yan Li, Edward Hung, Korris Chung. A subspace decision cluster classifier for text classification. Expert Systems with Applications, 2011, 38 (10): [4] L. Zhang, L. Y. Zhang, S. Y. Chen, et al. Research on customer classification based on fuzzy clustering. Journal of Computational Information Systems, 2007, 3 (5): [5] Witold Pedrycz, Partab Rai. Collaborative clustering with the use of Fuzzy C-Means and its quantification. Fuzzy Sets and Systems, 2008, 159 (18): [6] X. F. Liu, H. L. Zeng, B. C. Lv. Clustering analysis of dot density function weighted fuzzy c-means algorithm. Computer Engineering and Applications, 2004, 40 (24): [7] K. L. Wu, M. S. Yang. Alternative C-means clustering algorithms. Pattern Recognition, 2002, 35 (10): [8] M. S. Yang. On a class of fuzzy classification maximum likelihood procedures. Fuzzy Sets and System, 1993, 57 (3): [9] J. S. Lin. Fuzzy clustering using a compensated fuzzy hopfield network. Neural Processing Letters, 1999, 10 (1): [10] Daniel Graves, Witold Pedrycz. Kernel-based fuzzy clustering and fuzzy clustering: A comparative experimental study. Fuzzy Sets and Systems, 2010, 161 (4): [11] S. M. Xiang, F. P. Nie, C. S. Zhang. Learning a Mahalanobis distance metric for data clustering and classification. Pattern Recognition, 2008, 41 (12): [12] W. L. Hung, M. S. Yang, D. H. Chen. Bootstrapping approach to feature-weight selection in fuzzy C-means algorithms with an application in color image segmentation. Pattern Recognition Letters, 2008, 29 (9): [13] C. H. Cheng, J. W. Wang, M. C. Wu. OWA-weighted based clustering method for classification problem. Expert System with Applications, 2009, 36 (3): [14] D.M. Tsai, C. C.Lin. Fuzzy C-means based clustering for linearly and nonlinearly separable data. Pattern Recognition, 2011, 44 (8): [15] J. Wang, S. T. Wang. Double indices FCM algorithm based on hybrid distance metric learning. Journal of Software, 2010, 21 (8):

11 X. Lou et al. /Journal of Computational Information Systems 8: 2 (2012) [16] J. C. Bezdek. Pattern Recognition with Fuzzy Objective Function Algorithms. MA, USA, Springer, [17] J. Y. Cai, F. D. Xie, Y. Zhang. Fuzzy c-means algorithm based on adaptive Mahalanobis distance. Computer Engineering and Applications, 2010, 46 (34):

### Review of Computer Engineering Research WEB PAGES CATEGORIZATION BASED ON CLASSIFICATION & OUTLIER ANALYSIS THROUGH FSVM. Geeta R.B.* Shobha R.B.

Review of Computer Engineering Research journal homepage: http://www.pakinsight.com/?ic=journal&journal=76 WEB PAGES CATEGORIZATION BASED ON CLASSIFICATION & OUTLIER ANALYSIS THROUGH FSVM Geeta R.B.* Department

### Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016

Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with

### D-optimal plans in observational studies

D-optimal plans in observational studies Constanze Pumplün Stefan Rüping Katharina Morik Claus Weihs October 11, 2005 Abstract This paper investigates the use of Design of Experiments in observational

### A Survey of Kernel Clustering Methods

A Survey of Kernel Clustering Methods Maurizio Filippone, Francesco Camastra, Francesco Masulli and Stefano Rovetta Presented by: Kedar Grama Outline Unsupervised Learning and Clustering Types of clustering

### Lecture 20: Clustering

Lecture 20: Clustering Wrap-up of neural nets (from last lecture Introduction to unsupervised learning K-means clustering COMP-424, Lecture 20 - April 3, 2013 1 Unsupervised learning In supervised learning,

### Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris

Class #6: Non-linear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Non-linear classification Linear Support Vector Machines

### A Modified K-Means Clustering with a Density-Sensitive Distance Metric

A Modified K-Means Clustering with a Density-Sensitive Distance Metric Ling Wang, Liefeng Bo, Licheng Jiao Institute of Intelligent Information Processing, Xidian University Xi an 710071, China {wliiip,blf0218}@163.com,

### Fraudulent Behavior Forecast in Telecom Industry Based on Data Mining Technology

Communications of the IIMA Volume 7 Issue 4 Article 1 2007 Fraudulent Behavior Forecast in Telecom Industry Based on Data Mining Technology Sen Wu School of Economics and Management University of Science

### Analysis of kiva.com Microlending Service! Hoda Eydgahi Julia Ma Andy Bardagjy December 9, 2010 MAS.622j

Analysis of kiva.com Microlending Service! Hoda Eydgahi Julia Ma Andy Bardagjy December 9, 2010 MAS.622j What is Kiva? An organization that allows people to lend small amounts of money via the Internet

### A Health Degree Evaluation Algorithm for Equipment Based on Fuzzy Sets and the Improved SVM

Journal of Computational Information Systems 10: 17 (2014) 7629 7635 Available at http://www.jofcis.com A Health Degree Evaluation Algorithm for Equipment Based on Fuzzy Sets and the Improved SVM Tian

### An effective fuzzy kernel clustering analysis approach for gene expression data

Bio-Medical Materials and Engineering 26 (2015) S1863 S1869 DOI 10.3233/BME-151489 IOS Press S1863 An effective fuzzy kernel clustering analysis approach for gene expression data Lin Sun a,b,, Jiucheng

### 4. GPCRs PREDICTION USING GREY INCIDENCE DEGREE MEASURE AND PRINCIPAL COMPONENT ANALYIS

4. GPCRs PREDICTION USING GREY INCIDENCE DEGREE MEASURE AND PRINCIPAL COMPONENT ANALYIS The GPCRs sequences are made up of amino acid polypeptide chains. We can also call them sub units. The number and

### An Analysis of Missing Data Treatment Methods and Their Application to Health Care Dataset

P P P Health An Analysis of Missing Data Treatment Methods and Their Application to Health Care Dataset Peng Liu 1, Elia El-Darzi 2, Lei Lei 1, Christos Vasilakis 2, Panagiotis Chountas 2, and Wei Huang

### Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov

Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical Main

Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototype-based Fuzzy c-means

### Dynamic intelligent cleaning model of dirty electric load data

Available online at www.sciencedirect.com Energy Conversion and Management 49 (2008) 564 569 www.elsevier.com/locate/enconman Dynamic intelligent cleaning model of dirty electric load data Zhang Xiaoxing

### Credal classification of uncertain data using belief functions

23 IEEE International Conference on Systems, Man, and Cybernetics Credal classification of uncertain data using belief functions Zhun-ga Liu a,c,quanpan a, Jean Dezert b, Gregoire Mercier c a School of

### Support Vector Machines with Clustering for Training with Very Large Datasets

Support Vector Machines with Clustering for Training with Very Large Datasets Theodoros Evgeniou Technology Management INSEAD Bd de Constance, Fontainebleau 77300, France theodoros.evgeniou@insead.fr Massimiliano

### Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu

Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?

### A MATLAB Toolbox and its Web based Variant for Fuzzy Cluster Analysis

A MATLAB Toolbox and its Web based Variant for Fuzzy Cluster Analysis Tamas Kenesei, Balazs Balasko, and Janos Abonyi University of Pannonia, Department of Process Engineering, P.O. Box 58, H-82 Veszprem,

### Methodology for Emulating Self Organizing Maps for Visualization of Large Datasets

Methodology for Emulating Self Organizing Maps for Visualization of Large Datasets Macario O. Cordel II and Arnulfo P. Azcarraga College of Computer Studies *Corresponding Author: macario.cordel@dlsu.edu.ph

### Segmentation of stock trading customers according to potential value

Expert Systems with Applications 27 (2004) 27 33 www.elsevier.com/locate/eswa Segmentation of stock trading customers according to potential value H.W. Shin a, *, S.Y. Sohn b a Samsung Economy Research

### A FUZZY BASED APPROACH TO TEXT MINING AND DOCUMENT CLUSTERING

A FUZZY BASED APPROACH TO TEXT MINING AND DOCUMENT CLUSTERING Sumit Goswami 1 and Mayank Singh Shishodia 2 1 Indian Institute of Technology-Kharagpur, Kharagpur, India sumit_13@yahoo.com 2 School of Computer

### CITY UNIVERSITY OF HONG KONG 香 港 城 市 大 學. Self-Organizing Map: Visualization and Data Handling 自 組 織 神 經 網 絡 : 可 視 化 和 數 據 處 理

CITY UNIVERSITY OF HONG KONG 香 港 城 市 大 學 Self-Organizing Map: Visualization and Data Handling 自 組 織 神 經 網 絡 : 可 視 化 和 數 據 處 理 Submitted to Department of Electronic Engineering 電 子 工 程 學 系 in Partial Fulfillment

### Gustafson-Kessel-like clustering algorithm based on typicality degrees

Gustafson-Kessel-like clustering algorithm based on typicality degrees Marie-Jeanne Lesot FIN, Otto-von-Guericke Universität, Magdeburg, Germany lesot@iws.cs.uni-magdeburg.de Rudolf Kruse FIN, Otto-von-Guericke

### A Simple Introduction to Support Vector Machines

A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University Outline A brief history of SVM Large-margin linear

### A Study on the Comparison of Electricity Forecasting Models: Korea and China

Communications for Statistical Applications and Methods 2015, Vol. 22, No. 6, 675 683 DOI: http://dx.doi.org/10.5351/csam.2015.22.6.675 Print ISSN 2287-7843 / Online ISSN 2383-4757 A Study on the Comparison

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### An Initial Study on High-Dimensional Data Visualization Through Subspace Clustering

An Initial Study on High-Dimensional Data Visualization Through Subspace Clustering A. Barbosa, F. Sadlo and L. G. Nonato ICMC Universidade de São Paulo, São Carlos, Brazil IWR Heidelberg University, Heidelberg,

### Classifying Large Data Sets Using SVMs with Hierarchical Clusters. Presented by :Limou Wang

Classifying Large Data Sets Using SVMs with Hierarchical Clusters Presented by :Limou Wang Overview SVM Overview Motivation Hierarchical micro-clustering algorithm Clustering-Based SVM (CB-SVM) Experimental

### Optimize Position and Path Planning of Automated Optical Inspection

Journal of Computational Information Systems 8: 7 (2012) 2957 2963 Available at http://www.jofcis.com Optimize Position and Path Planning of Automated Optical Inspection Hao WU, Yongcong KUANG, Gaofei

### 3 An Illustrative Example

Objectives An Illustrative Example Objectives - Theory and Examples -2 Problem Statement -2 Perceptron - Two-Input Case -4 Pattern Recognition Example -5 Hamming Network -8 Feedforward Layer -8 Recurrent

### Social Media Mining. Data Mining Essentials

Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

### Support Vector Machines Explained

March 1, 2009 Support Vector Machines Explained Tristan Fletcher www.cs.ucl.ac.uk/staff/t.fletcher/ Introduction This document has been written in an attempt to make the Support Vector Machines (SVM),

### CSE 494 CSE/CBS 598 (Fall 2007): Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye

CSE 494 CSE/CBS 598 Fall 2007: Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye 1 Introduction One important method for data compression and classification is to organize

### ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 http://intelligentoptimization.org/lionbook Roberto Battiti

### An Order-Invariant Time Series Distance Measure [Position on Recent Developments in Time Series Analysis]

An Order-Invariant Time Series Distance Measure [Position on Recent Developments in Time Series Analysis] Stephan Spiegel and Sahin Albayrak DAI-Lab, Technische Universität Berlin, Ernst-Reuter-Platz 7,

### PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA

PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA Prakash Singh 1, Aarohi Surya 2 1 Department of Finance, IIM Lucknow, Lucknow, India 2 Department of Computer Science, LNMIIT, Jaipur,

### Introduction to Support Vector Machines. Colin Campbell, Bristol University

Introduction to Support Vector Machines Colin Campbell, Bristol University 1 Outline of talk. Part 1. An Introduction to SVMs 1.1. SVMs for binary classification. 1.2. Soft margins and multi-class classification.

### An Imbalanced Spam Mail Filtering Method

, pp. 119-126 http://dx.doi.org/10.14257/ijmue.2015.10.3.12 An Imbalanced Spam Mail Filtering Method Zhiqiang Ma, Rui Yan, Donghong Yuan and Limin Liu (College of Information Engineering, Inner Mongolia

### Data Mining Cluster Analysis: Advanced Concepts and Algorithms. ref. Chapter 9. Introduction to Data Mining

Data Mining Cluster Analysis: Advanced Concepts and Algorithms ref. Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar 1 Outline Prototype-based Fuzzy c-means Mixture Model Clustering Density-based

### Classification algorithm in Data mining: An Overview

Classification algorithm in Data mining: An Overview S.Neelamegam #1, Dr.E.Ramaraj *2 #1 M.phil Scholar, Department of Computer Science and Engineering, Alagappa University, Karaikudi. *2 Professor, Department

### Standardization and Its Effects on K-Means Clustering Algorithm

Research Journal of Applied Sciences, Engineering and Technology 6(7): 399-3303, 03 ISSN: 040-7459; e-issn: 040-7467 Maxwell Scientific Organization, 03 Submitted: January 3, 03 Accepted: February 5, 03

### SOME CLUSTERING ALGORITHMS TO ENHANCE THE PERFORMANCE OF THE NETWORK INTRUSION DETECTION SYSTEM

SOME CLUSTERING ALGORITHMS TO ENHANCE THE PERFORMANCE OF THE NETWORK INTRUSION DETECTION SYSTEM Mrutyunjaya Panda, 2 Manas Ranjan Patra Department of E&TC Engineering, GIET, Gunupur, India 2 Department

### Categorical Data Visualization and Clustering Using Subjective Factors

Categorical Data Visualization and Clustering Using Subjective Factors Chia-Hui Chang and Zhi-Kai Ding Department of Computer Science and Information Engineering, National Central University, Chung-Li,

### K Thangadurai P.G. and Research Department of Computer Science, Government Arts College (Autonomous), Karur, India. ktramprasad04@yahoo.

Enhanced Binary Small World Optimization Algorithm for High Dimensional Datasets K Thangadurai P.G. and Research Department of Computer Science, Government Arts College (Autonomous), Karur, India. ktramprasad04@yahoo.com

### Linear Threshold Units

Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

### Clustering & Association

Clustering - Overview What is cluster analysis? Grouping data objects based only on information found in the data describing these objects and their relationships Maximize the similarity within objects

### Triangular Visualization

Triangular Visualization Tomasz Maszczyk and W lodzis law Duch Department of Informatics, Nicolaus Copernicus University, Toruń, Poland tmaszczyk@is.umk.pl;google:w.duch http://www.is.umk.pl Abstract.

### Enhanced Customer Relationship Management Using Fuzzy Clustering

Enhanced Customer Relationship Management Using Fuzzy Clustering Gayathri. A, Mohanavalli. S Department of Information Technology,Sri Sivasubramaniya Nadar College of Engineering, Kalavakkam, Chennai,

### A Novel Density based improved k-means Clustering Algorithm Dbkmeans

A Novel Density based improved k-means Clustering Algorithm Dbkmeans K. Mumtaz 1 and Dr. K. Duraiswamy 2, 1 Vivekanandha Institute of Information and Management Studies, Tiruchengode, India 2 KS Rangasamy

### Prototype-based classification by fuzzification of cases

Prototype-based classification by fuzzification of cases Parisa KordJamshidi Dep.Telecommunications and Information Processing Ghent university pkord@telin.ugent.be Bernard De Baets Dep. Applied Mathematics

### Detection of DDoS Attack Scheme

Chapter 4 Detection of DDoS Attac Scheme In IEEE 802.15.4 low rate wireless personal area networ, a distributed denial of service attac can be launched by one of three adversary types, namely, jamming

### EFFICIENT DATA PRE-PROCESSING FOR DATA MINING

EFFICIENT DATA PRE-PROCESSING FOR DATA MINING USING NEURAL NETWORKS JothiKumar.R 1, Sivabalan.R.V 2 1 Research scholar, Noorul Islam University, Nagercoil, India Assistant Professor, Adhiparasakthi College

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

### FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT MINING SYSTEM

International Journal of Innovative Computing, Information and Control ICIC International c 0 ISSN 34-48 Volume 8, Number 8, August 0 pp. 4 FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT

### A Review of Missing Data Treatment Methods

A Review of Missing Data Treatment Methods Liu Peng, Lei Lei Department of Information Systems, Shanghai University of Finance and Economics, Shanghai, 200433, P.R. China ABSTRACT Missing data is a common

### Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

### A Novel Feature Selection Method Based on an Integrated Data Envelopment Analysis and Entropy Mode

A Novel Feature Selection Method Based on an Integrated Data Envelopment Analysis and Entropy Mode Seyed Mojtaba Hosseini Bamakan, Peyman Gholami RESEARCH CENTRE OF FICTITIOUS ECONOMY & DATA SCIENCE UNIVERSITY

### A 0.9 0.9. Figure A: Maximum circle of compatibility for position A, related to B and C

MEASURING IN WEIGHTED ENVIRONMENTS (Moving from Metric to Order Topology) Claudio Garuti Fulcrum Ingenieria Ltda. claudiogaruti@fulcrum.cl Abstract: This article addresses the problem of measuring closeness

### A Study of Web Log Analysis Using Clustering Techniques

A Study of Web Log Analysis Using Clustering Techniques Hemanshu Rana 1, Mayank Patel 2 Assistant Professor, Dept of CSE, M.G Institute of Technical Education, Gujarat India 1 Assistant Professor, Dept

### Distance Metric Learning in Data Mining (Part I) Fei Wang and Jimeng Sun IBM TJ Watson Research Center

Distance Metric Learning in Data Mining (Part I) Fei Wang and Jimeng Sun IBM TJ Watson Research Center 1 Outline Part I - Applications Motivation and Introduction Patient similarity application Part II

### Java Modules for Time Series Analysis

Java Modules for Time Series Analysis Agenda Clustering Non-normal distributions Multifactor modeling Implied ratings Time series prediction 1. Clustering + Cluster 1 Synthetic Clustering + Time series

### Statistical Databases and Registers with some datamining

Unsupervised learning - Statistical Databases and Registers with some datamining a course in Survey Methodology and O cial Statistics Pages in the book: 501-528 Department of Statistics Stockholm University

### Unsupervised learning: Clustering

Unsupervised learning: Clustering Salissou Moutari Centre for Statistical Science and Operational Research CenSSOR 17 th September 2013 Unsupervised learning: Clustering 1/52 Outline 1 Introduction What

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Two Heads Better Than One: Metric+Active Learning and Its Applications for IT Service Classification

29 Ninth IEEE International Conference on Data Mining Two Heads Better Than One: Metric+Active Learning and Its Applications for IT Service Classification Fei Wang 1,JimengSun 2,TaoLi 1, Nikos Anerousis

### Going Big in Data Dimensionality:

LUDWIG- MAXIMILIANS- UNIVERSITY MUNICH DEPARTMENT INSTITUTE FOR INFORMATICS DATABASE Going Big in Data Dimensionality: Challenges and Solutions for Mining High Dimensional Data Peer Kröger Lehrstuhl für

### Improving fuzzy c-means clustering based on feature-weight learning

Pattern Recognition Letters 25 (2004) 1123 1132 www.elsevier.com/locate/patrec Improving fuzzy c-means clustering based on feature-weight learning Xizhao Wang a, *, Yadong Wang b, Luan Wang a,b a Department

### Supply Chain Forecasting Model Using Computational Intelligence Techniques

CMU.J.Nat.Sci Special Issue on Manufacturing Technology (2011) Vol.10(1) 19 Supply Chain Forecasting Model Using Computational Intelligence Techniques Wimalin S. Laosiritaworn Department of Industrial

### A new Approach for Intrusion Detection in Computer Networks Using Data Mining Technique

A new Approach for Intrusion Detection in Computer Networks Using Data Mining Technique Aida Parbaleh 1, Dr. Heirsh Soltanpanah 2* 1 Department of Computer Engineering, Islamic Azad University, Sanandaj

### 1. χ 2 minimization 2. Fits in case of of systematic errors

Data fitting Volker Blobel University of Hamburg March 2005 1. χ 2 minimization 2. Fits in case of of systematic errors Keys during display: enter = next page; = next page; = previous page; home = first

### Customer Segmentation and Customer Profiling for a Mobile Telecommunications Company Based on Usage Behavior

Customer Segmentation and Customer Profiling for a Mobile Telecommunications Company Based on Usage Behavior A Vodafone Case Study S.M.H. Jansen July 17, 2007 Acknowledgments This Master thesis was written

### CPSC 340: Machine Learning and Data Mining. K-Means Clustering Fall 2015

CPSC 340: Machine Learning and Data Mining K-Means Clustering Fall 2015 Admin Assignment 1 solutions posted after class. Tutorials for Assignment 2 on Monday. Random Forests Random forests are one of the

### . Learn the number of classes and the structure of each class using similarity between unlabeled training patterns

Outline Part 1: of data clustering Non-Supervised Learning and Clustering : Problem formulation cluster analysis : Taxonomies of Clustering Techniques : Data types and Proximity Measures : Difficulties

### Data, Measurements, Features

Data, Measurements, Features Middle East Technical University Dep. of Computer Engineering 2009 compiled by V. Atalay What do you think of when someone says Data? We might abstract the idea that data are

### EFFICIENT K-MEANS CLUSTERING ALGORITHM USING RANKING METHOD IN DATA MINING

EFFICIENT K-MEANS CLUSTERING ALGORITHM USING RANKING METHOD IN DATA MINING Navjot Kaur, Jaspreet Kaur Sahiwal, Navneet Kaur Lovely Professional University Phagwara- Punjab Abstract Clustering is an essential

### Regression Using Support Vector Machines: Basic Foundations

Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering

### DEA implementation and clustering analysis using the K-Means algorithm

Data Mining VI 321 DEA implementation and clustering analysis using the K-Means algorithm C. A. A. Lemos, M. P. E. Lins & N. F. F. Ebecken COPPE/Universidade Federal do Rio de Janeiro, Brazil Abstract

### Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 by Tan, Steinbach, Kumar 1 What is Cluster Analysis? Finding groups of objects such that the objects in a group will

### Face Recognition using Principle Component Analysis

Face Recognition using Principle Component Analysis Kyungnam Kim Department of Computer Science University of Maryland, College Park MD 20742, USA Summary This is the summary of the basic idea about PCA

### Study of Euclidean and Manhattan Distance Metrics using Simple K-Means Clustering

Study of and Distance Metrics using Simple K-Means Clustering Deepak Sinwar #1, Rahul Kaushik * # Assistant Professor, * M.Tech Scholar Department of Computer Science & Engineering BRCM College of Engineering

### Classification Problems

Classification Read Chapter 4 in the text by Bishop, except omit Sections 4.1.6, 4.1.7, 4.2.4, 4.3.3, 4.3.5, 4.3.6, 4.4, and 4.5. Also, review sections 1.5.1, 1.5.2, 1.5.3, and 1.5.4. Classification Problems

### Chapter ML:XI (continued)

Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained

### Chapter 7. Diagnosis and Prognosis of Breast Cancer using Histopathological Data

Chapter 7 Diagnosis and Prognosis of Breast Cancer using Histopathological Data In the previous chapter, a method for classification of mammograms using wavelet analysis and adaptive neuro-fuzzy inference

### A Survey on Outlier Detection Techniques for Credit Card Fraud Detection

IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661, p- ISSN: 2278-8727Volume 16, Issue 2, Ver. VI (Mar-Apr. 2014), PP 44-48 A Survey on Outlier Detection Techniques for Credit Card Fraud

### Lecture 9: Introduction to Pattern Analysis

Lecture 9: Introduction to Pattern Analysis g Features, patterns and classifiers g Components of a PR system g An example g Probability definitions g Bayes Theorem g Gaussian densities Features, patterns

### An Enhanced Clustering Algorithm to Analyze Spatial Data

International Journal of Engineering and Technical Research (IJETR) ISSN: 2321-0869, Volume-2, Issue-7, July 2014 An Enhanced Clustering Algorithm to Analyze Spatial Data Dr. Mahesh Kumar, Mr. Sachin Yadav

### CLASSIFICATION AND CLUSTERING. Anveshi Charuvaka

CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training

### MACHINE LEARNING. Introduction. Alessandro Moschitti

MACHINE LEARNING Introduction Alessandro Moschitti Department of Computer Science and Information Engineering University of Trento Email: moschitti@disi.unitn.it Course Schedule Lectures Tuesday, 14:00-16:00

### Comparing large datasets structures through unsupervised learning

Comparing large datasets structures through unsupervised learning Guénaël Cabanes and Younès Bennani LIPN-CNRS, UMR 7030, Université de Paris 13 99, Avenue J-B. Clément, 93430 Villetaneuse, France cabanes@lipn.univ-paris13.fr

### Data Mining Project Report. Document Clustering. Meryem Uzun-Per

Data Mining Project Report Document Clustering Meryem Uzun-Per 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. K-means algorithm...

### International Journal of Software and Web Sciences (IJSWS) Web Log Mining Based on Improved FCM Algorithm using Multiobjective

International Association of Scientific Innovation and Research (IASIR) (An Association Unifying the Sciences, Engineering, and Applied Research) ISSN (Print): 2279-0063 ISSN (Online): 2279-0071 International

### Visualization by Linear Projections as Information Retrieval

Visualization by Linear Projections as Information Retrieval Jaakko Peltonen Helsinki University of Technology, Department of Information and Computer Science, P. O. Box 5400, FI-0015 TKK, Finland jaakko.peltonen@tkk.fi

### THREE DIMENSIONAL REPRESENTATION OF AMINO ACID CHARAC- TERISTICS

THREE DIMENSIONAL REPRESENTATION OF AMINO ACID CHARAC- TERISTICS O.U. Sezerman 1, R. Islamaj 2, E. Alpaydin 2 1 Laborotory of Computational Biology, Sabancı University, Istanbul, Turkey. 2 Computer Engineering

### Neural Networks Lesson 5 - Cluster Analysis

Neural Networks Lesson 5 - Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt. - Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29

### Unsupervised Learning. What is clustering for? What is clustering for? (cont )

Unsupervised Learning c: Artificial Intelligence The University of Iowa Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with