Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering Steffen Unkel

Transcription

1 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering Steffen Unkel Institut für Statistik LMU München Sommersemester 2013

2 Outline 1 Setting the scene 2 Methods for fuzzy clustering 3 The assessment of fuzzy clustering 4 Software for fuzzy clustering 5 Example 6 Literature Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 1 / 20

3 1 Setting the scene Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 2 / 20

4 The concept of a membership function In fuzzy clustering, also referred to as soft clustering, objects are not assigned to a particular cluster. The objects possess a membership function indicating the strength of membership in all or some of the clusters. In most of all other existing clustering techniques, strength of membership is either zero or one, with an object being either in or not in a cluster (hard clustering). An exception is perhaps the finite mixture approach, where strength of membership might be taken as the posteriori probability of belonging to a cluster. Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 3 / 20

5 The concept of a membership function / 2 In fuzzy clustering jargon, methods where strength of membership is zero or one are known as crisp methods. Fuzzy clustering has the advantage over crisp methods that memberships for any given object indicate whether there is a second best cluster that is almost as good as the best cluster. The concept of a membership function derives from fuzzy logic. The term fuzzy logic was introduced with the 1965 proposal of fuzzy set theory by Lotfi A. Zadeh. Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 4 / 20

6 Fuzzy clustering and fuzzy logic Fuzzy logic is an extension of Boolean logic, in which the concepts of true and false are replaced by that of partial truth. Let a 100 ml glass contain 30 ml of water and consider two concepts: empty and full. One person might define the glass as being 0.7 empty and 0.3 full. Another might design a membership function where the glass would be considered full for all values down to 50 ml. The connection between fuzzy cluster analysis and fuzzy logic is usually only through the application of membership functions, and not the more comprehensive theory. Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 5 / 20

7 Example of a membership function A fuzzy membership function for the verbal description of intelligence quotient (IQ). Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 6 / 20

8 2 Methods for fuzzy clustering Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 7 / 20

9 Fuzzy partitioning versus hard partitioning Given a data matrix X R n p of n observations measured on p variables, a partition (membership) matrix U = (u ik ) R n K is sought such that 1 u ik [0, 1] (i = 1,..., n; k = 1,..., K ); 2 K u ik = 1 k=1 (i = 1,..., n); 3 0 < n u ik < n (k = 1,..., K ). i=1 In hard partitioning, the first condition above would be replaced by u ik {0, 1} (i = 1,..., n; k = 1,..., K ). Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 8 / 20

10 Fuzzy k-means (fuzzy c-means) The fuzzy k-means algorithm minimizes K k=1 i=1 n u ν ik d 2 (x i, m k ), where m k is the centre of cluster k, d(x i, m k ) are Euclidean distances between the data point and the cluster centres, and ν is the fuzzifier that affects the membership distribution. The cluster centres are weighted cluster means: m k = n n u ν ik x i/ u ν ik. i=1 i=1 If the u ik are restricted to zero or one, the k-means method is obtained. Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 9 / 20

11 FANNY (Fuzzy analysis) Given a proximity matrix D = (d ij ) R n n and number of clusters K, the unknown membership strengths, u ik, are found by minimizing the objective function K k=1 n u ν ik uν jk d ij i,j=1 2 n u ν lk l=1, where ν is the membership exponent. The objective function is minimized subject to the nonnegativity and unit sum restrictions by using an iterative algorithm. FANNY only requires a proximity matrix and is relatively robust to nonspherical clusters. Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 10 / 20

12 3 The assessment of fuzzy clustering Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 11 / 20

13 The silhouette plot Assume the data have been clustered via any technique. For each datum i (i = 1,..., n), let a(i) be the average dissimilarity of i with all other data within the same cluster. Then find the average dissimilarity of i with the data of another single cluster. Repeat this for every cluster of which i is not a member. Denote the lowest average dissimilarity to i of any such cluster by b(i). The cluster with this lowest average dissimilarity is the neighbouring cluster of i as it is, aside from the cluster i is assigned, the cluster in which i fits best. Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 12 / 20

14 The silhouette plot / 2 Define s(i) = which can be written as s(i) = b(i) a(i) max{a(i), b(i)}, 1 a(i) b(i) : if a(i) < b(i) 0 : if a(i) = b(i). b(i) a(i) 1 : if a(i) > b(i) It holds that 1 s(i) 1 for all i = 1,..., n. An s(i) close to 1 means that the datum is appropriately clustered. If s(i) is close to 1, then it would be more appropriate if it was clustered in its neighbouring cluster. When the index is near zero it is not clear whether the object should have been assigned to its current cluster or its neighbouring cluster. Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 13 / 20

15 Silhouette plot of Iris data clusters n = clusters C j j : n j ave i Cj s i 1 : : : Average silhouette width : 0.57 Silhouette width s i Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 14 / 20

16 Dunn s partition coefficient Dunn s partition coefficient (Dunn, 1974) is a criterion for assessing the strength of membership. It can be computed as D K = n K i=1 k=1 u 2 ik n (D K [1/K, 1]). When normalized to lie in the range [0,1], it has the form K D K 1 K 1. Dunn s coefficient and silhouette plots give information to allow a number of clusters to be chosen so that a balance can be struck in the degree of fuzziness in different clusters. Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 15 / 20

17 4 Software for fuzzy clustering Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 16 / 20

18 User-written packages for the open-source software R cluster: Cluster Analysis (extended original from Peter J. Rousseeuw, Anja Struyf and Mia Hubert), written by Martin Maechler, Anja Struyf and Mia Hubert. This package is part of the base R distribution. Function fanny(). e1071: Miscellaneous functions of the Department of Statistics (e1071), TU Wien, written by David Meyer, Evgenia Dimitriadou, Kurt Hornik, Andreas Weingessel and Friedrich Leisch. Function cmeans(). Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 17 / 20

19 5 Example (R code: fcim-examples-03fuzzy.r) Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 18 / 20

20 6 Literature Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 19 / 20

21 Book Kaufman, L. and Rousseeuw, P. J. (2005): Finding Groups in Data: An Introduction to Cluster Analysis, Wiley. Chapter 4: Fuzzy Analysis (Program FANNY) Steffen Unkel c Sommersemester 2013 Fortgeschrittene Computerintensive Methoden: Fuzzy Clustering 20 / 20