Unsupervised learning: Clustering

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Unsupervised learning: Clustering"

Transcription

1 Unsupervised learning: Clustering Salissou Moutari Centre for Statistical Science and Operational Research CenSSOR 17 th September 2013 Unsupervised learning: Clustering 1/52

2 Outline 1 Introduction What is Unsupervised learning? Fundamental aspects of clustering 2 Clustering algorithms Hierarchical clustering Partitional clustering 3 Clustering evaluation metrics Unsupervised learning: Clustering h mn 2/52

3 Introduction What is Unsupervised learning? What is Unsupervised learning? Problem Given a set of records (e.g. observations or variables) with no target attribute, organise them into groups, without advance knowledge of the definitions of the groups. Unsupervised learning Unsupervised learning consists of approaches, which attempt to address the above problem by exploring the unlabelled data to find some intrinsic natural structures within them. Unsupervised learning: Clustering h mn 3/52

4 Introduction What is Unsupervised learning? What is Unsupervised learning? Problem Given a set of records (e.g. observations or variables) with no target attribute, organise them into groups, without advance knowledge of the definitions of the groups. Unsupervised learning Unsupervised learning consists of approaches, which attempt to address the above problem by exploring the unlabelled data to find some intrinsic natural structures within them. Unsupervised learning: Clustering h mn 3/52

5 Introduction What is Unsupervised learning? What is Unsupervised learning? Examples of unsupervised learning approaches Clustering Self-organising maps Association rule Blind signal separation etc. This session will focus on Clustering. Why Clustering? Clustering is one of the most utilised unsupervised learning techniques. Unsupervised learning: Clustering h mn 4/52

6 Introduction What is Unsupervised learning? What is Unsupervised learning? Examples of unsupervised learning approaches Clustering Self-organising maps Association rule Blind signal separation etc. This session will focus on Clustering. Why Clustering? Clustering is one of the most utilised unsupervised learning techniques. Unsupervised learning: Clustering h mn 4/52

7 Introduction What is Unsupervised learning? What is Unsupervised learning? Examples of unsupervised learning approaches Clustering Self-organising maps Association rule Blind signal separation etc. This session will focus on Clustering. Why Clustering? Clustering is one of the most utilised unsupervised learning techniques. Unsupervised learning: Clustering h mn 4/52

8 Introduction What is Unsupervised learning? What is Unsupervised learning? Examples of unsupervised learning approaches Clustering Self-organising maps Association rule Blind signal separation etc. This session will focus on Clustering. Why Clustering? Clustering is one of the most utilised unsupervised learning techniques. Unsupervised learning: Clustering h mn 4/52

9 Introduction Fundamental aspects of clustering Fundamental aspects of clustering Definition Clustering, also termed Cluster Analysis is the collection of methods for grouping unlabelled data into subsets (called clusters) that are believed to reflect the underlying structure of the data, based on similarity groups within the data. What is clustering for? Identification of new tumor classes using gene expression profiles; Identification of groups of co-regulated genes, e.g. using a large number of yeast experiments; Grouping similar proteins together with respect to their chemical structure and/or functionality etc; Detect experimental artifacts. Unsupervised learning: Clustering h mn 5/52

10 Introduction Fundamental aspects of clustering Fundamental aspects of clustering Definition Clustering, also termed Cluster Analysis is the collection of methods for grouping unlabelled data into subsets (called clusters) that are believed to reflect the underlying structure of the data, based on similarity groups within the data. What is clustering for? Identification of new tumor classes using gene expression profiles; Identification of groups of co-regulated genes, e.g. using a large number of yeast experiments; Grouping similar proteins together with respect to their chemical structure and/or functionality etc; Detect experimental artifacts. Unsupervised learning: Clustering h mn 5/52

11 Introduction Fundamental aspects of clustering Fundamental aspects of clustering Definition Clustering, also termed Cluster Analysis is the collection of methods for grouping unlabelled data into subsets (called clusters) that are believed to reflect the underlying structure of the data, based on similarity groups within the data. What is clustering for? Identification of new tumor classes using gene expression profiles; Identification of groups of co-regulated genes, e.g. using a large number of yeast experiments; Grouping similar proteins together with respect to their chemical structure and/or functionality etc; Detect experimental artifacts. Unsupervised learning: Clustering h mn 5/52

12 Introduction Fundamental aspects of clustering Fundamental aspects of clustering Basic concepts Clustering deals with data for which the groups are unknown and undefined. Thus we need to conceptualise the groups. Intra-clusters distance: Inter-clusters distance: Intra-cluster distance Inter-cluster distance Unsupervised learning: Clustering h mn 6/52

13 Challenges Introduction Fundamental aspects of clustering Notion of a Cluster can 1 Definition of the inter-cluster and intra-cluster distances. 2 The number of clusters. 3 The type of clusters. 4 Clusters quality. How many clusters? for these data? Unsupervised learning: Clustering h mn 7/52

14 Challenges Introduction Fundamental aspects of clustering Notion of a Cluster can 1 Definition of the inter-cluster and intra-cluster distances. 2 The number of clusters. 3 The type of clusters. 4 Clusters quality. How many clusters? for these data? Unsupervised learning: Clustering h mn 7/52

15 Introduction Fundamental aspects of clustering Challenges Two clusters? How many clusters? r can be Ambiguous Why not six clusters? Two Clusters Tan,Steinbach, Kumar Introduction to Data Mining Six Clusters Unsupervised learning: Clustering h mn 8/52

16 Challenges Introduction Fundamental aspects of clustering Definition of intra-clusters distance Type of distance measurement to be used to determine how close two data points are to each other. It is commonly called the distance, similarity or dissimilarity measure. Definition of inter-clusters distance Type of distance measurement to be used to determine how close two clusters are to each other. It is commonly called the linkage function or linkage criteria. It is is often both data (cluster shape) and context dependent and may depend on the distance measure. Unsupervised learning: Clustering h mn 9/52

17 Introduction Fundamental aspects of clustering Distance measures Fundamental axioms Assume that the data are in an n-dimensional Euclidean space, and let x =[x 1, x 2,...,x n ], y =[y 1, y 2,...,y n ]andz =[z 1, z 2,...,z n ]define three data points. Fundamental axioms of a distance measure d are: 1 d(x, x) =0 2 d(x, y) =d(y, x) 3 d(x, y) apple d(x, z)+d(z, y) Remark The choice of a distance measure will influence the shape of the clusters, as some elements may be close to one another according to one distance and farther away according to another. Unsupervised learning: Clustering h mn 10 / 52

18 Introduction Fundamental aspects of clustering Distance measures Fundamental axioms Assume that the data are in an n-dimensional Euclidean space, and let x =[x 1, x 2,...,x n ], y =[y 1, y 2,...,y n ]andz =[z 1, z 2,...,z n ]define three data points. Fundamental axioms of a distance measure d are: 1 d(x, x) =0 2 d(x, y) =d(y, x) 3 d(x, y) apple d(x, z)+d(z, y) Remark The choice of a distance measure will influence the shape of the clusters, as some elements may be close to one another according to one distance and farther away according to another. Unsupervised learning: Clustering h mn 10 / 52

19 Distance measures Introduction Fundamental aspects of clustering Examples of distance metrics Some commonly used metrics for clustering include: Euclidian distance (L 2 norm): d(x, y) = p P n i=1 (x i y i ) 2 nx Manhattan distance (L 1 norm): d(x, y) = kx i y i k i=1 Chebychev maximum distance (L 1 norm): d(x, y) = Minkowski distance (L p norm): d(x, y) = max i=1,...,n kx i! 1/p nx kx i y i k p Mahalanobis distance: d(x, y) = p P n i=1 (x i y i )R 1 (x i y i ), where R denotes the covariance matrix associated to the data. i=1 y i k Unsupervised learning: Clustering h mn 11 / 52

20 Linkage criteria Introduction Fundamental aspects of clustering Examples of linkage criteria or linkage functions Let C 1 and C 2 be two candidate clusters and let d be the chosen distance metric. Commonly used linkage functions between C 1 and C2 include: Single linkage: f (C 1, C 2 )=min{d(x, y) : x 2 C 1, y 2 C 2 } Complete linkage: f (C 1, C 2 )=max{d(x, y) : x 2 C 1, y 2 C 2 } 1 X X Average linkage: f (C 1, C 2 )= d(x, y) C 1 C 2 x2c 1 y2c 2 Ward s criterion: The distance between C 1 and C 2 is given by where µ i is the centre of cluster i. f (C 1, C 2 )= C 1 C 2 C 1 + C 2 µ 1 µ 2 2, Unsupervised learning: Clustering h mn 12 / 52

21 Clustering algorithms Clustering algorithms Hierarchical clustering Create a hierarchical decomposition of a data set by finding successive clusters using previously established clusters. Hierarchical clustering methods produce a tree diagram known as dendrogram or phenogram, which can be built in two distinct ways: Bottom-up known as Agglomerative clustering and Top-down called Divisive clustering. Partitional clustering Decompose the data set into a set of disjoint clusters, i.e. a set of non-overlapping clusters such that each data point is in exactly one subset cluster. Unsupervised learning: Clustering h mn 13 / 52

22 Clustering algorithms Clustering algorithms Hierarchical clustering Create a hierarchical decomposition of a data set by finding successive clusters using previously established clusters. Hierarchical clustering methods produce a tree diagram known as dendrogram or phenogram, which can be built in two distinct ways: Bottom-up known as Agglomerative clustering and Top-down called Divisive clustering. Partitional clustering Decompose the data set into a set of disjoint clusters, i.e. a set of non-overlapping clusters such that each data point is in exactly one subset cluster. Unsupervised learning: Clustering h mn 13 / 52

23 Clustering algorithms Hierarchical clustering Hierarchical clustering Agglomerative clustering Start with the points as individual clusters; At each step, merge the closest pair of clusters until all the data points are in a single cluster or until certain termination conditions are satisfied. Divisive clustering Start with one, all-inclusive cluster; At each step, split a cluster until each cluster contains a single data point or until certain termination conditions are satisfied. Unsupervised learning: Clustering h mn 14 / 52

24 Clustering algorithms Hierarchical clustering Hierarchical clustering Agglomerative clustering Start with the points as individual clusters; At each step, merge the closest pair of clusters until all the data points are in a single cluster or until certain termination conditions are satisfied. Divisive clustering Start with one, all-inclusive cluster; At each step, split a cluster until each cluster contains a single data point or until certain termination conditions are satisfied. Unsupervised learning: Clustering h mn 14 / 52

25 Clustering algorithms Hierarchical clustering Agglomerative clustering Algorithm The algorithm forms clusters in a bottom-up manner, as follows: 1 Initially, put each data point in its own cluster. 2 Among all current clusters, pick the two clusters which optimise the chosen linkage function. 3 Replace these two clusters with a new cluster, formed by merging the two original ones. 4 Repeat the steps 2 and 3 until there is only one remaining cluster in the pool, or until certain termination conditions are satisfied. Unsupervised learning: Clustering h mn 15 / 52

26 Clustering algorithms Hierarchical clustering Agglomerative clustering: Illustration with R Distance measure The function dist(x, method="metric") returns the distance matrix of anumericalmatrixx using a specified metric, which must be one of the followings: "euclidean", "maximum", "manhattan", "canberra", "binary" or "minkowski". Clustering The function hclust(d, method="linkage") performs hierarchical agglomerative clustering using a given distance matrix d and a specified linkage function, which must be one of the followings: "single", "complete", "average", "mcquitty", "median" or "centroid". Unsupervised learning: Clustering h mn 16 / 52

27 Clustering algorithms Hierarchical clustering Agglomerative clustering: Illustration with R Let us consider the following data set X : Unsupervised learning: Clustering h mn 17 / 52

28 Clustering algorithms Hierarchical clustering Agglomerative clustering: Illustration with R Rscript library(stats) d<-dist(x, method="euclidean") hc<-hclust(d, method="single") ggdendrogram(hc, theme dendro=false) Agglomerative clustering using euclidian distance measure and single linkage. Unsupervised learning: Clustering h mn 18 / 52

29 Clustering algorithms Hierarchical clustering Agglomerative clustering: Illustration with R Single linkage: Impact of the choice of the distance measure. Euclidian distance Chebychev distance Unsupervised learning: Clustering h mn 19 / 52

30 Clustering algorithms Hierarchical clustering Agglomerative clustering: Illustration with R Complete linkage: Impact of the choice of the distance measure. Euclidian distance Chebychev distance Unsupervised learning: Clustering h mn 20 / 52

31 Clustering algorithms Hierarchical clustering Agglomerative clustering: Illustration with R Average linkage: Impact of the choice of the distance measure. Euclidian distance Chebychev distance Unsupervised learning: Clustering h mn 21 / 52

32 Clustering algorithms Hierarchical clustering Agglomerative clustering: Illustration with R Euclidean distance: Impact of the choice of the linkage function. Single linkage Complete linkage Average linkage Unsupervised learning: Clustering h mn 22 / 52

33 Clustering algorithms Hierarchical clustering Agglomerative clustering: Illustration with R Chebychev distance: Impact of the choice of the linkage function. Single linkage Complete linkage Average linkage Unsupervised learning: Clustering h mn 23 / 52

34 Clustering algorithms Hierarchical clustering Agglomerative clustering Advantages No apriori information about the number of clusters required; Easy to implement; The obtained results may correspond to meaningful taxonomies. Limitations The algorithm does not enable to undo what was done previously Interpretation of the hierarchy can be complex or even confusing Depending on the type of distance matrix used, the algorithm 1 can be sensitivity to noise and outliers, 2 tends to break large clusters. 3 can hardly handle di erent sized clusters. Unsupervised learning: Clustering h mn 24 / 52

35 Clustering algorithms Hierarchical clustering Agglomerative clustering Advantages No apriori information about the number of clusters required; Easy to implement; The obtained results may correspond to meaningful taxonomies. Limitations The algorithm does not enable to undo what was done previously Interpretation of the hierarchy can be complex or even confusing Depending on the type of distance matrix used, the algorithm 1 can be sensitivity to noise and outliers, 2 tends to break large clusters. 3 can hardly handle di erent sized clusters. Unsupervised learning: Clustering h mn 24 / 52

36 Clustering algorithms Hierarchical clustering Divisive clustering Algorithm The algorithm forms clusters in a up-down manner, as follows: 1 Initially, put all objects in one cluster. 2 Among all current clusters, pick the one which satisfies a specified criterion and split it using a specified method. 3 Replace this cluster with the new clusters, formed by splitting the original one. 4 Repeat the steps 2 and 3 until all clusters are singletons or or until certain termination conditions are satisfied. Unsupervised learning: Clustering h mn 25 / 52

37 Clustering algorithms Hierarchical clustering Divisive clustering: Illustration with R Clustering The function diana(x, diss = inherits(x, "dist"), metric = "metric") performs hierarchical divisive clustering a numerical matrix X using a specified distance metric, which must be one of the followings: "euclidean" or "manhattan". Let us consider the following data set X : Unsupervised learning: Clustering h mn 26 / 52

38 Clustering algorithms Hierarchical clustering Divisive clustering: Illustration with R Rscript library(cluster) dc<-diana(x, diss=inherits(x, "dist"), metric="euclidean") plot(dc) Divisive clustering using euclidian distance measure. Unsupervised learning: Clustering h mn 27 / 52

39 Clustering algorithms Hierarchical clustering Divisive clustering: Illustration with R Impact of the choice of the distance measure Euclidian distance Manhattan distance Unsupervised learning: Clustering h mn 28 / 52

40 Divisive clustering Clustering algorithms Hierarchical clustering Advantages No apriori information about the number of clusters required; The obtained result may correspond to meaningful taxonomies. Limitations The algorithm does not enable to undo what was done previously; Computational di culties when considering all possible divisions into two groups; Depending on the type of distance matrix used, the algorithm 1 can be sensitivity to noise and outliers 2 tends to break large clusters Unsupervised learning: Clustering h mn 29 / 52

41 Clustering algorithms Partitional clustering Partitional clustering Basic concept Given, k the number of clusters, partitional clustering algorithms construct a partition of a data set into k clusters that optimises the chosen partitioning criterion. Partitionning techniques 1 Global optimal method: Exhaustive enumeration of all partitions (NP hard problem) 2 Heuristic methods: e.g. k-means clustering Each cluster is represented by its centre k-medoids clustering or PAM (Partition Around Medoids): Each cluster is represented by one of its components Unsupervised learning: Clustering h mn 30 / 52

42 Clustering algorithms Partitional clustering Partitional clustering Basic concept Given, k the number of clusters, partitional clustering algorithms construct a partition of a data set into k clusters that optimises the chosen partitioning criterion. Partitionning techniques 1 Global optimal method: Exhaustive enumeration of all partitions (NP hard problem) 2 Heuristic methods: e.g. k-means clustering Each cluster is represented by its centre k-medoids clustering or PAM (Partition Around Medoids): Each cluster is represented by one of its components Unsupervised learning: Clustering h mn 30 / 52

43 k-means clustering Clustering algorithms Partitional clustering Basic concept Given an integer k asetx of n points (n Euclidean space, denoted by k) in a m-dimensional X = {x i =(x i1,...,x im ) T 2 R m, i =1,...,n}. Find an assignment of the n points into k disjoint clusters C =(C 1,...,C k ) centered at cluster means µ j (j =1,...,k), based on a certain criteria, e.g. by minimising f (X, C) = kx X C j j=1 i=1 x (j) i µ j 2, where C j is the number of points in the cluster C j,andx (j) i in C j. is the point i Unsupervised learning: Clustering h mn 31 / 52

44 Clustering algorithms Partitional clustering k-means clustering Algorithm The k-means clustering algorithm can be summarised as follows: 1 Select k data points randomly in a domain containing all the points in the data set. These k points represent the centres of the initial clusters. 2 Assign each point to the cluster that has the closest centre. 3 Recompute the cluster centers (means) using the current cluster memberships. 4 Repeat the steps 2 and 3 until the centres no longer change, or until certain termination conditions are satisfied. Unsupervised learning: Clustering h mn 32 / 52

45 Clustering algorithms Partitional clustering k-means clustering: Illustration with R Clustering The function kmeans(x, centers, iter.max = 1000, nstart = 10) performs k-means clustering given a numerical matrix of data x, the maximum number of iterations, and the number of random initial sets to be chosen when centres is greater than 1. Let us consider the following data set X : Unsupervised learning: Clustering h mn 33 / 52

46 Clustering algorithms Partitional clustering k-mean clustering: Illustration with R Rscript library(stats) kc <- kmeans(x, centers= 4, iter.max=1000, nstart=10000) k-mean clustering using four clusters. Unsupervised learning: Clustering h mn 34 / 52

47 Clustering algorithms Partitional clustering k-mean clustering: Illustration with R Impact of the choice of the number of clusters Three clusters Four clusters Unsupervised learning: Clustering h mn 35 / 52

48 Clustering algorithms Partitional clustering k-mean clustering: Illustration with R Impact of the choice of the number of clusters Five clusters Six clusters Unsupervised learning: Clustering h mn 36 / 52

49 Clustering algorithms Partitional clustering k-mean Clustering: Illustration with R Impact of the choice of the number of clusters Number of clusters vs Within clusters sum of squares. Unsupervised learning: Clustering h mn 37 / 52

50 Clustering algorithms Partitional clustering k-mean clustering Advantages Relatively easy to implement. A simple iterative algorithm works quite well in practice. Limitations Need to specify k, the number of clusters, in advance. Applicable only when the mean is defined, hence it can t handle categorical data. Not suitable to discover clusters with non-convex shapes. Unable to handle noisy data and outliers. Unsupervised learning: Clustering h mn 38 / 52

51 k-medoids clustering Clustering algorithms Partitional clustering Basic concept Given an integer k asetx of n points (n Euclidean space, denoted by k) in a m-dimensional X = {x i =(x i1,...,x im ) T 2 R m, i =1,...,n}. Find an assignment of the n points into k disjoint clusters C =(C 1,...,C k ) centered at cluster points m j (j =1,...,k) called medoids, based on a certain criteria, e.g. by minimising f (X, C) = kx X C j j=1 i=1 x (j) i m j, where C j is the number of points in the cluster C j,andx (j) i in C j. is the point i Unsupervised learning: Clustering h mn 39 / 52

52 Clustering algorithms Partitional clustering k-medoids clustering PAM (Partitioning Around Medoids) Algorithm The PAM is a k-medoids clustering algorithm, which is similar to the k-means algorithm. It can be summarised as follows: 1 Select randomly k data points from the given data set. These k points represent the medoids of the initial clusters. 2 Assign each point to the cluster that has the closest medoid. 3 Iteratively replace one of the medoids by one of the non-medoids which improve the chosen criterion. 4 Repeat the steps 2 and 3 until the medoids no longer change, or until certain termination conditions are satisfied. Unsupervised learning: Clustering h mn 40 / 52

53 Clustering algorithms Partitional clustering k-medoids clustering PAM Algorithm Advantages: Works e ectively for small data sets Limitations: Does not scale well for large data sets CLARA (Clustering Large Applications) Based on multiple sampling from the data set and application of PAM on each sample, it provides the best clustering as the output. Advantages: Deals with larger data sets than PAM Limitations: E ciency depends on the sample size Unsupervised learning: Clustering h mn 41 / 52

54 Clustering algorithms Partitional clustering k-medoids clustering: Illustration with R CLARA The function clara(x, k, metric = "metric", samples = r) performs CLARA clustering given a numerical matrix of data x, the number of cluster, the distance metric, and the number of samples to be drawn from the data set X. Let us consider the following data set X : Unsupervised learning: Clustering h mn 42 / 52

55 Clustering algorithms Partitional clustering k-medoids clustering: Illustration with R Rscript library(cluster) km <- clara(x, k, metric = "euclidean", samples = 10) CLARA clustering using 5 clusters and 10 samples. Unsupervised learning: Clustering h mn 43 / 52

56 Clustering algorithms Partitional clustering k-medoids clustering: Illustration with R CLARA: Impact of the choice of the distance metric Euclidean distance Manhattan distance Unsupervised learning: Clustering h mn 44 / 52

57 Clustering evaluation metrics So... which method to use for the data set X?!!!?? Hierarchical clustering? If yes Agglomerative or Divisive? For either method 1 which metric distance and/or linkage function? 2 where to cut the dendrogram? Partitional clustering? If yes k-means or CLARA? For either method 1 which metric distance? 2 how many clusters? Unsupervised learning: Clustering h mn 45 / 52

58 Clustering evaluation metrics So... which method to use for the data set X?!!!?? Hierarchical clustering? If yes Agglomerative or Divisive? For either method 1 which metric distance and/or linkage function? 2 where to cut the dendrogram? Partitional clustering? If yes k-means or CLARA? For either method 1 which metric distance? 2 how many clusters? Unsupervised learning: Clustering h mn 45 / 52

59 Clustering evaluation metrics So... which method to use for the data set X?!!!?? Hierarchical clustering? If yes Agglomerative or Divisive? For either method 1 which metric distance and/or linkage function? 2 where to cut the dendrogram? Partitional clustering? If yes k-means or CLARA? For either method 1 which metric distance? 2 how many clusters? Unsupervised learning: Clustering h mn 45 / 52

60 Clustering evaluation metrics So... which method to use for the data set X?!!!?? Hierarchical clustering? If yes Agglomerative or Divisive? For either method 1 which metric distance and/or linkage function? 2 where to cut the dendrogram? Partitional clustering? If yes k-means or CLARA? For either method 1 which metric distance? 2 how many clusters? Unsupervised learning: Clustering h mn 45 / 52

61 Clustering evaluation metrics So... which method to use for the data set X?!!!?? Hierarchical clustering? If yes Agglomerative or Divisive? For either method 1 which metric distance and/or linkage function? 2 where to cut the dendrogram? Partitional clustering? If yes k-means or CLARA? For either method 1 which metric distance? 2 how many clusters? Unsupervised learning: Clustering h mn 45 / 52

62 Clustering evaluation metrics Clustering evaluation metrics Silhouette Coe cient Provides a graphical representation of how well each object lies within its cluster. The silhouette coe cient of a data point i is defined as s i = (b i a i ) max(a i, b i ), where a i denotes the average distance between the data point i and all other data points in its cluster, and b i denotes the minimum average distance between i and the data points in other clusters. Data points with large silhouette coe cient s i are well-clustered, those with small s i tend to lie between clusters. Unsupervised learning: Clustering h mn 46 / 52

63 Clustering evaluation metrics Clustering evaluation metrics Classification-oriented measures Use of the classification approach to compare clustering techniques with the ground truth. Some of these measures are 1 Entropy 2 Purity 3 Recall 4 F -measure Unsupervised learning: Clustering h mn 47 / 52

64 Clustering evaluation metrics Clustering evaluation metrics Entropy Measures the degree to which each cluster consists of data points from a single class. The entropy of a cluster i is given by E i = lx j=1 n ij n i log nij n i, where n ij is the number of data points of class i in cluster j, n i is the number of data points in cluster i and l is the number of classes. The total entropy for a set of clusters is given by E = kx i=1 n i n E i, where k is the number of clusters and n is the total number of data points. Unsupervised learning: Clustering h mn 48 / 52

65 Clustering evaluation metrics Clustering evaluation metrics Purity Measures the extent to which a cluster contains data points of a single class. Using the previous notations, the purity for a cluster i is given by Pur i =max j n ij n i, whereas the overall purity of a clustering is given by Pur = kx i=1 n i n Pur i. Unsupervised learning: Clustering h mn 49 / 52

66 Clustering evaluation metrics Clustering evaluation metrics Precision Measures the fraction of a cluster that consists of objects of a specified class. Using the previous notations, the precision of cluster i with respect to class j is given by Pre(i, j) = n ij n i Recall Measures the extent to which a cluster contains all objects of a specified class. The recall of cluster i with respect to class j is given by Rec(i, j) = n ij n j, where n ij is the number of data points of class i in cluster j and n j is the number of data points in class j. Unsupervised learning: Clustering h mn 50 / 52

67 Clustering evaluation metrics Clustering evaluation metrics F -measure It combines the precision and the recall to measure the extent to which a cluster contains only data points of a particular class and all points of that class. The F -measure of cluster i with respect to class j is given by F (i, j) = 2Pre(i, j) Rec(i, j) Pre(i, j)+rec(i, j). Unsupervised learning: Clustering h mn 51 / 52

68 End End Thank you for your attention! Unsupervised learning: Clustering h mn 52 / 52

Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016

Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with

More information

Cluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico

Cluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from

More information

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDD-LAB ISTI- CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar

More information

Neural Networks Lesson 5 - Cluster Analysis

Neural Networks Lesson 5 - Cluster Analysis Neural Networks Lesson 5 - Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt. - Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29

More information

Clustering UE 141 Spring 2013

Clustering UE 141 Spring 2013 Clustering UE 141 Spring 013 Jing Gao SUNY Buffalo 1 Definition of Clustering Finding groups of obects such that the obects in a group will be similar (or related) to one another and different from (or

More information

Data Mining Clustering (2) Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining

Data Mining Clustering (2) Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining Data Mining Clustering (2) Toon Calders Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining Outline Partitional Clustering Distance-based K-means, K-medoids,

More information

Clustering. Adrian Groza. Department of Computer Science Technical University of Cluj-Napoca

Clustering. Adrian Groza. Department of Computer Science Technical University of Cluj-Napoca Clustering Adrian Groza Department of Computer Science Technical University of Cluj-Napoca Outline 1 Cluster Analysis What is Datamining? Cluster Analysis 2 K-means 3 Hierarchical Clustering What is Datamining?

More information

Text Clustering. Clustering

Text Clustering. Clustering Text Clustering 1 Clustering Partition unlabeled examples into disoint subsets of clusters, such that: Examples within a cluster are very similar Examples in different clusters are very different Discover

More information

Chapter 7. Cluster Analysis

Chapter 7. Cluster Analysis Chapter 7. Cluster Analysis. What is Cluster Analysis?. A Categorization of Major Clustering Methods. Partitioning Methods. Hierarchical Methods 5. Density-Based Methods 6. Grid-Based Methods 7. Model-Based

More information

Clustering & Association

Clustering & Association Clustering - Overview What is cluster analysis? Grouping data objects based only on information found in the data describing these objects and their relationships Maximize the similarity within objects

More information

Cluster Analysis using R

Cluster Analysis using R Cluster analysis or clustering is the task of assigning a set of objects into groups (called clusters) so that the objects in the same cluster are more similar (in some sense or another) to each other

More information

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/2004 Hierarchical

More information

Example: Document Clustering. Clustering: Definition. Notion of a Cluster can be Ambiguous. Types of Clusterings. Hierarchical Clustering

Example: Document Clustering. Clustering: Definition. Notion of a Cluster can be Ambiguous. Types of Clusterings. Hierarchical Clustering Overview Prognostic Models and Data Mining in Medicine, part I Cluster Analsis What is Cluster Analsis? K-Means Clustering Hierarchical Clustering Cluster Validit Eample: Microarra data analsis 6 Summar

More information

Distance based clustering

Distance based clustering // Distance based clustering Chapter ² ² Clustering Clustering is the art of finding groups in data (Kaufman and Rousseeuw, 99). What is a cluster? Group of objects separated from other clusters Means

More information

Cluster Analysis: Basic Concepts and Algorithms

Cluster Analysis: Basic Concepts and Algorithms Cluster Analsis: Basic Concepts and Algorithms What does it mean clustering? Applications Tpes of clustering K-means Intuition Algorithm Choosing initial centroids Bisecting K-means Post-processing Strengths

More information

CLASSIFICATION AND CLUSTERING. Anveshi Charuvaka

CLASSIFICATION AND CLUSTERING. Anveshi Charuvaka CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training

More information

Clustering. 15-381 Artificial Intelligence Henry Lin. Organizing data into clusters such that there is

Clustering. 15-381 Artificial Intelligence Henry Lin. Organizing data into clusters such that there is Clustering 15-381 Artificial Intelligence Henry Lin Modified from excellent slides of Eamonn Keogh, Ziv Bar-Joseph, and Andrew Moore What is Clustering? Organizing data into clusters such that there is

More information

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 by Tan, Steinbach, Kumar 1 What is Cluster Analysis? Finding groups of objects such that the objects in a group will

More information

L15: statistical clustering

L15: statistical clustering Similarity measures Criterion functions Cluster validity Flat clustering algorithms k-means ISODATA L15: statistical clustering Hierarchical clustering algorithms Divisive Agglomerative CSCE 666 Pattern

More information

Fig. 1 A typical Knowledge Discovery process [2]

Fig. 1 A typical Knowledge Discovery process [2] Volume 4, Issue 7, July 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Review on Clustering

More information

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining /8/ What is Cluster

More information

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 http://intelligentoptimization.org/lionbook Roberto Battiti

More information

Information Retrieval and Web Search Engines

Information Retrieval and Web Search Engines Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 10 th, 2013 Wolf-Tilo Balke and Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig

More information

Social Media Mining. Data Mining Essentials

Social Media Mining. Data Mining Essentials Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

More information

K-Means Cluster Analysis. Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1

K-Means Cluster Analysis. Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 K-Means Cluster Analsis Chapter 3 PPDM Class Tan,Steinbach, Kumar Introduction to Data Mining 4/18/4 1 What is Cluster Analsis? Finding groups of objects such that the objects in a group will be similar

More information

Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu

Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?

More information

Chapter ML:XI (continued)

Chapter ML:XI (continued) Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained

More information

UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS

UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS Dwijesh C. Mishra I.A.S.R.I., Library Avenue, New Delhi-110 012 dcmishra@iasri.res.in What is Learning? "Learning denotes changes in a system that enable

More information

Cluster Analysis: Basic Concepts and Methods

Cluster Analysis: Basic Concepts and Methods 10 Cluster Analysis: Basic Concepts and Methods Imagine that you are the Director of Customer Relationships at AllElectronics, and you have five managers working for you. You would like to organize all

More information

Statistical Databases and Registers with some datamining

Statistical Databases and Registers with some datamining Unsupervised learning - Statistical Databases and Registers with some datamining a course in Survey Methodology and O cial Statistics Pages in the book: 501-528 Department of Statistics Stockholm University

More information

Distances, Clustering, and Classification. Heatmaps

Distances, Clustering, and Classification. Heatmaps Distances, Clustering, and Classification Heatmaps 1 Distance Clustering organizes things that are close into groups What does it mean for two genes to be close? What does it mean for two samples to be

More information

Data Mining Project Report. Document Clustering. Meryem Uzun-Per

Data Mining Project Report. Document Clustering. Meryem Uzun-Per Data Mining Project Report Document Clustering Meryem Uzun-Per 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. K-means algorithm...

More information

10-810 /02-710 Computational Genomics. Clustering expression data

10-810 /02-710 Computational Genomics. Clustering expression data 10-810 /02-710 Computational Genomics Clustering expression data What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally,

More information

Lecture 20: Clustering

Lecture 20: Clustering Lecture 20: Clustering Wrap-up of neural nets (from last lecture Introduction to unsupervised learning K-means clustering COMP-424, Lecture 20 - April 3, 2013 1 Unsupervised learning In supervised learning,

More information

Clustering. Data Mining. Abraham Otero. Data Mining. Agenda

Clustering. Data Mining. Abraham Otero. Data Mining. Agenda Clustering 1/46 Agenda Introduction Distance K-nearest neighbors Hierarchical clustering Quick reference 2/46 1 Introduction It seems logical that in a new situation we should act in a similar way as in

More information

An Introduction to Cluster Analysis for Data Mining

An Introduction to Cluster Analysis for Data Mining An Introduction to Cluster Analysis for Data Mining 10/02/2000 11:42 AM 1. INTRODUCTION... 4 1.1. Scope of This Paper... 4 1.2. What Cluster Analysis Is... 4 1.3. What Cluster Analysis Is Not... 5 2. OVERVIEW...

More information

Cluster Analysis: Basic Concepts and Algorithms

Cluster Analysis: Basic Concepts and Algorithms 8 Cluster Analysis: Basic Concepts and Algorithms Cluster analysis divides data into groups (clusters) that are meaningful, useful, or both. If meaningful groups are the goal, then the clusters should

More information

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Clustering Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Clustering Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Clustering Algorithms K-means and its variants Hierarchical clustering

More information

Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov

Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical Main

More information

Data Mining Clustering. Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining

Data Mining Clustering. Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining Data Mining Clustering Toon Calders Sheets are based on the those provided b Tan, Steinbach, and Kumar. Introduction to Data Mining What is Cluster Analsis? Finding groups of objects such that the objects

More information

Machine Learning using MapReduce

Machine Learning using MapReduce Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous

More information

Unsupervised Learning and Data Mining. Unsupervised Learning and Data Mining. Clustering. Supervised Learning. Supervised Learning

Unsupervised Learning and Data Mining. Unsupervised Learning and Data Mining. Clustering. Supervised Learning. Supervised Learning Unsupervised Learning and Data Mining Unsupervised Learning and Data Mining Clustering Decision trees Artificial neural nets K-nearest neighbor Support vectors Linear regression Logistic regression...

More information

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 What is

More information

ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING)

ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Preliminaries Classification and Clustering Applications

More information

Cluster Algorithms. Adriano Cruz adriano@nce.ufrj.br. 28 de outubro de 2013

Cluster Algorithms. Adriano Cruz adriano@nce.ufrj.br. 28 de outubro de 2013 Cluster Algorithms Adriano Cruz adriano@nce.ufrj.br 28 de outubro de 2013 Adriano Cruz adriano@nce.ufrj.br () Cluster Algorithms 28 de outubro de 2013 1 / 80 Summary 1 K-Means Adriano Cruz adriano@nce.ufrj.br

More information

There are a number of different methods that can be used to carry out a cluster analysis; these methods can be classified as follows:

There are a number of different methods that can be used to carry out a cluster analysis; these methods can be classified as follows: Statistics: Rosie Cornish. 2007. 3.1 Cluster Analysis 1 Introduction This handout is designed to provide only a brief introduction to cluster analysis and how it is done. Books giving further details are

More information

Robotics 2 Clustering & EM. Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard

Robotics 2 Clustering & EM. Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard Robotics 2 Clustering & EM Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard 1 Clustering (1) Common technique for statistical data analysis to detect structure (machine learning,

More information

Cluster analysis Cosmin Lazar. COMO Lab VUB

Cluster analysis Cosmin Lazar. COMO Lab VUB Cluster analysis Cosmin Lazar COMO Lab VUB Introduction Cluster analysis foundations rely on one of the most fundamental, simple and very often unnoticed ways (or methods) of understanding and learning,

More information

Clustering: Techniques & Applications. Nguyen Sinh Hoa, Nguyen Hung Son. 15 lutego 2006 Clustering 1

Clustering: Techniques & Applications. Nguyen Sinh Hoa, Nguyen Hung Son. 15 lutego 2006 Clustering 1 Clustering: Techniques & Applications Nguyen Sinh Hoa, Nguyen Hung Son 15 lutego 2006 Clustering 1 Agenda Introduction Clustering Methods Applications: Outlier Analysis Gene clustering Summary and Conclusions

More information

Comparison and Analysis of Various Clustering Methods in Data mining On Education data set Using the weak tool

Comparison and Analysis of Various Clustering Methods in Data mining On Education data set Using the weak tool Comparison and Analysis of Various Clustering Metho in Data mining On Education data set Using the weak tool Abstract:- Data mining is used to find the hidden information pattern and relationship between

More information

Flat Clustering K-Means Algorithm

Flat Clustering K-Means Algorithm Flat Clustering K-Means Algorithm 1. Purpose. Clustering algorithms group a set of documents into subsets or clusters. The cluster algorithms goal is to create clusters that are coherent internally, but

More information

Machine Learning and Data Mining. Clustering. (adapted from) Prof. Alexander Ihler

Machine Learning and Data Mining. Clustering. (adapted from) Prof. Alexander Ihler Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand

More information

Clustering Hierarchical clustering and k-mean clustering

Clustering Hierarchical clustering and k-mean clustering Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein The clustering problem: A quick review partition genes into distinct sets with high homogeneity

More information

Data Mining. Cluster Analysis: Advanced Concepts and Algorithms

Data Mining. Cluster Analysis: Advanced Concepts and Algorithms Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototype-based clustering Density-based clustering Graph-based

More information

Summary Data Mining & Process Mining (1BM46) Content. Made by S.P.T. Ariesen

Summary Data Mining & Process Mining (1BM46) Content. Made by S.P.T. Ariesen Summary Data Mining & Process Mining (1BM46) Made by S.P.T. Ariesen Content Data Mining part... 2 Lecture 1... 2 Lecture 2:... 4 Lecture 3... 7 Lecture 4... 9 Process mining part... 13 Lecture 5... 13

More information

Clustering and Data Mining in R

Clustering and Data Mining in R Clustering and Data Mining in R Workshop Supplement Thomas Girke December 10, 2011 Introduction Data Preprocessing Data Transformations Distance Methods Cluster Linkage Hierarchical Clustering Approaches

More information

Clustering Techniques: A Brief Survey of Different Clustering Algorithms

Clustering Techniques: A Brief Survey of Different Clustering Algorithms Clustering Techniques: A Brief Survey of Different Clustering Algorithms Deepti Sisodia Technocrates Institute of Technology, Bhopal, India Lokesh Singh Technocrates Institute of Technology, Bhopal, India

More information

. Learn the number of classes and the structure of each class using similarity between unlabeled training patterns

. Learn the number of classes and the structure of each class using similarity between unlabeled training patterns Outline Part 1: of data clustering Non-Supervised Learning and Clustering : Problem formulation cluster analysis : Taxonomies of Clustering Techniques : Data types and Proximity Measures : Difficulties

More information

Cluster Analysis: Advanced Concepts

Cluster Analysis: Advanced Concepts Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototype-based Fuzzy c-means

More information

An Enhanced Clustering Algorithm to Analyze Spatial Data

An Enhanced Clustering Algorithm to Analyze Spatial Data International Journal of Engineering and Technical Research (IJETR) ISSN: 2321-0869, Volume-2, Issue-7, July 2014 An Enhanced Clustering Algorithm to Analyze Spatial Data Dr. Mahesh Kumar, Mr. Sachin Yadav

More information

A comparison of various clustering methods and algorithms in data mining

A comparison of various clustering methods and algorithms in data mining Volume :2, Issue :5, 32-36 May 2015 www.allsubjectjournal.com e-issn: 2349-4182 p-issn: 2349-5979 Impact Factor: 3.762 R.Tamilselvi B.Sivasakthi R.Kavitha Assistant Professor A comparison of various clustering

More information

Standardization and Its Effects on K-Means Clustering Algorithm

Standardization and Its Effects on K-Means Clustering Algorithm Research Journal of Applied Sciences, Engineering and Technology 6(7): 399-3303, 03 ISSN: 040-7459; e-issn: 040-7467 Maxwell Scientific Organization, 03 Submitted: January 3, 03 Accepted: February 5, 03

More information

Hierarchical Cluster Analysis Some Basics and Algorithms

Hierarchical Cluster Analysis Some Basics and Algorithms Hierarchical Cluster Analysis Some Basics and Algorithms Nethra Sambamoorthi CRMportals Inc., 11 Bartram Road, Englishtown, NJ 07726 (NOTE: Please use always the latest copy of the document. Click on this

More information

Steven M. Ho!and. Department of Geology, University of Georgia, Athens, GA 30602-2501

Steven M. Ho!and. Department of Geology, University of Georgia, Athens, GA 30602-2501 CLUSTER ANALYSIS Steven M. Ho!and Department of Geology, University of Georgia, Athens, GA 30602-2501 January 2006 Introduction Cluster analysis includes a broad suite of techniques designed to find groups

More information

Data Clustering Techniques Qualifying Oral Examination Paper

Data Clustering Techniques Qualifying Oral Examination Paper Data Clustering Techniques Qualifying Oral Examination Paper Periklis Andritsos University of Toronto Department of Computer Science periklis@cs.toronto.edu March 11, 2002 1 Introduction During a cholera

More information

STATISTICA. Clustering Techniques. Case Study: Defining Clusters of Shopping Center Patrons. and

STATISTICA. Clustering Techniques. Case Study: Defining Clusters of Shopping Center Patrons. and Clustering Techniques and STATISTICA Case Study: Defining Clusters of Shopping Center Patrons STATISTICA Solutions for Business Intelligence, Data Mining, Quality Control, and Web-based Analytics Table

More information

PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA

PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA Prakash Singh 1, Aarohi Surya 2 1 Department of Finance, IIM Lucknow, Lucknow, India 2 Department of Computer Science, LNMIIT, Jaipur,

More information

Text Analytics. Text Clustering. Ulf Leser

Text Analytics. Text Clustering. Ulf Leser Text Analytics Text Clustering Ulf Leser Content of this Lecture (Text) clustering Cluster quality Clustering algorithms Application Ulf Leser: Text Analytics, Winter Semester 2010/2011 2 Clustering Clustering

More information

Territorial Analysis for Ratemaking. Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs

Territorial Analysis for Ratemaking. Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs Territorial Analysis for Ratemaking by Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs Department of Statistics and Applied Probability University

More information

Introduction to Statistical Machine Learning

Introduction to Statistical Machine Learning CHAPTER Introduction to Statistical Machine Learning We start with a gentle introduction to statistical machine learning. Readers familiar with machine learning may wish to skip directly to Section 2,

More information

SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING

SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING AAS 07-228 SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING INTRODUCTION James G. Miller * Two historical uncorrelated track (UCT) processing approaches have been employed using general perturbations

More information

Clustering Very Large Data Sets with Principal Direction Divisive Partitioning

Clustering Very Large Data Sets with Principal Direction Divisive Partitioning Clustering Very Large Data Sets with Principal Direction Divisive Partitioning David Littau 1 and Daniel Boley 2 1 University of Minnesota, Minneapolis MN 55455 littau@cs.umn.edu 2 University of Minnesota,

More information

EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set

EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set Amhmed A. Bhih School of Electrical and Electronic Engineering Princy Johnson School of Electrical and Electronic Engineering Martin

More information

Computational Complexity between K-Means and K-Medoids Clustering Algorithms for Normal and Uniform Distributions of Data Points

Computational Complexity between K-Means and K-Medoids Clustering Algorithms for Normal and Uniform Distributions of Data Points Journal of Computer Science 6 (3): 363-368, 2010 ISSN 1549-3636 2010 Science Publications Computational Complexity between K-Means and K-Medoids Clustering Algorithms for Normal and Uniform Distributions

More information

Personalized Hierarchical Clustering

Personalized Hierarchical Clustering Personalized Hierarchical Clustering Korinna Bade, Andreas Nürnberger Faculty of Computer Science, Otto-von-Guericke-University Magdeburg, D-39106 Magdeburg, Germany {kbade,nuernb}@iws.cs.uni-magdeburg.de

More information

Machine Learning for NLP

Machine Learning for NLP Natural Language Processing SoSe 2015 Machine Learning for NLP Dr. Mariana Neves May 4th, 2015 (based on the slides of Dr. Saeedeh Momtazi) Introduction Field of study that gives computers the ability

More information

Data Mining for Knowledge Management. Clustering

Data Mining for Knowledge Management. Clustering Data Mining for Knowledge Management Clustering Themis Palpanas University of Trento http://disi.unitn.eu/~themis Data Mining for Knowledge Management Thanks for slides to: Jiawei Han Eamonn Keogh Jeff

More information

Identification of noisy variables for nonmetric and symbolic data in cluster analysis

Identification of noisy variables for nonmetric and symbolic data in cluster analysis Identification of noisy variables for nonmetric and symbolic data in cluster analysis Marek Walesiak and Andrzej Dudek Wroclaw University of Economics, Department of Econometrics and Computer Science,

More information

HES-SO Master of Science in Engineering. Clustering. Prof. Laura Elena Raileanu HES-SO Yverdon-les-Bains (HEIG-VD)

HES-SO Master of Science in Engineering. Clustering. Prof. Laura Elena Raileanu HES-SO Yverdon-les-Bains (HEIG-VD) HES-SO Master of Science in Engineering Clustering Prof. Laura Elena Raileanu HES-SO Yverdon-les-Bains (HEIG-VD) Plan Motivation Hierarchical Clustering K-Means Clustering 2 Problem Setup Arrange items

More information

Cluster Analysis. Alison Merikangas Data Analysis Seminar 18 November 2009

Cluster Analysis. Alison Merikangas Data Analysis Seminar 18 November 2009 Cluster Analysis Alison Merikangas Data Analysis Seminar 18 November 2009 Overview What is cluster analysis? Types of cluster Distance functions Clustering methods Agglomerative K-means Density-based Interpretation

More information

Clustering in Machine Learning. By: Ibrar Hussain Student ID:

Clustering in Machine Learning. By: Ibrar Hussain Student ID: Clustering in Machine Learning By: Ibrar Hussain Student ID: 11021083 Presentation An Overview Introduction Definition Types of Learning Clustering in Machine Learning K-means Clustering Example of k-means

More information

Data Mining 資 料 探 勘. 分 群 分 析 (Cluster Analysis)

Data Mining 資 料 探 勘. 分 群 分 析 (Cluster Analysis) Data Mining 資 料 探 勘 Tamkang University 分 群 分 析 (Cluster Analysis) DM MI Wed,, (:- :) (B) Min-Yuh Day 戴 敏 育 Assistant Professor 專 任 助 理 教 授 Dept. of Information Management, Tamkang University 淡 江 大 學 資

More information

Data visualization and clustering. Genomics is to no small extend a data science

Data visualization and clustering. Genomics is to no small extend a data science Data visualization and clustering Genomics is to no small extend a data science [www.data2discovery.org] Data visualization and clustering Genomics is to no small extend a data science [Andersson et al.,

More information

Protein Protein Interaction Networks

Protein Protein Interaction Networks Functional Pattern Mining from Genome Scale Protein Protein Interaction Networks Young-Rae Cho, Ph.D. Assistant Professor Department of Computer Science Baylor University it My Definition of Bioinformatics

More information

Vector Quantization and Clustering

Vector Quantization and Clustering Vector Quantization and Clustering Introduction K-means clustering Clustering issues Hierarchical clustering Divisive (top-down) clustering Agglomerative (bottom-up) clustering Applications to speech recognition

More information

A Survey of Clustering Techniques

A Survey of Clustering Techniques A Survey of Clustering Techniques Pradeep Rai Asst. Prof., CSE Department, Kanpur Institute of Technology, Kanpur-0800 (India) Shubha Singh Asst. Prof., MCA Department, Kanpur Institute of Technology,

More information

SoSe 2014: M-TANI: Big Data Analytics

SoSe 2014: M-TANI: Big Data Analytics SoSe 2014: M-TANI: Big Data Analytics Lecture 4 21/05/2014 Sead Izberovic Dr. Nikolaos Korfiatis Agenda Recap from the previous session Clustering Introduction Distance mesures Hierarchical Clustering

More information

On Clustering Validation Techniques

On Clustering Validation Techniques Journal of Intelligent Information Systems, 17:2/3, 107 145, 2001 c 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques MARIA HALKIDI mhalk@aueb.gr YANNIS

More information

Data Mining Essentials

Data Mining Essentials This chapter is from Social Media Mining: An Introduction. By Reza Zafarani, Mohammad Ali Abbasi, and Huan Liu. Cambridge University Press, 2014. Draft version: April 20, 2014. Complete Draft and Slides

More information

Using Data Mining for Mobile Communication Clustering and Characterization

Using Data Mining for Mobile Communication Clustering and Characterization Using Data Mining for Mobile Communication Clustering and Characterization A. Bascacov *, C. Cernazanu ** and M. Marcu ** * Lasting Software, Timisoara, Romania ** Politehnica University of Timisoara/Computer

More information

Chapter 4 Data Mining A Short Introduction

Chapter 4 Data Mining A Short Introduction Chapter 4 Data Mining A Short Introduction Data Mining - 1 1 Today's Question 1. Data Mining Overview 2. Association Rule Mining 3. Clustering 4. Classification Data Mining - 2 2 3. Clustering - Descriptive

More information

Data Mining and Knowledge Discovery in Databases (KDD) State of the Art. Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland

Data Mining and Knowledge Discovery in Databases (KDD) State of the Art. Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland Data Mining and Knowledge Discovery in Databases (KDD) State of the Art Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland 1 Conference overview 1. Overview of KDD and data mining 2. Data

More information

USING THE AGGLOMERATIVE METHOD OF HIERARCHICAL CLUSTERING AS A DATA MINING TOOL IN CAPITAL MARKET 1. Vera Marinova Boncheva

USING THE AGGLOMERATIVE METHOD OF HIERARCHICAL CLUSTERING AS A DATA MINING TOOL IN CAPITAL MARKET 1. Vera Marinova Boncheva 382 [7] Reznik, A, Kussul, N., Sokolov, A.: Identification of user activity using neural networks. Cybernetics and computer techniques, vol. 123 (1999) 70 79. (in Russian) [8] Kussul, N., et al. : Multi-Agent

More information

Chapter 20: Data Analysis

Chapter 20: Data Analysis Chapter 20: Data Analysis Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 20: Data Analysis Decision Support Systems Data Warehousing Data Mining Classification

More information

Data Mining K-Clustering Problem

Data Mining K-Clustering Problem Data Mining K-Clustering Problem Elham Karoussi Supervisor Associate Professor Noureddine Bouhmala This Master s Thesis is carried out as a part of the education at the University of Agder and is therefore

More information

A Method for Decentralized Clustering in Large Multi-Agent Systems

A Method for Decentralized Clustering in Large Multi-Agent Systems A Method for Decentralized Clustering in Large Multi-Agent Systems Elth Ogston, Benno Overeinder, Maarten van Steen, and Frances Brazier Department of Computer Science, Vrije Universiteit Amsterdam {elth,bjo,steen,frances}@cs.vu.nl

More information

What is Cluster Analysis?

What is Cluster Analysis? What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping a set of data objects

More information

Chapter 4 Data Mining A Short Introduction. 2006/7, Karl Aberer, EPFL-IC, Laboratoire de systèmes d'informations répartis Data Mining - 1

Chapter 4 Data Mining A Short Introduction. 2006/7, Karl Aberer, EPFL-IC, Laboratoire de systèmes d'informations répartis Data Mining - 1 Chapter 4 Data Mining A Short Introduction 2006/7, Karl Aberer, EPFL-IC, Laboratoire de systèmes d'informations répartis Data Mining - 1 1 Today's Question 1. Data Mining Overview 2. Association Rule Mining

More information

Data Mining 5. Cluster Analysis

Data Mining 5. Cluster Analysis Data Mining 5. Cluster Analysis 5.2 Fall 2009 Instructor: Dr. Masoud Yaghini Outline Data Structures Interval-Valued (Numeric) Variables Binary Variables Categorical Variables Ordinal Variables Variables

More information

Clustering Connectionist and Statistical Language Processing

Clustering Connectionist and Statistical Language Processing Clustering Connectionist and Statistical Language Processing Frank Keller keller@coli.uni-sb.de Computerlinguistik Universität des Saarlandes Clustering p.1/21 Overview clustering vs. classification supervised

More information