Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Size: px
Start display at page:

Download "Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining"

Transcription

1 Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 What is Cluster Analsis? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from (or unrelated to) the objects in other groups Intra-cluster distances are minimized Inter-cluster distances are maimized Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4

2 Applications of Cluster Analsis Understanding Group related documents for browsing, group genes and proteins that have similar functionalit, or group stocks with similar price fluctuations 4 Discovered Clusters Applied-Matl-DOWN,Ba-Network-Down,-COM-DOWN, Cabletron-Ss-DOWN,CISCO-DOWN,HP-DOWN, DSC-Comm-DOWN,INTEL-DOWN,LSI-Logic-DOWN, Micron-Tech-DOWN,Teas-Inst-Down,Tellabs-Inc-Down, Natl-Semiconduct-DOWN,Oracl-DOWN,SGI-DOWN, Sun-DOWN Apple-Comp-DOWN,Autodesk-DOWN,DEC-DOWN, ADV-Micro-Device-DOWN,Andrew-Corp-DOWN, Computer-Assoc-DOWN,Circuit-Cit-DOWN, Compaq-DOWN, EMC-Corp-DOWN, Gen-Inst-DOWN, Motorola-DOWN,Microsoft-DOWN,Scientific-Atl-DOWN Fannie-Mae-DOWN,Fed-Home-Loan-DOWN, MBNA-Corp-DOWN,Morgan-Stanle-DOWN Baker-Hughes-UP,Dresser-Inds-UP,Halliburton-HLD-UP, Louisiana-Land-UP,Phillips-Petro-UP,Unocal-UP, Schlumberger-UP Industr Group Technolog-DOWN Technolog-DOWN Financial-DOWN Oil-UP Summarization Reduce the size of large data sets Clustering precipitation in Australia Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 What is not Cluster Analsis? Supervised classification Have class label information Simple segmentation Dividing students into different registration groups alphabeticall, b last name Results of a quer Groupings are a result of an eternal specification Graph partitioning Some mutual relevance and snerg, but areas are not identical Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 4

3 Notion of a Cluster can be Ambiguous How man clusters? Si Clusters Two Clusters Four Clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 5 Tpes of Clusterings A clustering is a set of clusters Important distinction between hierarchical and partitional sets of clusters Partitional Clustering A division data objects into non-overlapping subsets (clusters) such that each data object is in eactl one subset Hierarchical clustering A set of nested clusters organized as a hierarchical tree Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 6

4 Partitional Clustering Original Points A Partitional Clustering Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 7 Hierarchical Clustering p p p p4 p p p p4 Traditional Hierarchical Clustering Traditional Dendrogram p p p p4 p p p p4 Non-traditional Hierarchical Clustering Non-traditional Dendrogram Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 8

5 Other Distinctions Between Sets of Clusters Eclusive versus non-eclusive In non-eclusive clusterings, points ma belong to multiple clusters. Can represent multiple classes or border points Fuzz versus non-fuzz In fuzz clustering, a point belongs to ever cluster with some weight between and Weights must sum to Probabilistic clustering has similar characteristics Partial versus complete In some cases, we onl want to cluster some of the data Heterogeneous versus homogeneous Cluster of widel different sizes, shapes, and densities Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 9 Tpes of Clusters Well-separated clusters Center-based clusters Contiguous clusters Densit-based clusters Propert or Conceptual Described b an Objective Function Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4

6 Tpes of Clusters: Well-Separated Well-Separated Clusters: A cluster is a set of points such that an point in a cluster is closer (or more similar) to ever other point in the cluster than to an point not in the cluster. well-separated clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 Tpes of Clusters: Center-Based Center-based A cluster is a set of objects such that an object in a cluster is closer (more similar) to the center of a cluster, than to the center of an other cluster The center of a cluster is often a centroid, the average of all the points in the cluster, or a medoid, the most representative point of a cluster 4 center-based clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4

7 Tpes of Clusters: Contiguit-Based Contiguous Cluster (Nearest neighbor or Transitive) A cluster is a set of points such that a point in a cluster is closer (or more similar) to one or more other points in the cluster than to an point not in the cluster. 8 contiguous clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 Tpes of Clusters: Densit-Based Densit-based A cluster is a dense region of points, which is separated b low-densit regions, from other regions of high densit. Used when the clusters are irregular or intertwined, and when noise and outliers are present. 6 densit-based clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 4

8 Tpes of Clusters: Conceptual Clusters Shared Propert or Conceptual Clusters Finds clusters that share some common propert or represent a particular concept.. Overlapping Circles Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 5 Tpes of Clusters: Objective Function Clusters Defined b an Objective Function Finds clusters that minimize or maimize an objective function. Enumerate all possible was of dividing the points into clusters and evaluate the `goodness' of each potential set of clusters b using the given objective function. (NP Hard) Can have global or local objectives. Hierarchical clustering algorithms tpicall have local objectives Partitional algorithms tpicall have global objectives A variation of the global objective function approach is to fit the data to a parameterized model. Parameters for the model are determined from the data. Miture models assume that the data is a miture' of a number of statistical distributions. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 6

9 Tpes of Clusters: Objective Function Map the clustering problem to a different domain and solve a related problem in that domain Proimit matri defines a weighted graph, where the nodes are the points being clustered, and the weighted edges represent the proimities between points Clustering is equivalent to breaking the graph into connected components, one for each cluster. Want to minimize the edge weight between clusters and maimize the edge weight within clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 7 Characteristics of the Input Data Are Important Tpe of proimit or densit measure This is a derived measure, but central to clustering Sparseness Dictates tpe of similarit Adds to efficienc Attribute tpe Dictates tpe of similarit Tpe of Data Dictates tpe of similarit Other characteristics, e.g., autocorrelation Dimensionalit Noise and Outliers Tpe of Distribution Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 8

10 Clustering Algorithms K-means and its variants Hierarchical clustering Densit-based clustering Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 9 K-means Clustering Partitional clustering approach Each cluster is associated with a centroid (center point) Each point is assigned to the cluster with the closest centroid Number of clusters, K, must be specified The basic algorithm is ver simple Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4

11 K-means Clustering Details Initial centroids are often chosen randoml. Clusters produced var from one run to another. The centroid is (tpicall) the mean of the points in the cluster. Closeness is measured b Euclidean distance, cosine similarit, correlation, etc. K-means will converge for common similarit measures mentioned above. Most of the convergence happens in the first few iterations. Often the stopping condition is changed to Until relativel few points change clusters Compleit is O( n * K * I * d ) n = number of points, K = number of clusters, I = number of iterations, d = number of attributes Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 Two different K-means Clusterings.5.5 Original Points Optimal Clustering Sub-optimal Clustering Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4

12 Importance of Choosing Initial Centroids Ite ration Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 Importance of Choosing Initial Centroids Iteration Iteration Iteration Iteration 4 Iteration 5 Iteration Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 4

13 Evaluating K-means Clusters Most common measure is Sum of Squared Error (SSE) For each point, the error is the distance to the nearest cluster To get SSE, we square these errors and sum them. SSE = K i= C i dist ( m, ) is a data point in cluster C i and m i is the representative point for cluster C i can show that m i corresponds to the center (mean) of the cluster Given two clusters, we can choose the one with the smallest error One eas wa to reduce SSE is to increase K, the number of clusters A good clustering with smaller K can have a lower SSE than a poor clustering with higher K i Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 5 Importance of Choosing Initial Centroids Ite ration Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 6

14 Importance of Choosing Initial Centroids Iteration Iteration Iteration Iteration 4 Iteration Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 7 Problems with Selecting Initial Points If there are K real clusters then the chance of selecting one centroid from each cluster is small. Chance is relativel small when K is large If clusters are the same size, n, then For eample, if K =, then probabilit =!/ =.6 Sometimes the initial centroids will readjust themselves in right wa, and sometimes the don t Consider an eample of five pairs of clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 8

15 Clusters Eample 8 Ite ration Starting with two initial centroids in one cluster of each pair of clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 9 Clusters Eample 8 Iteration 8 Iteration Iteration Iteration Starting with two initial centroids in one cluster of each pair of clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4

16 Clusters Eample 8 Ite ration Starting with some pairs of clusters having three initial centroids, while other have onl one. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 Clusters Eample 8 Iteration 8 Iteration Iteration Iteration Starting with some pairs of clusters having three initial centroids, while other have onl one. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4

17 Solutions to Initial Centroids Problem Multiple runs Helps, but probabilit is not on our side Sample and use hierarchical clustering to determine initial centroids Select more than k initial centroids and then select among these initial centroids Select most widel separated Postprocessing Bisecting K-means Not as susceptible to initialization issues Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 Handling Empt Clusters Basic K-means algorithm can ield empt clusters Several strategies Choose the point that contributes most to SSE Choose a point from the cluster with the highest SSE If there are several empt clusters, the above can be repeated several times. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 4

18 Updating Centers Incrementall In the basic K-means algorithm, centroids are updated after all points are assigned to a centroid An alternative is to update the centroids after each assignment (incremental approach) Each assignment updates zero or two centroids More epensive Introduces an order dependenc Never get an empt cluster Can use weights to change the impact Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 5 Pre-processing and Post-processing Pre-processing Normalize the data Eliminate outliers Post-processing Eliminate small clusters that ma represent outliers Split loose clusters, i.e., clusters with relativel high SSE Merge clusters that are close and that have relativel low SSE Can use these steps during the clustering process ISODATA Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 6

19 Bisecting K-means Bisecting K-means algorithm Variant of K-means that can produce a partitional or a hierarchical clustering Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 7 Bisecting K-means Eample Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 8

20 Limitations of K-means K-means has problems when clusters are of differing Sizes Densities Non-globular shapes K-means has problems when the data contains outliers. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 9 Limitations of K-means: Differing Sizes Original Points K-means ( Clusters) Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 4

21 Limitations of K-means: Differing Densit Original Points K-means ( Clusters) Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 4 Limitations of K-means: Non-globular Shapes Original Points K-means ( Clusters) Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 4

22 Overcoming K-means Limitations Original Points K-means Clusters One solution is to use man clusters. Find parts of clusters, but need to put together. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 4 Overcoming K-means Limitations Original Points K-means Clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 44

23 Overcoming K-means Limitations Original Points K-means Clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 45 Hierarchical Clustering Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram A tree like diagram that records the sequences of merges or splits Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 46

24 Strengths of Hierarchical Clustering Do not have to assume an particular number of clusters An desired number of clusters can be obtained b cutting the dendogram at the proper level The ma correspond to meaningful taonomies Eample in biological sciences (e.g., animal kingdom, phlogen reconstruction, ) Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 47 Hierarchical Clustering Two main tpes of hierarchical clustering Agglomerative: Start with the points as individual clusters At each step, merge the closest pair of clusters until onl one cluster (or k clusters) left Divisive: Start with one, all-inclusive cluster At each step, split a cluster until each cluster contains a point (or there are k clusters) Traditional hierarchical algorithms use a similarit or distance matri Merge or split one cluster at a time Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 48

25 Agglomerative Clustering Algorithm More popular hierarchical clustering technique Basic algorithm is straightforward. Compute the proimit matri. Let each data point be a cluster. Repeat 4. Merge the two closest clusters 5. Update the proimit matri 6. Until onl a single cluster remains Ke operation is the computation of the proimit of two clusters Different approaches to defining the distance between clusters distinguish the different algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 49 Starting Situation Start with clusters of individual points and a proimit matri p p p p4 p5.. p p p p4 p5.... Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining 4/8/ p p p p4 p9 p p p

26 Intermediate Situation After some merging steps, we have some clusters C C C C4 C5 C C C C C4 C C4 C5 Proimit Matri C C5 Tan,Steinbach, Kumar Introduction to Data Mining 4/8/ p p p p4 p9 p p p Intermediate Situation We want to merge the two closest clusters (C and C5) and update the proimit matri. C C C C C4 C5 C C C C4 C C4 C5 Proimit Matri C C5 Tan,Steinbach, Kumar Introduction to Data Mining 4/8/ p p p p4 p9 p p p

27 After Merging The question is How do we update the proimit matri? C C U C5 C C4 C? C C C4 C U C5???? C? C4? Proimit Matri C U C5 Tan,Steinbach, Kumar Introduction to Data Mining 4/8/ p p p p4 p9 p p p How to Define Inter-Cluster Similarit p p p p4 p5... Similarit? p p MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p p4 p5... Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 54

28 How to Define Inter-Cluster Similarit p p p p4 p5... p p p p4 MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p5... Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 55 How to Define Inter-Cluster Similarit p p p p4 p5... p p p p4 MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p5... Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 56

29 How to Define Inter-Cluster Similarit p p p p4 p5... p p p p4 MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p5... Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 57 How to Define Inter-Cluster Similarit p p p p p4 p5... p p p4 MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p5... Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 58

30 Cluster Similarit: MIN or Single Link Similarit of two clusters is based on the two most similar (closest) points in the different clusters Determined b one pair of points, i.e., b one link in the proimit graph. I I I I4 I5 I I I I I Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 59 Hierarchical Clustering: MIN Nested Clusters Dendrogram Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 6

31 Strength of MIN Original Points Two Clusters Can handle non-elliptical shapes Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 6 Limitations of MIN Original Points Two Clusters Sensitive to noise and outliers Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 6

32 Cluster Similarit: MAX or Complete Linkage Similarit of two clusters is based on the two least similar (most distant) points in the different clusters Determined b all pairs of points in the two clusters I I I I4 I5 I I I I I Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 6 Hierarchical Clustering: MAX Nested Clusters Dendrogram Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 64

33 Strength of MAX Original Points Two Clusters Less susceptible to noise and outliers Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 65 Limitations of MAX Original Points Two Clusters Tends to break large clusters Biased towards globular clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 66

34 Cluster Similarit: Group Average Proimit of two clusters is the average of pairwise proimit between points in the two clusters. proimit(pi,pj) pi Clusteri p Cluster j j proimit(clusteri,clusterj) = Cluster Cluster Need to use average connectivit for scalabilit since total proimit favors large clusters I I I I4 I5 I I I I I Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 67 i j Hierarchical Clustering: Group Average Nested Clusters Dendrogram Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 68

35 Hierarchical Clustering: Group Average Compromise between Single and Complete Link Strengths Less susceptible to noise and outliers Limitations Biased towards globular clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 69 Cluster Similarit: Ward s Method Similarit of two clusters is based on the increase in squared error when two clusters are merged Similar to group average if distance between points is distance squared Less susceptible to noise and outliers Biased towards globular clusters Hierarchical analogue of K-means Can be used to initialize K-means Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 7

36 Hierarchical Clustering: Comparison MIN MAX Group Average Ward s Method Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 7 Hierarchical Clustering: Time and Space requirements O(N ) space since it uses the proimit matri. N is the number of points. O(N ) time in man cases There are N steps and at each step the size, N, proimit matri must be updated and searched Compleit can be reduced to O(N log(n) ) time for some approaches Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 7

37 Hierarchical Clustering: Problems and Limitations Once a decision is made to combine two clusters, it cannot be undone No objective function is directl minimized Different schemes have problems with one or more of the following: Sensitivit to noise and outliers Difficult handling different sized clusters and conve shapes Breaking large clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 7 MST: Divisive Hierarchical Clustering Build MST (Minimum Spanning Tree) Start with a tree that consists of an point In successive steps, look for the closest pair of points (p, q) such that one point (p) is in the current tree but the other (q) is not Add q to the tree and put an edge between p and q Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 74

38 MST: Divisive Hierarchical Clustering Use MST for constructing hierarch of clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 75 DBSCAN DBSCAN is a densit-based algorithm. Densit = number of points within a specified radius (Eps) A point is a core point if it has more than a specified number of points (MinPts) within Eps These are points that are at the interior of a cluster A border point has fewer than MinPts within Eps, but is in the neighborhood of a core point A noise point is an point that is not a core point or a border point. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 76

39 DBSCAN: Core, Border, and Noise Points Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 77 DBSCAN Algorithm Eliminate noise points Perform clustering on the remaining points Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 78

40 DBSCAN: Core, Border and Noise Points Original Points Point tpes: core, border and noise Eps =, MinPts = 4 Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 79 When DBSCAN Works Well Original Points Clusters Resistant to Noise Can handle clusters of different shapes and sizes Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 8

41 When DBSCAN Does NOT Work Well Original Points (MinPts=4, Eps=9.75). Varing densities High-dimensional data (MinPts=4, Eps=9.9) Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 8 DBSCAN: Determining EPS and MinPts Idea is that for points in a cluster, their k th nearest neighbors are at roughl the same distance Noise points have the k th nearest neighbor at farther distance So, plot sorted distance of ever point to its k th nearest neighbor Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 8

42 Cluster Validit For supervised classification we have a variet of measures to evaluate how good our model is Accurac, precision, recall For cluster analsis, the analogous question is how to evaluate the goodness of the resulting clusters? But clusters are in the ee of the beholder! Then wh do we want to evaluate them? To avoid finding patterns in noise To compare clustering algorithms To compare two sets of clusters To compare two clusters Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 8 Clusters found in Random Data Random Points DBSCAN K-means Complete Link Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 84

43 Different Aspects of Cluster Validation. Determining the clustering tendenc of a set of data, i.e., distinguishing whether non-random structure actuall eists in the data.. Comparing the results of a cluster analsis to eternall known results, e.g., to eternall given class labels.. Evaluating how well the results of a cluster analsis fit the data without reference to eternal information. - Use onl the data 4. Comparing the results of two different sets of cluster analses to determine which is better. 5. Determining the correct number of clusters. For,, and 4, we can further distinguish whether we want to evaluate the entire clustering or just individual clusters. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 85 Measures of Cluster Validit Numerical measures that are applied to judge various aspects of cluster validit, are classified into the following three tpes. Eternal Inde: Used to measure the etent to which cluster labels match eternall supplied class labels. Entrop Internal Inde: Used to measure the goodness of a clustering structure without respect to eternal information. Sum of Squared Error (SSE) Relative Inde: Used to compare two different clusterings or clusters. Often an eternal or internal inde is used for this function, e.g., SSE or entrop Sometimes these are referred to as criteria instead of indices However, sometimes criterion is the general strateg and inde is the numerical measure that implements the criterion. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 86

44 Measuring Cluster Validit Via Correlation Two matrices Proimit Matri Incidence Matri One row and one column for each data point An entr is if the associated pair of points belong to the same cluster An entr is if the associated pair of points belongs to different clusters Compute the correlation between the two matrices Since the matrices are smmetric, onl the correlation between n(n-) / entries needs to be calculated. High correlation indicates that points that belong to the same cluster are close to each other. Not a good measure for some densit or contiguit based clusters. Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 87 Measuring Cluster Validit Via Correlation Correlation of incidence and proimit matrices for the K-means clusterings of the following two data sets Corr = -.95 Corr = -.58 Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 88

45 Using Similarit Matri for Cluster Validation Order the similarit matri with respect to cluster labels and inspect visuall Points Similarit Points Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 89 Using Similarit Matri for Cluster Validation Clusters in random data are not so crisp Points Similarit Points DBSCAN Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 9

46 Using Similarit Matri for Cluster Validation Clusters in random data are not so crisp Points Similarit Points K-means Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 9 Using Similarit Matri for Cluster Validation Clusters in random data are not so crisp Points Similarit Points Complete Link Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 9

47 Using Similarit Matri for Cluster Validation DBSCAN Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 9 Internal Measures: SSE Clusters in more complicated figures aren t well separated Internal Inde: Used to measure the goodness of a clustering structure without respect to eternal information SSE SSE is good for comparing two clusterings or two clusters (average SSE). Can also be used to estimate the number of clusters K Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 94 SSE

48 Internal Measures: SSE SSE curve for a more complicated data set SSE of clusters found using K-means Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 95 Framework for Cluster Validit Need a framework to interpret an measure. For eample, if our measure of evaluation has the value,, is that good, fair, or poor? Statistics provide a framework for cluster validit The more atpical a clustering result is, the more likel it represents valid structure in the data Can compare the values of an inde that result from random data or clusterings to those of a clustering result. If the value of the inde is unlikel, then the cluster results are valid These approaches are more complicated and harder to understand. For comparing the results of two different sets of cluster analses, a framework is less necessar. However, there is the question of whether the difference between two inde values is significant Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 96

49 Statistical Framework for SSE Eample Compare SSE of.5 against three clusters in random data Histogram shows SSE of three clusters in 5 sets of random data points of size distributed over the range..8 for and values Count SSE Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 97 Statistical Framework for Correlation Correlation of incidence and proimit matrices for the K-means clusterings of the following two data sets Corr = -.95 Corr = -.58 Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 98

50 Internal Measures: Cohesion and Separation Cluster Cohesion: Measures how closel related are objects in a cluster Eample: SSE Cluster Separation: Measure how distinct or wellseparated a cluster is from other clusters Eample: Squared Error Cohesion is measured b the within cluster sum of squares (SSE) WSS = ( mi ) i C Separation is measured b the between cluster sum of squares BSS = C ( m i m i i Where C i is the size of cluster i i Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 99 ) Internal Measures: Cohesion and Separation Eample: SSE BSS + WSS = constant m m 4 m 5 K= cluster: K= clusters: WSS= ( ) + ( ) + (4 ) + (5 ) = BSS= 4 ( ) Total = + = WSS= (.5) + (.5) + (4 4.5) + (5 4.5) = BSS= (.5) + (4.5 ) Total = + 9 = = = 9 Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4

51 Internal Measures: Cohesion and Separation A proimit graph based approach can also be used for cohesion and separation. Cluster cohesion is the sum of the weight of all links within a cluster. Cluster separation is the sum of the weights between nodes in the cluster and nodes outside the cluster. cohesion separation Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 Internal Measures: Silhouette Coefficient Silhouette Coefficient combine ideas of both cohesion and separation, but for individual points, as well as clusters and clusterings For an individual point, i Calculate a = average distance of i to the points in its cluster Calculate b = min (average distance of i to points in another cluster) The silhouette coefficient for a point is then given b s = a/b if a < b, (or s = b/a - if a b, not the usual case) Tpicall between and. The closer to the better. a b Can calculate the Average Silhouette width for a cluster or a clustering Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4

52 Eternal Measures of Cluster Validit: Entrop and Purit Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 Final Comment on Cluster Validit The validation of clustering structures is the most difficult and frustrating part of cluster analsis. Without a strong effort in this direction, cluster analsis will remain a black art accessible onl to those true believers who have eperience and great courage. Algorithms for Clustering Data, Jain and Dubes Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 4

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining /8/ What is Cluster

More information

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Clustering Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Clustering Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Clustering Algorithms K-means and its variants Hierarchical clustering

More information

K-Means Cluster Analysis. Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1

K-Means Cluster Analysis. Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 K-Means Cluster Analsis Chapter 3 PPDM Class Tan,Steinbach, Kumar Introduction to Data Mining 4/18/4 1 What is Cluster Analsis? Finding groups of objects such that the objects in a group will be similar

More information

For supervised classification we have a variety of measures to evaluate how good our model is Accuracy, precision, recall

For supervised classification we have a variety of measures to evaluate how good our model is Accuracy, precision, recall Cluster Validation Cluster Validit For supervised classification we have a variet of measures to evaluate how good our model is Accurac, precision, recall For cluster analsis, the analogous question is

More information

Example: Document Clustering. Clustering: Definition. Notion of a Cluster can be Ambiguous. Types of Clusterings. Hierarchical Clustering

Example: Document Clustering. Clustering: Definition. Notion of a Cluster can be Ambiguous. Types of Clusterings. Hierarchical Clustering Overview Prognostic Models and Data Mining in Medicine, part I Cluster Analsis What is Cluster Analsis? K-Means Clustering Hierarchical Clustering Cluster Validit Eample: Microarra data analsis 6 Summar

More information

Cluster Analysis: Basic Concepts and Algorithms

Cluster Analysis: Basic Concepts and Algorithms Cluster Analsis: Basic Concepts and Algorithms What does it mean clustering? Applications Tpes of clustering K-means Intuition Algorithm Choosing initial centroids Bisecting K-means Post-processing Strengths

More information

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/2004 Hierarchical

More information

Data Mining Clustering (2) Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining

Data Mining Clustering (2) Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining Data Mining Clustering (2) Toon Calders Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining Outline Partitional Clustering Distance-based K-means, K-medoids,

More information

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDD-LAB ISTI- CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar

More information

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 by Tan, Steinbach, Kumar 1 What is Cluster Analysis? Finding groups of objects such that the objects in a group will

More information

Cluster Analysis Overview. Data Mining Techniques: Cluster Analysis. What is Cluster Analysis? What is Cluster Analysis?

Cluster Analysis Overview. Data Mining Techniques: Cluster Analysis. What is Cluster Analysis? What is Cluster Analysis? Cluster Analsis Overview Data Mining Techniques: Cluster Analsis Mirek Riedewald Man slides based on presentations b Han/Kamber, Tan/Steinbach/Kumar, and Andrew Moore Introduction Foundations: Measuring

More information

Clustering. Adrian Groza. Department of Computer Science Technical University of Cluj-Napoca

Clustering. Adrian Groza. Department of Computer Science Technical University of Cluj-Napoca Clustering Adrian Groza Department of Computer Science Technical University of Cluj-Napoca Outline 1 Cluster Analysis What is Datamining? Cluster Analysis 2 K-means 3 Hierarchical Clustering What is Datamining?

More information

Clustering. Data Mining. Abraham Otero. Data Mining. Agenda

Clustering. Data Mining. Abraham Otero. Data Mining. Agenda Clustering 1/46 Agenda Introduction Distance K-nearest neighbors Hierarchical clustering Quick reference 2/46 1 Introduction It seems logical that in a new situation we should act in a similar way as in

More information

Cluster Analysis: Basic Concepts and Algorithms

Cluster Analysis: Basic Concepts and Algorithms 8 Cluster Analysis: Basic Concepts and Algorithms Cluster analysis divides data into groups (clusters) that are meaningful, useful, or both. If meaningful groups are the goal, then the clusters should

More information

Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining

Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

More information

Cluster Analysis: Advanced Concepts

Cluster Analysis: Advanced Concepts Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototype-based Fuzzy c-means

More information

Clustering UE 141 Spring 2013

Clustering UE 141 Spring 2013 Clustering UE 141 Spring 013 Jing Gao SUNY Buffalo 1 Definition of Clustering Finding groups of obects such that the obects in a group will be similar (or related) to one another and different from (or

More information

Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining

Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

More information

Cluster Analysis. Alison Merikangas Data Analysis Seminar 18 November 2009

Cluster Analysis. Alison Merikangas Data Analysis Seminar 18 November 2009 Cluster Analysis Alison Merikangas Data Analysis Seminar 18 November 2009 Overview What is cluster analysis? Types of cluster Distance functions Clustering methods Agglomerative K-means Density-based Interpretation

More information

Unsupervised learning: Clustering

Unsupervised learning: Clustering Unsupervised learning: Clustering Salissou Moutari Centre for Statistical Science and Operational Research CenSSOR 17 th September 2013 Unsupervised learning: Clustering 1/52 Outline 1 Introduction What

More information

Data Mining. Cluster Analysis: Advanced Concepts and Algorithms

Data Mining. Cluster Analysis: Advanced Concepts and Algorithms Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototype-based clustering Density-based clustering Graph-based

More information

Clustering. Clustering. What is Clustering? What is Clustering? What is Clustering? Types of Data in Cluster Analysis

Clustering. Clustering. What is Clustering? What is Clustering? What is Clustering? Types of Data in Cluster Analysis What is Clustering? Clustering Tpes of Data in Cluster Analsis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods What is Clustering? Clustering of data is

More information

Cluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico

Cluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from

More information

An Introduction to Cluster Analysis for Data Mining

An Introduction to Cluster Analysis for Data Mining An Introduction to Cluster Analysis for Data Mining 10/02/2000 11:42 AM 1. INTRODUCTION... 4 1.1. Scope of This Paper... 4 1.2. What Cluster Analysis Is... 4 1.3. What Cluster Analysis Is Not... 5 2. OVERVIEW...

More information

Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016

Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with

More information

Client Based Power Iteration Clustering Algorithm to Reduce Dimensionality in Big Data

Client Based Power Iteration Clustering Algorithm to Reduce Dimensionality in Big Data Client Based Power Iteration Clustering Algorithm to Reduce Dimensionalit in Big Data Jaalatchum. D 1, Thambidurai. P 1, Department of CSE, PKIET, Karaikal, India Abstract - Clustering is a group of objects

More information

Chapter 7. Cluster Analysis

Chapter 7. Cluster Analysis Chapter 7. Cluster Analysis. What is Cluster Analysis?. A Categorization of Major Clustering Methods. Partitioning Methods. Hierarchical Methods 5. Density-Based Methods 6. Grid-Based Methods 7. Model-Based

More information

Cluster Analysis: Basic Concepts and Methods

Cluster Analysis: Basic Concepts and Methods 10 Cluster Analysis: Basic Concepts and Methods Imagine that you are the Director of Customer Relationships at AllElectronics, and you have five managers working for you. You would like to organize all

More information

Clustering & Visualization

Clustering & Visualization Chapter 5 Clustering & Visualization Clustering in high-dimensional databases is an important problem and there are a number of different clustering paradigms which are applicable to high-dimensional data.

More information

Neural Networks Lesson 5 - Cluster Analysis

Neural Networks Lesson 5 - Cluster Analysis Neural Networks Lesson 5 - Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt. - Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29

More information

How To Cluster

How To Cluster Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical Main

More information

SoSe 2014: M-TANI: Big Data Analytics

SoSe 2014: M-TANI: Big Data Analytics SoSe 2014: M-TANI: Big Data Analytics Lecture 4 21/05/2014 Sead Izberovic Dr. Nikolaos Korfiatis Agenda Recap from the previous session Clustering Introduction Distance mesures Hierarchical Clustering

More information

CLUSTER ANALYSIS FOR SEGMENTATION

CLUSTER ANALYSIS FOR SEGMENTATION CLUSTER ANALYSIS FOR SEGMENTATION Introduction We all understand that consumers are not all alike. This provides a challenge for the development and marketing of profitable products and services. Not every

More information

Data Mining Project Report. Document Clustering. Meryem Uzun-Per

Data Mining Project Report. Document Clustering. Meryem Uzun-Per Data Mining Project Report Document Clustering Meryem Uzun-Per 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. K-means algorithm...

More information

Clustering. 15-381 Artificial Intelligence Henry Lin. Organizing data into clusters such that there is

Clustering. 15-381 Artificial Intelligence Henry Lin. Organizing data into clusters such that there is Clustering 15-381 Artificial Intelligence Henry Lin Modified from excellent slides of Eamonn Keogh, Ziv Bar-Joseph, and Andrew Moore What is Clustering? Organizing data into clusters such that there is

More information

Chapter ML:XI (continued)

Chapter ML:XI (continued) Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained

More information

Territorial Analysis for Ratemaking. Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs

Territorial Analysis for Ratemaking. Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs Territorial Analysis for Ratemaking by Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs Department of Statistics and Applied Probability University

More information

Solving Quadratic Equations by Graphing. Consider an equation of the form. y ax 2 bx c a 0. In an equation of the form

Solving Quadratic Equations by Graphing. Consider an equation of the form. y ax 2 bx c a 0. In an equation of the form SECTION 11.3 Solving Quadratic Equations b Graphing 11.3 OBJECTIVES 1. Find an ais of smmetr 2. Find a verte 3. Graph a parabola 4. Solve quadratic equations b graphing 5. Solve an application involving

More information

Distances, Clustering, and Classification. Heatmaps

Distances, Clustering, and Classification. Heatmaps Distances, Clustering, and Classification Heatmaps 1 Distance Clustering organizes things that are close into groups What does it mean for two genes to be close? What does it mean for two samples to be

More information

Cluster analysis Cosmin Lazar. COMO Lab VUB

Cluster analysis Cosmin Lazar. COMO Lab VUB Cluster analysis Cosmin Lazar COMO Lab VUB Introduction Cluster analysis foundations rely on one of the most fundamental, simple and very often unnoticed ways (or methods) of understanding and learning,

More information

A Survey of Clustering Techniques

A Survey of Clustering Techniques A Survey of Clustering Techniques Pradeep Rai Asst. Prof., CSE Department, Kanpur Institute of Technology, Kanpur-0800 (India) Shubha Singh Asst. Prof., MCA Department, Kanpur Institute of Technology,

More information

A comparison of various clustering methods and algorithms in data mining

A comparison of various clustering methods and algorithms in data mining Volume :2, Issue :5, 32-36 May 2015 www.allsubjectjournal.com e-issn: 2349-4182 p-issn: 2349-5979 Impact Factor: 3.762 R.Tamilselvi B.Sivasakthi R.Kavitha Assistant Professor A comparison of various clustering

More information

Social Media Mining. Data Mining Essentials

Social Media Mining. Data Mining Essentials Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

More information

. Learn the number of classes and the structure of each class using similarity between unlabeled training patterns

. Learn the number of classes and the structure of each class using similarity between unlabeled training patterns Outline Part 1: of data clustering Non-Supervised Learning and Clustering : Problem formulation cluster analysis : Taxonomies of Clustering Techniques : Data types and Proximity Measures : Difficulties

More information

Information Retrieval and Web Search Engines

Information Retrieval and Web Search Engines Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 10 th, 2013 Wolf-Tilo Balke and Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig

More information

Summary Data Mining & Process Mining (1BM46) Content. Made by S.P.T. Ariesen

Summary Data Mining & Process Mining (1BM46) Content. Made by S.P.T. Ariesen Summary Data Mining & Process Mining (1BM46) Made by S.P.T. Ariesen Content Data Mining part... 2 Lecture 1... 2 Lecture 2:... 4 Lecture 3... 7 Lecture 4... 9 Process mining part... 13 Lecture 5... 13

More information

Concept of Cluster Analysis

Concept of Cluster Analysis RESEARCH PAPER ON CLUSTER TECHNIQUES OF DATA VARIATIONS Er. Arpit Gupta 1,Er.Ankit Gupta 2,Er. Amit Mishra 3 arpit_jp@yahoo.co.in, ank_mgcgv@yahoo.co.in,amitmishra.mtech@gmail.com Faculty Of Engineering

More information

Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu

Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?

More information

Cluster Analysis using R

Cluster Analysis using R Cluster analysis or clustering is the task of assigning a set of objects into groups (called clusters) so that the objects in the same cluster are more similar (in some sense or another) to each other

More information

Comparison and Analysis of Various Clustering Methods in Data mining On Education data set Using the weak tool

Comparison and Analysis of Various Clustering Methods in Data mining On Education data set Using the weak tool Comparison and Analysis of Various Clustering Metho in Data mining On Education data set Using the weak tool Abstract:- Data mining is used to find the hidden information pattern and relationship between

More information

OUTLIER ANALYSIS. Data Mining 1

OUTLIER ANALYSIS. Data Mining 1 OUTLIER ANALYSIS Data Mining 1 What Are Outliers? Outlier: A data object that deviates significantly from the normal objects as if it were generated by a different mechanism Ex.: Unusual credit card purchase,

More information

PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA

PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA Prakash Singh 1, Aarohi Surya 2 1 Department of Finance, IIM Lucknow, Lucknow, India 2 Department of Computer Science, LNMIIT, Jaipur,

More information

Unsupervised Learning and Data Mining. Unsupervised Learning and Data Mining. Clustering. Supervised Learning. Supervised Learning

Unsupervised Learning and Data Mining. Unsupervised Learning and Data Mining. Clustering. Supervised Learning. Supervised Learning Unsupervised Learning and Data Mining Unsupervised Learning and Data Mining Clustering Decision trees Artificial neural nets K-nearest neighbor Support vectors Linear regression Logistic regression...

More information

2 Basic Concepts and Techniques of Cluster Analysis

2 Basic Concepts and Techniques of Cluster Analysis The Challenges of Clustering High Dimensional Data * Michael Steinbach, Levent Ertöz, and Vipin Kumar Abstract Cluster analysis divides data into groups (clusters) for the purposes of summarization or

More information

There are a number of different methods that can be used to carry out a cluster analysis; these methods can be classified as follows:

There are a number of different methods that can be used to carry out a cluster analysis; these methods can be classified as follows: Statistics: Rosie Cornish. 2007. 3.1 Cluster Analysis 1 Introduction This handout is designed to provide only a brief introduction to cluster analysis and how it is done. Books giving further details are

More information

Hierarchical Cluster Analysis Some Basics and Algorithms

Hierarchical Cluster Analysis Some Basics and Algorithms Hierarchical Cluster Analysis Some Basics and Algorithms Nethra Sambamoorthi CRMportals Inc., 11 Bartram Road, Englishtown, NJ 07726 (NOTE: Please use always the latest copy of the document. Click on this

More information

UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS

UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS Dwijesh C. Mishra I.A.S.R.I., Library Avenue, New Delhi-110 012 dcmishra@iasri.res.in What is Learning? "Learning denotes changes in a system that enable

More information

ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING)

ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Preliminaries Classification and Clustering Applications

More information

Data Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining

Data Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining Data Mining: Exploring Data Lecture Notes for Chapter 3 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 1 What is data exploration? A preliminary

More information

Decision Support System Methodology Using a Visual Approach for Cluster Analysis Problems

Decision Support System Methodology Using a Visual Approach for Cluster Analysis Problems Decision Support System Methodology Using a Visual Approach for Cluster Analysis Problems Ran M. Bittmann School of Business Administration Ph.D. Thesis Submitted to the Senate of Bar-Ilan University Ramat-Gan,

More information

Data Mining and Knowledge Discovery in Databases (KDD) State of the Art. Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland

Data Mining and Knowledge Discovery in Databases (KDD) State of the Art. Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland Data Mining and Knowledge Discovery in Databases (KDD) State of the Art Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland 1 Conference overview 1. Overview of KDD and data mining 2. Data

More information

Performance Metrics for Graph Mining Tasks

Performance Metrics for Graph Mining Tasks Performance Metrics for Graph Mining Tasks 1 Outline Introduction to Performance Metrics Supervised Learning Performance Metrics Unsupervised Learning Performance Metrics Optimizing Metrics Statistical

More information

USING THE AGGLOMERATIVE METHOD OF HIERARCHICAL CLUSTERING AS A DATA MINING TOOL IN CAPITAL MARKET 1. Vera Marinova Boncheva

USING THE AGGLOMERATIVE METHOD OF HIERARCHICAL CLUSTERING AS A DATA MINING TOOL IN CAPITAL MARKET 1. Vera Marinova Boncheva 382 [7] Reznik, A, Kussul, N., Sokolov, A.: Identification of user activity using neural networks. Cybernetics and computer techniques, vol. 123 (1999) 70 79. (in Russian) [8] Kussul, N., et al. : Multi-Agent

More information

A Comparative Study of clustering algorithms Using weka tools

A Comparative Study of clustering algorithms Using weka tools A Comparative Study of clustering algorithms Using weka tools Bharat Chaudhari 1, Manan Parikh 2 1,2 MECSE, KITRC KALOL ABSTRACT Data clustering is a process of putting similar data into groups. A clustering

More information

Classification Techniques (1)

Classification Techniques (1) 10 10 Overview Classification Techniques (1) Today Classification Problem Classification based on Regression Distance-based Classification (KNN) Net Lecture Decision Trees Classification using Rules Quality

More information

Steven M. Ho!and. Department of Geology, University of Georgia, Athens, GA 30602-2501

Steven M. Ho!and. Department of Geology, University of Georgia, Athens, GA 30602-2501 CLUSTER ANALYSIS Steven M. Ho!and Department of Geology, University of Georgia, Athens, GA 30602-2501 January 2006 Introduction Cluster analysis includes a broad suite of techniques designed to find groups

More information

Unsupervised Data Mining (Clustering)

Unsupervised Data Mining (Clustering) Unsupervised Data Mining (Clustering) Javier Béjar KEMLG December 01 Javier Béjar (KEMLG) Unsupervised Data Mining (Clustering) December 01 1 / 51 Introduction Clustering in KDD One of the main tasks in

More information

Machine Learning using MapReduce

Machine Learning using MapReduce Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous

More information

A Method for Decentralized Clustering in Large Multi-Agent Systems

A Method for Decentralized Clustering in Large Multi-Agent Systems A Method for Decentralized Clustering in Large Multi-Agent Systems Elth Ogston, Benno Overeinder, Maarten van Steen, and Frances Brazier Department of Computer Science, Vrije Universiteit Amsterdam {elth,bjo,steen,frances}@cs.vu.nl

More information

7.3 Parabolas. 7.3 Parabolas 505

7.3 Parabolas. 7.3 Parabolas 505 7. Parabolas 0 7. Parabolas We have alread learned that the graph of a quadratic function f() = a + b + c (a 0) is called a parabola. To our surprise and delight, we ma also define parabolas in terms of

More information

SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING

SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING AAS 07-228 SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING INTRODUCTION James G. Miller * Two historical uncorrelated track (UCT) processing approaches have been employed using general perturbations

More information

Data Preprocessing. Week 2

Data Preprocessing. Week 2 Data Preprocessing Week 2 Topics Data Types Data Repositories Data Preprocessing Present homework assignment #1 Team Homework Assignment #2 Read pp. 227 240, pp. 250 250, and pp. 259 263 the text book.

More information

On Clustering Validation Techniques

On Clustering Validation Techniques Journal of Intelligent Information Systems, 17:2/3, 107 145, 2001 c 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques MARIA HALKIDI mhalk@aueb.gr YANNIS

More information

Time series clustering and the analysis of film style

Time series clustering and the analysis of film style Time series clustering and the analysis of film style Nick Redfern Introduction Time series clustering provides a simple solution to the problem of searching a database containing time series data such

More information

A Study of Web Log Analysis Using Clustering Techniques

A Study of Web Log Analysis Using Clustering Techniques A Study of Web Log Analysis Using Clustering Techniques Hemanshu Rana 1, Mayank Patel 2 Assistant Professor, Dept of CSE, M.G Institute of Technical Education, Gujarat India 1 Assistant Professor, Dept

More information

SECTION 5-1 Exponential Functions

SECTION 5-1 Exponential Functions 354 5 Eponential and Logarithmic Functions Most of the functions we have considered so far have been polnomial and rational functions, with a few others involving roots or powers of polnomial or rational

More information

Constrained Clustering of Territories in the Context of Car Insurance

Constrained Clustering of Territories in the Context of Car Insurance Constrained Clustering of Territories in the Context of Car Insurance Samuel Perreault Jean-Philippe Le Cavalier Laval University July 2014 Perreault & Le Cavalier (ULaval) Constrained Clustering July

More information

STATISTICA. Clustering Techniques. Case Study: Defining Clusters of Shopping Center Patrons. and

STATISTICA. Clustering Techniques. Case Study: Defining Clusters of Shopping Center Patrons. and Clustering Techniques and STATISTICA Case Study: Defining Clusters of Shopping Center Patrons STATISTICA Solutions for Business Intelligence, Data Mining, Quality Control, and Web-based Analytics Table

More information

Introduction to Clustering

Introduction to Clustering Introduction to Clustering Yumi Kondo Student Seminar LSK301 Sep 25, 2010 Yumi Kondo (University of British Columbia) Introduction to Clustering Sep 25, 2010 1 / 36 Microarray Example N=65 P=1756 Yumi

More information

COM CO P 5318 Da t Da a t Explora Explor t a ion and Analysis y Chapte Chapt r e 3

COM CO P 5318 Da t Da a t Explora Explor t a ion and Analysis y Chapte Chapt r e 3 COMP 5318 Data Exploration and Analysis Chapter 3 What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping

More information

0.1 What is Cluster Analysis?

0.1 What is Cluster Analysis? Cluster Analysis 1 2 0.1 What is Cluster Analysis? Cluster analysis is concerned with forming groups of similar objects based on several measurements of different kinds made on the objects. The key idea

More information

Clustering. Chapter 7. 7.1 Introduction to Clustering Techniques. 7.1.1 Points, Spaces, and Distances

Clustering. Chapter 7. 7.1 Introduction to Clustering Techniques. 7.1.1 Points, Spaces, and Distances 240 Chapter 7 Clustering Clustering is the process of examining a collection of points, and grouping the points into clusters according to some distance measure. The goal is that points in the same cluster

More information

Linköpings Universitet - ITN TNM033 2011-11-30 DBSCAN. A Density-Based Spatial Clustering of Application with Noise

Linköpings Universitet - ITN TNM033 2011-11-30 DBSCAN. A Density-Based Spatial Clustering of Application with Noise DBSCAN A Density-Based Spatial Clustering of Application with Noise Henrik Bäcklund (henba892), Anders Hedblom (andh893), Niklas Neijman (nikne866) 1 1. Introduction Today data is received automatically

More information

Statistical Databases and Registers with some datamining

Statistical Databases and Registers with some datamining Unsupervised learning - Statistical Databases and Registers with some datamining a course in Survey Methodology and O cial Statistics Pages in the book: 501-528 Department of Statistics Stockholm University

More information

Data Mining K-Clustering Problem

Data Mining K-Clustering Problem Data Mining K-Clustering Problem Elham Karoussi Supervisor Associate Professor Noureddine Bouhmala This Master s Thesis is carried out as a part of the education at the University of Agder and is therefore

More information

Personalized Hierarchical Clustering

Personalized Hierarchical Clustering Personalized Hierarchical Clustering Korinna Bade, Andreas Nürnberger Faculty of Computer Science, Otto-von-Guericke-University Magdeburg, D-39106 Magdeburg, Germany {kbade,nuernb}@iws.cs.uni-magdeburg.de

More information

Forschungskolleg Data Analytics Methods and Techniques

Forschungskolleg Data Analytics Methods and Techniques Forschungskolleg Data Analytics Methods and Techniques Martin Hahmann, Gunnar Schröder, Phillip Grosse Prof. Dr.-Ing. Wolfgang Lehner Why do we need it? We are drowning in data, but starving for knowledge!

More information

Mining Social-Network Graphs

Mining Social-Network Graphs 342 Chapter 10 Mining Social-Network Graphs There is much information to be gained by analyzing the large-scale data that is derived from social networks. The best-known example of a social network is

More information

Data Exploration Data Visualization

Data Exploration Data Visualization Data Exploration Data Visualization What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping to select

More information

Graph Mining and Social Network Analysis

Graph Mining and Social Network Analysis Graph Mining and Social Network Analysis Data Mining and Text Mining (UIC 583 @ Politecnico di Milano) References Jiawei Han and Micheline Kamber, "Data Mining: Concepts and Techniques", The Morgan Kaufmann

More information

DHL Data Mining Project. Customer Segmentation with Clustering

DHL Data Mining Project. Customer Segmentation with Clustering DHL Data Mining Project Customer Segmentation with Clustering Timothy TAN Chee Yong Aditya Hridaya MISRA Jeffery JI Jun Yao 3/30/2010 DHL Data Mining Project Table of Contents Introduction to DHL and the

More information

Clustering Techniques: A Brief Survey of Different Clustering Algorithms

Clustering Techniques: A Brief Survey of Different Clustering Algorithms Clustering Techniques: A Brief Survey of Different Clustering Algorithms Deepti Sisodia Technocrates Institute of Technology, Bhopal, India Lokesh Singh Technocrates Institute of Technology, Bhopal, India

More information

An Analysis on Density Based Clustering of Multi Dimensional Spatial Data

An Analysis on Density Based Clustering of Multi Dimensional Spatial Data An Analysis on Density Based Clustering of Multi Dimensional Spatial Data K. Mumtaz 1 Assistant Professor, Department of MCA Vivekanandha Institute of Information and Management Studies, Tiruchengode,

More information

Visualizing non-hierarchical and hierarchical cluster analyses with clustergrams

Visualizing non-hierarchical and hierarchical cluster analyses with clustergrams Visualizing non-hierarchical and hierarchical cluster analyses with clustergrams Matthias Schonlau RAND 7 Main Street Santa Monica, CA 947 USA Summary In hierarchical cluster analysis dendrogram graphs

More information

Cross-validation for detecting and preventing overfitting

Cross-validation for detecting and preventing overfitting Cross-validation for detecting and preventing overfitting Note to other teachers and users of these slides. Andrew would be delighted if ou found this source material useful in giving our own lectures.

More information

How To Perform An Ensemble Analysis

How To Perform An Ensemble Analysis Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598 Outlier Ensembles Keynote, Outlier Detection and Description Workshop, 2013 Based on the ACM SIGKDD Explorations Position Paper: Outlier

More information

Data Mining and Visualization

Data Mining and Visualization Data Mining and Visualization Jeremy Walton NAG Ltd, Oxford Overview Data mining components Functionality Example application Quality control Visualization Use of 3D Example application Market research

More information

Data Clustering Techniques Qualifying Oral Examination Paper

Data Clustering Techniques Qualifying Oral Examination Paper Data Clustering Techniques Qualifying Oral Examination Paper Periklis Andritsos University of Toronto Department of Computer Science periklis@cs.toronto.edu March 11, 2002 1 Introduction During a cholera

More information

Going Big in Data Dimensionality:

Going Big in Data Dimensionality: LUDWIG- MAXIMILIANS- UNIVERSITY MUNICH DEPARTMENT INSTITUTE FOR INFORMATICS DATABASE Going Big in Data Dimensionality: Challenges and Solutions for Mining High Dimensional Data Peer Kröger Lehrstuhl für

More information

LESSON EIII.E EXPONENTS AND LOGARITHMS

LESSON EIII.E EXPONENTS AND LOGARITHMS LESSON EIII.E EXPONENTS AND LOGARITHMS LESSON EIII.E EXPONENTS AND LOGARITHMS OVERVIEW Here s what ou ll learn in this lesson: Eponential Functions a. Graphing eponential functions b. Applications of eponential

More information