Data Mining Cluster Analysis: Basic Concepts and Algorithms. Clustering Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining


 Emil Leonard
 2 years ago
 Views:
Transcription
1 Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Clustering Algorithms Kmeans and its variants Hierarchical clustering Densitbased clustering Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram A tree like diagram that records the sequences of merges or splits Strengths of Hierarchical Clustering Do not have to assume an particular number of clusters An desired number of clusters can be obtained b cutting the dendogram at the proper level The ma correspond to meaningful taonomies Eample in biological sciences (eg, animal kingdom, phlogen reconstruction, ) Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering Two main tpes of hierarchical clustering Agglomerative: Start with the points as individual clusters At each step, merge the closest pair of clusters until onl one cluster (or k clusters) left Divisive: Start with one, allinclusive cluster At each step, split a cluster until each cluster contains a point (or there are k clusters) Traditional hierarchical algorithms use a similarit or distance matri Merge or split one cluster at a time Agglomerative Clustering Algorithm Most popular hierarchical clustering technique Basic algorithm is straightforward Compute the proimit matri Let each data point be a cluster Repeat Merge the two closest clusters Update the proimit matri Until onl a single cluster remains Ke operation is the computation of the proimit of two clusters Different approaches to defining the distance between clusters distinguish the different algorithms Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/
2 Starting Situation Start with clusters of individual points and a proimit matri p p p p p p p p p p Proimit Matri Intermediate Situation After some merging steps, we have some clusters C C C C C C C C C C C C Proimit Matri C C C Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8 Intermediate Situation After Merging We want to merge the two closest clusters (C and C) and update the proimit matri C C C C C C C Proimit Matri C C C C C C The question is How do we update the proimit matri? C C C C C U C C C C C U C??????? C C Proimit Matri C C C U C Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/ How to Define InterCluster Similarit How to Define InterCluster Similarit p p p p p p p p p p Similarit? p p p p p p p p MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p Proimit Matri MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/
3 How to Define InterCluster Similarit How to Define InterCluster Similarit MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p p p p p p p p p p Proimit Matri MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p p p p p p p p p p Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ How to Define InterCluster Similarit Cluster Similarit: MIN or Single Link p p p p p p p p Similarit of two clusters is based on the two most similar (closest) points in the different clusters p MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p Proimit Matri I I I I I I 9 I 9 7 I 7 I 8 I 8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering: MIN Strength of MIN Original Two Clusters Nested Clusters Dendrogram Can handle nonelliptical shapes Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8
4 Limitations of MIN Cluster Similarit: MAX or Complete Linkage Similarit of two clusters is based on the two least similar (most distant) points in the different clusters Original Sensitive to noise and outliers Two Clusters I I I I I I 9 I 9 7 I 7 I 8 I 8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering: MAX Strength of MAX Original Two Clusters Nested Clusters Dendrogram Less susceptible to noise and outliers Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Limitations of MAX Cluster Similarit: Group Average Proimit of two clusters is the average of pairwise proimit between points in the two clusters proimit(p i,pj) p p i Cluster i Cluster j j proimit(cluster i,clusterj) = Cluster Cluster i j Original Tends to break large clusters Biased towards globular clusters Two Clusters I I I I I I 9 I 9 7 I 7 I 8 I 8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/
5 Hierarchical Clustering: Group Average Hierarchical Clustering: Group Average Compromise between Single and Complete Linkage Strengths Less susceptible to noise and outliers Limitations Biased towards globular clusters Nested Clusters Dendrogram Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Cluster Similarit: Ward s Method Hierarchical Clustering: Comparison Similarit of two clusters is based on the increase in squared error when two clusters are merged Similar to group average if distance between points is squared euclidian distance Less susceptible to noise and outliers MIN MAX Biased towards globular clusters Hierarchical analogue of Kmeans Can be used to initialize Kmeans Ward s Method Group Average Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8 Hierarchical Clustering: Eample Hierarchical Clustering: Eample D Use Manhattan distance as distance measure For eample: d(,) = = 7 Look for smallest nondiagonal entr in matri Merge the corresponding clusters Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/
6 Hierarchical Clustering: Eample Hierarchical Clustering: Eample Objects and are most similar, since entr (,) has the smallest value of the distance matri Make a new cluster {,} and remove the single point clusters {} and {} Compute the distance between the newl formed cluster and the remaining clusters {}, {} and {} For eample (single linkage): d({},{,}) = min{d(,),d(,)} = min{7,7} = 7 D {,} 7 7 {,} 7 8 D D {,} {,} {,} 8 {,,} {,} {,,} {,} 8 {,} 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Eample: data points Eample: Dendrogram Cluster Dendrogram 7 Height 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Single Linkage hclust (*, "single") Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering: Time and Space requirements Hierarchical Clustering: Problems and Limitations O(N ) space since it uses the proimit matri N is the number of points O(N ) time in man cases There are N steps and at each step the size, N, proimit matri must be updated and searched Compleit can be reduced to O(N log(n) ) time for some approaches Once a decision is made to combine two clusters, it cannot be undone No objective function is directl minimized Different schemes have problems with one or more of the following: Sensitivit to noise and outliers Difficult handling different sized clusters and conve shapes Breaking large clusters Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/
7 DBSCAN DBSCAN: Core, Border, and Noise DBSCAN is a densitbased algorithm Densit = number of points within a specified radius (Eps) A point is a core point if it has at least a specified number of points (MinPts) within Eps (including the point itself) These are points that are at the interior of a cluster A border point has fewer than MinPts within Eps, but is in the neighborhood of a core point A noise point is an point that is not a core point or a border point Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8 DBSCAN Algorithm DBSCAN: Core, Border and Noise Label all points as core, border or noise Eliminate noise points Perform clustering on the remaining points Put an edge between all core points within Eps of each other Make each group of connected core points into a separate cluster Assign each border point to the cluster of one of its associated core points Original Point tpes: core, border and noise Eps =, MinPts = Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/ When DBSCAN Works Well Potential Problem with DBSCAN A B C D noise noise Original Clusters If we choose Minpts small enough that C and D are found as separate clusters, than A+B+surrounding noise will become a single cluster Resistant to Noise Can handle clusters of different shapes and sizes Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/
8 DBSCAN: Determining EPS and MinPts Idea is that for points in a cluster, their k th nearest neighbors are at roughl the same distance Noise points have the k th nearest neighbor at farther distance So, plot sorted distance of ever point to its k th nearest neighbor Choose Eps = because of sharp increase in distance to th nearest neighbour Cluster Validit For supervised classification we have a variet of measures to evaluate how good our model is Accurac, precision, recall For cluster analsis, the analogous question is how to evaluate the goodness of the resulting clusters? But clusters are in the ee of the beholder! Then wh do we want to evaluate them? To avoid finding patterns in noise To compare clustering algorithms To compare two sets of clusters To compare two clusters Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Clusters found in Random Data Different Aspects of Cluster Validation Random Kmeans DBSCAN Complete Link Determining the clustering tendenc of a set of data, ie, distinguishing whether nonrandom structure actuall eists in the data Comparing the results of a cluster analsis to eternall known results, eg, to eternall given class labels Evaluating how well the results of a cluster analsis fit the data without reference to eternal information  Use onl the data Comparing two sets of clusters to determine which is better Determining the correct number of clusters For,, and, we can further distinguish whether we want to evaluate the entire clustering or just individual clusters 8 8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Measures of Cluster Validit Numerical measures that are applied to judge various aspects of cluster validit, are classified into the following three tpes Eternal Inde: Used to measure the etent to which cluster labels match eternall supplied class labels Entrop Internal Inde: Used to measure the goodness of a clustering structure without reference to eternal information Sum of Squared Error (SSE) Relative Inde: Used to compare two different clusterings or clusters Often an eternal or internal inde is used for this function, eg, SSE or entrop Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Measuring Cluster Validit Via Correlation Two matrices Proimit Matri Incidence Matri One row and one column for each data point An entr is if the associated pair of points belong to the same cluster An entr is if the associated pair of points belongs to different clusters Compute the correlation between the two matrices Since the matrices are smmetric, onl the correlation between n(n) / entries needs to be calculated Strong correlation indicates that points that belong to the same cluster are close to each other Not a good measure for some densit or contiguit based clusters Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8
9 Measuring Cluster Validit Via Correlation Correlation Eample Correlation of incidence and distance matrices for the Kmeans clusterings of the following two data sets D C = {,,} C = {,} (,) (,) (,) corr = 998 Corr = 9 Corr = 8 (,) Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Using Similarit Matri for Cluster Validation Order the similarit matri with respect to cluster labels and inspect visuall Using Similarit Matri for Cluster Validation Clusters in random data are not so crisp Similarit Similarit DBSCAN Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Using Similarit Matri for Cluster Validation Clusters in random data are not so crisp Using Similarit Matri for Cluster Validation Clusters in random data are not so crisp Similarit 8 8 Similarit 8 Kmeans Complete Link Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/
10 Internal Measures: SSE Clusters in more complicated figures aren t well separated Internal Inde: Used to measure the qualit of a clustering without respect to eternal information SSE SSE is good for comparing two clusterings or two clusters (average SSE) Can also be used to estimate the number of clusters Internal Measures: SSE SSE curve for a more complicated data set SSE K Tan,Steinbach, Kumar Introduction to Data Mining /8/ SSE of clusters found using Kmeans Tan,Steinbach, Kumar Introduction to Data Mining /8/ Framework for Cluster Validit Need a framework to interpret an measure For eample, if our measure of evaluation has the value,, is that good, fair, or poor? Statistics provide a framework for cluster validit The more atpical a clustering result is, the more likel it represents valid structure in the data Can compare the values of an inde that result from random data or clusterings to those of a clustering result If the value of the inde is unlikel, then the cluster results are valid These approaches are more complicated and harder to understand For comparing the results of two different sets of clusters, a framework is less necessar However, there is the question of whether the difference between two inde values is significant Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Statistical Framework for SSE Eample Compare SSE of against three clusters in random data Histogram shows SSE of three clusters in sets of random data points of size distributed over the range 8 for and values 8 Count 8 8 SSE Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8 Statistical Framework for Correlation Correlation of incidence and proimit matrices for the Kmeans clusterings of the following two data sets Corr = 9 Corr = 8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Internal Measures: Cohesion and Separation Cluster Cohesion: Measures how closel related are objects in a cluster Eample: SSE Cluster Separation: Measure how distinct or wellseparated a cluster is from other clusters Eample: Squared Error Cohesion is measured b the within cluster sum of squares (SSE) WSS = ( mi ) i C i Separation is measured b the between cluster sum of squares BSS = C ( m i m i i Where C i is the size of cluster i and m is the overall mean Tan,Steinbach, Kumar Introduction to Data Mining /8/ )
11 Internal Measures: Cohesion and Separation Eample: SSE BSS + WSS = constant m m m Internal Measures: Cohesion and Separation A proimit graph based approach can also be used for cohesion and separation Cluster cohesion is the sum of the weight of all links within a cluster Cluster separation is the sum of the weights between nodes in the cluster and nodes outside the cluster K= cluster: WSS= ( ) + ( ) + ( ) + ( ) = BSS= ( ) = Total = + = K= clusters: WSS= ( ) + ( ) + ( ) + ( ) = BSS= ( ) + ( ) = 9 Total = + 9 = cohesion separation Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Internal Measures: Silhouette Coefficient Eternal Measures of Cluster Validit: Entrop and Purit Silhouette Coefficient combine ideas of both cohesion and separation, but for individual points, as well as clusters and clusterings For an individual point, i Calculate a = average distance of i to the points in its cluster Calculate b = min (average distance of i to points in another cluster) The silhouette coefficient for a point is then given b s = a/b if a < b, (or s = b/a  if a b, not the usual case) Tpicall between and The closer to the better a b Can calculate the Average Silhouette width for a cluster or a clustering: does not necessaril increase with k Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Final Comment on Cluster Validit The validation of clustering structures is the most difficult and frustrating part of cluster analsis Without a strong effort in this direction, cluster analsis will remain a black art accessible onl to those true believers who have eperience and great courage Algorithms for Clustering Data, Jain and Dubes Tan,Steinbach, Kumar Introduction to Data Mining /8/
For supervised classification we have a variety of measures to evaluate how good our model is Accuracy, precision, recall
Cluster Validation Cluster Validit For supervised classification we have a variet of measures to evaluate how good our model is Accurac, precision, recall For cluster analsis, the analogous question is
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 What is
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining /8/ What is Cluster
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/2004 Hierarchical
More informationData Mining Clustering (2) Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining
Data Mining Clustering (2) Toon Calders Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining Outline Partitional Clustering Distancebased Kmeans, Kmedoids,
More informationExample: Document Clustering. Clustering: Definition. Notion of a Cluster can be Ambiguous. Types of Clusterings. Hierarchical Clustering
Overview Prognostic Models and Data Mining in Medicine, part I Cluster Analsis What is Cluster Analsis? KMeans Clustering Hierarchical Clustering Cluster Validit Eample: Microarra data analsis 6 Summar
More informationDATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS
DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDDLAB ISTI CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar
More informationCluster Analysis Overview. Data Mining Techniques: Cluster Analysis. What is Cluster Analysis? What is Cluster Analysis?
Cluster Analsis Overview Data Mining Techniques: Cluster Analsis Mirek Riedewald Man slides based on presentations b Han/Kamber, Tan/Steinbach/Kumar, and Andrew Moore Introduction Foundations: Measuring
More informationKMeans Cluster Analysis. Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1
KMeans Cluster Analsis Chapter 3 PPDM Class Tan,Steinbach, Kumar Introduction to Data Mining 4/18/4 1 What is Cluster Analsis? Finding groups of objects such that the objects in a group will be similar
More informationCluster Analysis: Basic Concepts and Algorithms
Cluster Analsis: Basic Concepts and Algorithms What does it mean clustering? Applications Tpes of clustering Kmeans Intuition Algorithm Choosing initial centroids Bisecting Kmeans Postprocessing Strengths
More informationData Mining Clustering. Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining
Data Mining Clustering Toon Calders Sheets are based on the those provided b Tan, Steinbach, and Kumar. Introduction to Data Mining What is Cluster Analsis? Finding groups of objects such that the objects
More informationClustering. Data Mining. Abraham Otero. Data Mining. Agenda
Clustering 1/46 Agenda Introduction Distance Knearest neighbors Hierarchical clustering Quick reference 2/46 1 Introduction It seems logical that in a new situation we should act in a similar way as in
More informationClustering. Adrian Groza. Department of Computer Science Technical University of ClujNapoca
Clustering Adrian Groza Department of Computer Science Technical University of ClujNapoca Outline 1 Cluster Analysis What is Datamining? Cluster Analysis 2 Kmeans 3 Hierarchical Clustering What is Datamining?
More informationClustering UE 141 Spring 2013
Clustering UE 141 Spring 013 Jing Gao SUNY Buffalo 1 Definition of Clustering Finding groups of obects such that the obects in a group will be similar (or related) to one another and different from (or
More informationCLASSIFICATION AND CLUSTERING. Anveshi Charuvaka
CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training
More informationData Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004
More informationData Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004
More informationCluster Analysis: Advanced Concepts
Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototypebased Fuzzy cmeans
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 by Tan, Steinbach, Kumar 1 What is Cluster Analysis? Finding groups of objects such that the objects in a group will
More informationCluster Analysis. Alison Merikangas Data Analysis Seminar 18 November 2009
Cluster Analysis Alison Merikangas Data Analysis Seminar 18 November 2009 Overview What is cluster analysis? Types of cluster Distance functions Clustering methods Agglomerative Kmeans Densitybased Interpretation
More informationCluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico
Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from
More informationData Mining. Cluster Analysis: Advanced Concepts and Algorithms
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototypebased clustering Densitybased clustering Graphbased
More informationUnsupervised learning: Clustering
Unsupervised learning: Clustering Salissou Moutari Centre for Statistical Science and Operational Research CenSSOR 17 th September 2013 Unsupervised learning: Clustering 1/52 Outline 1 Introduction What
More informationSoSe 2014: MTANI: Big Data Analytics
SoSe 2014: MTANI: Big Data Analytics Lecture 4 21/05/2014 Sead Izberovic Dr. Nikolaos Korfiatis Agenda Recap from the previous session Clustering Introduction Distance mesures Hierarchical Clustering
More informationClustering. Clustering. What is Clustering? What is Clustering? What is Clustering? Types of Data in Cluster Analysis
What is Clustering? Clustering Tpes of Data in Cluster Analsis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods What is Clustering? Clustering of data is
More informationCluster Analysis: Basic Concepts and Algorithms
8 Cluster Analysis: Basic Concepts and Algorithms Cluster analysis divides data into groups (clusters) that are meaningful, useful, or both. If meaningful groups are the goal, then the clusters should
More informationChapter 7. Hierarchical cluster analysis. Contents 71
71 Chapter 7 Hierarchical cluster analysis In Part 2 (Chapters 4 to 6) we defined several different ways of measuring distance (or dissimilarity as the case may be) between the rows or between the columns
More informationClustering and Data Mining in R
Clustering and Data Mining in R Workshop Supplement Thomas Girke December 10, 2011 Introduction Data Preprocessing Data Transformations Distance Methods Cluster Linkage Hierarchical Clustering Approaches
More informationClustering & Association
Clustering  Overview What is cluster analysis? Grouping data objects based only on information found in the data describing these objects and their relationships Maximize the similarity within objects
More informationClient Based Power Iteration Clustering Algorithm to Reduce Dimensionality in Big Data
Client Based Power Iteration Clustering Algorithm to Reduce Dimensionalit in Big Data Jaalatchum. D 1, Thambidurai. P 1, Department of CSE, PKIET, Karaikal, India Abstract  Clustering is a group of objects
More informationClustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016
Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with
More informationFig. 1 A typical Knowledge Discovery process [2]
Volume 4, Issue 7, July 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Review on Clustering
More informationCluster Analysis using R
Cluster analysis or clustering is the task of assigning a set of objects into groups (called clusters) so that the objects in the same cluster are more similar (in some sense or another) to each other
More information10810 /02710 Computational Genomics. Clustering expression data
10810 /02710 Computational Genomics Clustering expression data What is Clustering? Organizing data into clusters such that there is high intracluster similarity low intercluster similarity Informally,
More informationSteven M. Ho!and. Department of Geology, University of Georgia, Athens, GA 306022501
CLUSTER ANALYSIS Steven M. Ho!and Department of Geology, University of Georgia, Athens, GA 306022501 January 2006 Introduction Cluster analysis includes a broad suite of techniques designed to find groups
More informationDistances, Clustering, and Classification. Heatmaps
Distances, Clustering, and Classification Heatmaps 1 Distance Clustering organizes things that are close into groups What does it mean for two genes to be close? What does it mean for two samples to be
More informationNeural Networks Lesson 5  Cluster Analysis
Neural Networks Lesson 5  Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt.  Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29
More informationThere are a number of different methods that can be used to carry out a cluster analysis; these methods can be classified as follows:
Statistics: Rosie Cornish. 2007. 3.1 Cluster Analysis 1 Introduction This handout is designed to provide only a brief introduction to cluster analysis and how it is done. Books giving further details are
More informationSummary Data Mining & Process Mining (1BM46) Content. Made by S.P.T. Ariesen
Summary Data Mining & Process Mining (1BM46) Made by S.P.T. Ariesen Content Data Mining part... 2 Lecture 1... 2 Lecture 2:... 4 Lecture 3... 7 Lecture 4... 9 Process mining part... 13 Lecture 5... 13
More informationData Clustering. Dec 2nd, 2013 Kyrylo Bessonov
Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms kmeans Hierarchical Main
More information0.1 What is Cluster Analysis?
Cluster Analysis 1 2 0.1 What is Cluster Analysis? Cluster analysis is concerned with forming groups of similar objects based on several measurements of different kinds made on the objects. The key idea
More informationCluster analysis Cosmin Lazar. COMO Lab VUB
Cluster analysis Cosmin Lazar COMO Lab VUB Introduction Cluster analysis foundations rely on one of the most fundamental, simple and very often unnoticed ways (or methods) of understanding and learning,
More informationTerritorial Analysis for Ratemaking. Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs
Territorial Analysis for Ratemaking by Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs Department of Statistics and Applied Probability University
More informationDistance based clustering
// Distance based clustering Chapter ² ² Clustering Clustering is the art of finding groups in data (Kaufman and Rousseeuw, 99). What is a cluster? Group of objects separated from other clusters Means
More informationData Mining Classification: Alternative Techniques. InstanceBased Classifiers. Lecture Notes for Chapter 5. Introduction to Data Mining
Data Mining Classification: Alternative Techniques InstanceBased Classifiers Lecture Notes for Chapter 5 Introduction to Data Mining by Tan, Steinbach, Kumar Set of Stored Cases Atr1... AtrN Class A B
More informationDecision Support System Methodology Using a Visual Approach for Cluster Analysis Problems
Decision Support System Methodology Using a Visual Approach for Cluster Analysis Problems Ran M. Bittmann School of Business Administration Ph.D. Thesis Submitted to the Senate of BarIlan University RamatGan,
More informationHierarchical Cluster Analysis Some Basics and Algorithms
Hierarchical Cluster Analysis Some Basics and Algorithms Nethra Sambamoorthi CRMportals Inc., 11 Bartram Road, Englishtown, NJ 07726 (NOTE: Please use always the latest copy of the document. Click on this
More informationClustering & Visualization
Chapter 5 Clustering & Visualization Clustering in highdimensional databases is an important problem and there are a number of different clustering paradigms which are applicable to highdimensional data.
More informationA Comparative Study of clustering algorithms Using weka tools
A Comparative Study of clustering algorithms Using weka tools Bharat Chaudhari 1, Manan Parikh 2 1,2 MECSE, KITRC KALOL ABSTRACT Data clustering is a process of putting similar data into groups. A clustering
More informationExploratory data analysis (Chapter 2) Fall 2011
Exploratory data analysis (Chapter 2) Fall 2011 Data Examples Example 1: Survey Data 1 Data collected from a Stat 371 class in Fall 2005 2 They answered questions about their: gender, major, year in school,
More informationData Mining Project Report. Document Clustering. Meryem UzunPer
Data Mining Project Report Document Clustering Meryem UzunPer 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. Kmeans algorithm...
More informationData Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining
Data Mining: Exploring Data Lecture Notes for Chapter 3 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 1 What is data exploration? A preliminary
More informationAn Introduction to Cluster Analysis for Data Mining
An Introduction to Cluster Analysis for Data Mining 10/02/2000 11:42 AM 1. INTRODUCTION... 4 1.1. Scope of This Paper... 4 1.2. What Cluster Analysis Is... 4 1.3. What Cluster Analysis Is Not... 5 2. OVERVIEW...
More informationText Clustering. Clustering
Text Clustering 1 Clustering Partition unlabeled examples into disoint subsets of clusters, such that: Examples within a cluster are very similar Examples in different clusters are very different Discover
More informationIntroduction to Clustering
Introduction to Clustering Yumi Kondo Student Seminar LSK301 Sep 25, 2010 Yumi Kondo (University of British Columbia) Introduction to Clustering Sep 25, 2010 1 / 36 Microarray Example N=65 P=1756 Yumi
More informationChapter 10  Practice Problems 2
Chapter  Practice Problems 2 MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. Determine the regression equation for the data. Round the final values
More informationChapter 7. Cluster Analysis
Chapter 7. Cluster Analysis. What is Cluster Analysis?. A Categorization of Major Clustering Methods. Partitioning Methods. Hierarchical Methods 5. DensityBased Methods 6. GridBased Methods 7. ModelBased
More informationSocial Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers
More informationARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING)
ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Preliminaries Classification and Clustering Applications
More informationData Mining and Visualization
Data Mining and Visualization Jeremy Walton NAG Ltd, Oxford Overview Data mining components Functionality Example application Quality control Visualization Use of 3D Example application Market research
More informationCLUSTER ANALYSIS FOR SEGMENTATION
CLUSTER ANALYSIS FOR SEGMENTATION Introduction We all understand that consumers are not all alike. This provides a challenge for the development and marketing of profitable products and services. Not every
More informationSolving Quadratic Equations by Graphing. Consider an equation of the form. y ax 2 bx c a 0. In an equation of the form
SECTION 11.3 Solving Quadratic Equations b Graphing 11.3 OBJECTIVES 1. Find an ais of smmetr 2. Find a verte 3. Graph a parabola 4. Solve quadratic equations b graphing 5. Solve an application involving
More informationData Preprocessing. Week 2
Data Preprocessing Week 2 Topics Data Types Data Repositories Data Preprocessing Present homework assignment #1 Team Homework Assignment #2 Read pp. 227 240, pp. 250 250, and pp. 259 263 the text book.
More informationA Cluster Analysis Approach for Banks Risk Profile: The Romanian Evidence
109 European Research Studies, Volume XII, Issue (1) 2009 A Cluster Analysis Approach for Banks Risk Profile: The Romanian Evidence By Nicolae DARDAC 1 Iustina Alina BOITAN 2 Abstract: Cluster analysis,
More informationPerformance Metrics for Graph Mining Tasks
Performance Metrics for Graph Mining Tasks 1 Outline Introduction to Performance Metrics Supervised Learning Performance Metrics Unsupervised Learning Performance Metrics Optimizing Metrics Statistical
More informationClustering and Cluster Evaluation. Josh Stuart Tuesday, Feb 24, 2004 Read chap 4 in Causton
Clustering and Cluster Evaluation Josh Stuart Tuesday, Feb 24, 2004 Read chap 4 in Causton Clustering Methods Agglomerative Start with all separate, end with some connected Partitioning / Divisive Start
More informationTime series clustering and the analysis of film style
Time series clustering and the analysis of film style Nick Redfern Introduction Time series clustering provides a simple solution to the problem of searching a database containing time series data such
More informationChapter ML:XI (continued)
Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis DensityBased Cluster Analysis Cluster Evaluation Constrained
More informationData visualization and clustering. Genomics is to no small extend a data science
Data visualization and clustering Genomics is to no small extend a data science [www.data2discovery.org] Data visualization and clustering Genomics is to no small extend a data science [Andersson et al.,
More informationROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015
ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 http://intelligentoptimization.org/lionbook Roberto Battiti
More informationClassification Techniques (1)
10 10 Overview Classification Techniques (1) Today Classification Problem Classification based on Regression Distancebased Classification (KNN) Net Lecture Decision Trees Classification using Rules Quality
More information15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
More information7.2 Application to economics: Leontief Model
7 Application to economics: Leontief Model Wassil Leontief won the Nobel prize in economics in 97 The Leontief model is a model for the economics of a whole countr or region In the model there are n industries
More informationVisualizing nonhierarchical and hierarchical cluster analyses with clustergrams
Visualizing nonhierarchical and hierarchical cluster analyses with clustergrams Matthias Schonlau RAND 7 Main Street Santa Monica, CA 947 USA Summary In hierarchical cluster analysis dendrogram graphs
More informationClustering. 15381 Artificial Intelligence Henry Lin. Organizing data into clusters such that there is
Clustering 15381 Artificial Intelligence Henry Lin Modified from excellent slides of Eamonn Keogh, Ziv BarJoseph, and Andrew Moore What is Clustering? Organizing data into clusters such that there is
More informationCELLULAR MANUFACTURING
CELLULAR MANUFACTURING Grouping Machines logically so that material handling (move time, wait time for moves and using smaller batch sizes) and setup (part family tooling and sequencing) can be minimized.
More informationInformation Retrieval and Web Search Engines
Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 10 th, 2013 WolfTilo Balke and Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig
More informationPERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA
PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA Prakash Singh 1, Aarohi Surya 2 1 Department of Finance, IIM Lucknow, Lucknow, India 2 Department of Computer Science, LNMIIT, Jaipur,
More informationAn Enhanced Clustering Algorithm to Analyze Spatial Data
International Journal of Engineering and Technical Research (IJETR) ISSN: 23210869, Volume2, Issue7, July 2014 An Enhanced Clustering Algorithm to Analyze Spatial Data Dr. Mahesh Kumar, Mr. Sachin Yadav
More informationL15: statistical clustering
Similarity measures Criterion functions Cluster validity Flat clustering algorithms kmeans ISODATA L15: statistical clustering Hierarchical clustering algorithms Divisive Agglomerative CSCE 666 Pattern
More informationCOM CO P 5318 Da t Da a t Explora Explor t a ion and Analysis y Chapte Chapt r e 3
COMP 5318 Data Exploration and Analysis Chapter 3 What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping
More informationLinköpings Universitet  ITN TNM033 20111130 DBSCAN. A DensityBased Spatial Clustering of Application with Noise
DBSCAN A DensityBased Spatial Clustering of Application with Noise Henrik Bäcklund (henba892), Anders Hedblom (andh893), Niklas Neijman (nikne866) 1 1. Introduction Today data is received automatically
More informationMedical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu
Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?
More informationBIRCH: An Efficient Data Clustering Method For Very Large Databases
BIRCH: An Efficient Data Clustering Method For Very Large Databases Tian Zhang, Raghu Ramakrishnan, Miron Livny CPSC 504 Presenter: Discussion Leader: Sophia (Xueyao) Liang HelenJr, Birches. Online Image.
More informationRobotics 2 Clustering & EM. Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard
Robotics 2 Clustering & EM Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard 1 Clustering (1) Common technique for statistical data analysis to detect structure (machine learning,
More informationClustering. Chapter 7. 7.1 Introduction to Clustering Techniques. 7.1.1 Points, Spaces, and Distances
240 Chapter 7 Clustering Clustering is the process of examining a collection of points, and grouping the points into clusters according to some distance measure. The goal is that points in the same cluster
More informationDHL Data Mining Project. Customer Segmentation with Clustering
DHL Data Mining Project Customer Segmentation with Clustering Timothy TAN Chee Yong Aditya Hridaya MISRA Jeffery JI Jun Yao 3/30/2010 DHL Data Mining Project Table of Contents Introduction to DHL and the
More informationCOC131 Data Mining  Clustering
COC131 Data Mining  Clustering Martin D. Sykora m.d.sykora@lboro.ac.uk Tutorial 05, Friday 20th March 2009 1. Fire up Weka (Waikako Environment for Knowledge Analysis) software, launch the explorer window
More informationConcept of Cluster Analysis
RESEARCH PAPER ON CLUSTER TECHNIQUES OF DATA VARIATIONS Er. Arpit Gupta 1,Er.Ankit Gupta 2,Er. Amit Mishra 3 arpit_jp@yahoo.co.in, ank_mgcgv@yahoo.co.in,amitmishra.mtech@gmail.com Faculty Of Engineering
More informationThe Graph of a Linear Equation
4.1 The Graph of a Linear Equation 4.1 OBJECTIVES 1. Find three ordered pairs for an equation in two variables 2. Graph a line from three points 3. Graph a line b the intercept method 4. Graph a line that
More informationChurn problem in retail banking Current methods in churn prediction models Fuzzy cmeans clustering algorithm vs. classical kmeans clustering
CHURN PREDICTION MODEL IN RETAIL BANKING USING FUZZY C MEANS CLUSTERING Džulijana Popović Consumer Finance, Zagrebačka banka d.d. Bojana Dalbelo Bašić Faculty of Electrical Engineering and Computing University
More informationCLASSIFYING SERVICES USING A BINARY VECTOR CLUSTERING ALGORITHM: PRELIMINARY RESULTS
CLASSIFYING SERVICES USING A BINARY VECTOR CLUSTERING ALGORITHM: PRELIMINARY RESULTS Venkat Venkateswaran Department of Engineering and Science Rensselaer Polytechnic Institute 275 Windsor Street Hartford,
More information2 Basic Concepts and Techniques of Cluster Analysis
The Challenges of Clustering High Dimensional Data * Michael Steinbach, Levent Ertöz, and Vipin Kumar Abstract Cluster analysis divides data into groups (clusters) for the purposes of summarization or
More informationMining SocialNetwork Graphs
342 Chapter 10 Mining SocialNetwork Graphs There is much information to be gained by analyzing the largescale data that is derived from social networks. The bestknown example of a social network is
More informationA comparison of various clustering methods and algorithms in data mining
Volume :2, Issue :5, 3236 May 2015 www.allsubjectjournal.com eissn: 23494182 pissn: 23495979 Impact Factor: 3.762 R.Tamilselvi B.Sivasakthi R.Kavitha Assistant Professor A comparison of various clustering
More informationKNIME TUTORIAL. Anna Monreale KDDLab, University of Pisa Email: annam@di.unipi.it
KNIME TUTORIAL Anna Monreale KDDLab, University of Pisa Email: annam@di.unipi.it Outline Introduction on KNIME KNIME components Exercise: Market Basket Analysis Exercise: Customer Segmentation Exercise:
More informationCONTENTS PREFACE 1 INTRODUCTION 1 2 DATA VISUALIZATION 19
PREFACE xi 1 INTRODUCTION 1 1.1 Overview 1 1.2 Definition 1 1.3 Preparation 2 1.3.1 Overview 2 1.3.2 Accessing Tabular Data 3 1.3.3 Accessing Unstructured Data 3 1.3.4 Understanding the Variables and Observations
More informationLecture 20: Clustering
Lecture 20: Clustering Wrapup of neural nets (from last lecture Introduction to unsupervised learning Kmeans clustering COMP424, Lecture 20  April 3, 2013 1 Unsupervised learning In supervised learning,
More information. Learn the number of classes and the structure of each class using similarity between unlabeled training patterns
Outline Part 1: of data clustering NonSupervised Learning and Clustering : Problem formulation cluster analysis : Taxonomies of Clustering Techniques : Data types and Proximity Measures : Difficulties
More informationSession 7 Bivariate Data and Analysis
Session 7 Bivariate Data and Analysis Key Terms for This Session Previously Introduced mean standard deviation New in This Session association bivariate analysis contingency table covariation least squares
More information