# Data Mining Cluster Analysis: Basic Concepts and Algorithms. Clustering Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Save this PDF as:

Size: px
Start display at page:

Download "Data Mining Cluster Analysis: Basic Concepts and Algorithms. Clustering Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining"

## Transcription

1 Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Clustering Algorithms K-means and its variants Hierarchical clustering Densit-based clustering Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram A tree like diagram that records the sequences of merges or splits Strengths of Hierarchical Clustering Do not have to assume an particular number of clusters An desired number of clusters can be obtained b cutting the dendogram at the proper level The ma correspond to meaningful taonomies Eample in biological sciences (eg, animal kingdom, phlogen reconstruction, ) Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering Two main tpes of hierarchical clustering Agglomerative: Start with the points as individual clusters At each step, merge the closest pair of clusters until onl one cluster (or k clusters) left Divisive: Start with one, all-inclusive cluster At each step, split a cluster until each cluster contains a point (or there are k clusters) Traditional hierarchical algorithms use a similarit or distance matri Merge or split one cluster at a time Agglomerative Clustering Algorithm Most popular hierarchical clustering technique Basic algorithm is straightforward Compute the proimit matri Let each data point be a cluster Repeat Merge the two closest clusters Update the proimit matri Until onl a single cluster remains Ke operation is the computation of the proimit of two clusters Different approaches to defining the distance between clusters distinguish the different algorithms Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/

2 Starting Situation Start with clusters of individual points and a proimit matri p p p p p p p p p p Proimit Matri Intermediate Situation After some merging steps, we have some clusters C C C C C C C C C C C C Proimit Matri C C C Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8 Intermediate Situation After Merging We want to merge the two closest clusters (C and C) and update the proimit matri C C C C C C C Proimit Matri C C C C C C The question is How do we update the proimit matri? C C C C C U C C C C C U C??????? C C Proimit Matri C C C U C Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/ How to Define Inter-Cluster Similarit How to Define Inter-Cluster Similarit p p p p p p p p p p Similarit? p p p p p p p p MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p Proimit Matri MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/

3 How to Define Inter-Cluster Similarit How to Define Inter-Cluster Similarit MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p p p p p p p p p p Proimit Matri MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p p p p p p p p p p Proimit Matri Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ How to Define Inter-Cluster Similarit Cluster Similarit: MIN or Single Link p p p p p p p p Similarit of two clusters is based on the two most similar (closest) points in the different clusters p MIN MAX Group Average Distance Between Centroids Other methods driven b an objective function Ward s Method uses squared error p Proimit Matri I I I I I I 9 I 9 7 I 7 I 8 I 8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering: MIN Strength of MIN Original Two Clusters Nested Clusters Dendrogram Can handle non-elliptical shapes Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8

4 Limitations of MIN Cluster Similarit: MAX or Complete Linkage Similarit of two clusters is based on the two least similar (most distant) points in the different clusters Original Sensitive to noise and outliers Two Clusters I I I I I I 9 I 9 7 I 7 I 8 I 8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering: MAX Strength of MAX Original Two Clusters Nested Clusters Dendrogram Less susceptible to noise and outliers Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Limitations of MAX Cluster Similarit: Group Average Proimit of two clusters is the average of pairwise proimit between points in the two clusters proimit(p i,pj) p p i Cluster i Cluster j j proimit(cluster i,clusterj) = Cluster Cluster i j Original Tends to break large clusters Biased towards globular clusters Two Clusters I I I I I I 9 I 9 7 I 7 I 8 I 8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/

5 Hierarchical Clustering: Group Average Hierarchical Clustering: Group Average Compromise between Single and Complete Linkage Strengths Less susceptible to noise and outliers Limitations Biased towards globular clusters Nested Clusters Dendrogram Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Cluster Similarit: Ward s Method Hierarchical Clustering: Comparison Similarit of two clusters is based on the increase in squared error when two clusters are merged Similar to group average if distance between points is squared euclidian distance Less susceptible to noise and outliers MIN MAX Biased towards globular clusters Hierarchical analogue of K-means Can be used to initialize K-means Ward s Method Group Average Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8 Hierarchical Clustering: Eample Hierarchical Clustering: Eample D Use Manhattan distance as distance measure For eample: d(,) = = 7 Look for smallest non-diagonal entr in matri Merge the corresponding clusters Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/

6 Hierarchical Clustering: Eample Hierarchical Clustering: Eample Objects and are most similar, since entr (,) has the smallest value of the distance matri Make a new cluster {,} and remove the single point clusters {} and {} Compute the distance between the newl formed cluster and the remaining clusters {}, {} and {} For eample (single linkage): d({},{,}) = min{d(,),d(,)} = min{7,7} = 7 D {,} 7 7 {,} 7 8 D D {,} {,} {,} 8 {,,} {,} {,,} {,} 8 {,} 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Eample: data points Eample: Dendrogram Cluster Dendrogram 7 Height 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Single Linkage hclust (*, "single") Tan,Steinbach, Kumar Introduction to Data Mining /8/ Hierarchical Clustering: Time and Space requirements Hierarchical Clustering: Problems and Limitations O(N ) space since it uses the proimit matri N is the number of points O(N ) time in man cases There are N steps and at each step the size, N, proimit matri must be updated and searched Compleit can be reduced to O(N log(n) ) time for some approaches Once a decision is made to combine two clusters, it cannot be undone No objective function is directl minimized Different schemes have problems with one or more of the following: Sensitivit to noise and outliers Difficult handling different sized clusters and conve shapes Breaking large clusters Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/

7 DBSCAN DBSCAN: Core, Border, and Noise DBSCAN is a densit-based algorithm Densit = number of points within a specified radius (Eps) A point is a core point if it has at least a specified number of points (MinPts) within Eps (including the point itself) These are points that are at the interior of a cluster A border point has fewer than MinPts within Eps, but is in the neighborhood of a core point A noise point is an point that is not a core point or a border point Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8 DBSCAN Algorithm DBSCAN: Core, Border and Noise Label all points as core, border or noise Eliminate noise points Perform clustering on the remaining points Put an edge between all core points within Eps of each other Make each group of connected core points into a separate cluster Assign each border point to the cluster of one of its associated core points Original Point tpes: core, border and noise Eps =, MinPts = Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/ When DBSCAN Works Well Potential Problem with DBSCAN A B C D noise noise Original Clusters If we choose Minpts small enough that C and D are found as separate clusters, than A+B+surrounding noise will become a single cluster Resistant to Noise Can handle clusters of different shapes and sizes Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/

8 DBSCAN: Determining EPS and MinPts Idea is that for points in a cluster, their k th nearest neighbors are at roughl the same distance Noise points have the k th nearest neighbor at farther distance So, plot sorted distance of ever point to its k th nearest neighbor Choose Eps = because of sharp increase in distance to th nearest neighbour Cluster Validit For supervised classification we have a variet of measures to evaluate how good our model is Accurac, precision, recall For cluster analsis, the analogous question is how to evaluate the goodness of the resulting clusters? But clusters are in the ee of the beholder! Then wh do we want to evaluate them? To avoid finding patterns in noise To compare clustering algorithms To compare two sets of clusters To compare two clusters Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Clusters found in Random Data Different Aspects of Cluster Validation Random K-means DBSCAN Complete Link Determining the clustering tendenc of a set of data, ie, distinguishing whether non-random structure actuall eists in the data Comparing the results of a cluster analsis to eternall known results, eg, to eternall given class labels Evaluating how well the results of a cluster analsis fit the data without reference to eternal information - Use onl the data Comparing two sets of clusters to determine which is better Determining the correct number of clusters For,, and, we can further distinguish whether we want to evaluate the entire clustering or just individual clusters 8 8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Measures of Cluster Validit Numerical measures that are applied to judge various aspects of cluster validit, are classified into the following three tpes Eternal Inde: Used to measure the etent to which cluster labels match eternall supplied class labels Entrop Internal Inde: Used to measure the goodness of a clustering structure without reference to eternal information Sum of Squared Error (SSE) Relative Inde: Used to compare two different clusterings or clusters Often an eternal or internal inde is used for this function, eg, SSE or entrop Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Measuring Cluster Validit Via Correlation Two matrices Proimit Matri Incidence Matri One row and one column for each data point An entr is if the associated pair of points belong to the same cluster An entr is if the associated pair of points belongs to different clusters Compute the correlation between the two matrices Since the matrices are smmetric, onl the correlation between n(n-) / entries needs to be calculated Strong correlation indicates that points that belong to the same cluster are close to each other Not a good measure for some densit or contiguit based clusters Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8

9 Measuring Cluster Validit Via Correlation Correlation Eample Correlation of incidence and distance matrices for the K-means clusterings of the following two data sets D C = {,,} C = {,} (,) (,) (,) corr = -998 Corr = -9 Corr = -8 (,) Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Tan,Steinbach, Kumar Introduction to Data Mining /8/ Using Similarit Matri for Cluster Validation Order the similarit matri with respect to cluster labels and inspect visuall Using Similarit Matri for Cluster Validation Clusters in random data are not so crisp Similarit Similarit DBSCAN Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Using Similarit Matri for Cluster Validation Clusters in random data are not so crisp Using Similarit Matri for Cluster Validation Clusters in random data are not so crisp Similarit 8 8 Similarit 8 K-means Complete Link Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/

10 Internal Measures: SSE Clusters in more complicated figures aren t well separated Internal Inde: Used to measure the qualit of a clustering without respect to eternal information SSE SSE is good for comparing two clusterings or two clusters (average SSE) Can also be used to estimate the number of clusters Internal Measures: SSE SSE curve for a more complicated data set SSE K Tan,Steinbach, Kumar Introduction to Data Mining /8/ SSE of clusters found using K-means Tan,Steinbach, Kumar Introduction to Data Mining /8/ Framework for Cluster Validit Need a framework to interpret an measure For eample, if our measure of evaluation has the value,, is that good, fair, or poor? Statistics provide a framework for cluster validit The more atpical a clustering result is, the more likel it represents valid structure in the data Can compare the values of an inde that result from random data or clusterings to those of a clustering result If the value of the inde is unlikel, then the cluster results are valid These approaches are more complicated and harder to understand For comparing the results of two different sets of clusters, a framework is less necessar However, there is the question of whether the difference between two inde values is significant Tan,Steinbach, Kumar Introduction to Data Mining /8/ 7 Statistical Framework for SSE Eample Compare SSE of against three clusters in random data Histogram shows SSE of three clusters in sets of random data points of size distributed over the range 8 for and values 8 Count 8 8 SSE Tan,Steinbach, Kumar Introduction to Data Mining /8/ 8 Statistical Framework for Correlation Correlation of incidence and proimit matrices for the K-means clusterings of the following two data sets Corr = -9 Corr = -8 Tan,Steinbach, Kumar Introduction to Data Mining /8/ 9 Internal Measures: Cohesion and Separation Cluster Cohesion: Measures how closel related are objects in a cluster Eample: SSE Cluster Separation: Measure how distinct or wellseparated a cluster is from other clusters Eample: Squared Error Cohesion is measured b the within cluster sum of squares (SSE) WSS = ( mi ) i C i Separation is measured b the between cluster sum of squares BSS = C ( m i m i i Where C i is the size of cluster i and m is the overall mean Tan,Steinbach, Kumar Introduction to Data Mining /8/ )

11 Internal Measures: Cohesion and Separation Eample: SSE BSS + WSS = constant m m m Internal Measures: Cohesion and Separation A proimit graph based approach can also be used for cohesion and separation Cluster cohesion is the sum of the weight of all links within a cluster Cluster separation is the sum of the weights between nodes in the cluster and nodes outside the cluster K= cluster: WSS= ( ) + ( ) + ( ) + ( ) = BSS= ( ) = Total = + = K= clusters: WSS= ( ) + ( ) + ( ) + ( ) = BSS= ( ) + ( ) = 9 Total = + 9 = cohesion separation Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Internal Measures: Silhouette Coefficient Eternal Measures of Cluster Validit: Entrop and Purit Silhouette Coefficient combine ideas of both cohesion and separation, but for individual points, as well as clusters and clusterings For an individual point, i Calculate a = average distance of i to the points in its cluster Calculate b = min (average distance of i to points in another cluster) The silhouette coefficient for a point is then given b s = a/b if a < b, (or s = b/a - if a b, not the usual case) Tpicall between and The closer to the better a b Can calculate the Average Silhouette width for a cluster or a clustering: does not necessaril increase with k Tan,Steinbach, Kumar Introduction to Data Mining /8/ Tan,Steinbach, Kumar Introduction to Data Mining /8/ Final Comment on Cluster Validit The validation of clustering structures is the most difficult and frustrating part of cluster analsis Without a strong effort in this direction, cluster analsis will remain a black art accessible onl to those true believers who have eperience and great courage Algorithms for Clustering Data, Jain and Dubes Tan,Steinbach, Kumar Introduction to Data Mining /8/

### For supervised classification we have a variety of measures to evaluate how good our model is Accuracy, precision, recall

Cluster Validation Cluster Validit For supervised classification we have a variet of measures to evaluate how good our model is Accurac, precision, recall For cluster analsis, the analogous question is

### Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 What is

### Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining /8/ What is Cluster

### Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/2004 Hierarchical

### Data Mining Clustering (2) Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining

Data Mining Clustering (2) Toon Calders Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining Outline Partitional Clustering Distance-based K-means, K-medoids,

### Example: Document Clustering. Clustering: Definition. Notion of a Cluster can be Ambiguous. Types of Clusterings. Hierarchical Clustering

Overview Prognostic Models and Data Mining in Medicine, part I Cluster Analsis What is Cluster Analsis? K-Means Clustering Hierarchical Clustering Cluster Validit Eample: Microarra data analsis 6 Summar

### DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDD-LAB ISTI- CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar

### Cluster Analysis Overview. Data Mining Techniques: Cluster Analysis. What is Cluster Analysis? What is Cluster Analysis?

Cluster Analsis Overview Data Mining Techniques: Cluster Analsis Mirek Riedewald Man slides based on presentations b Han/Kamber, Tan/Steinbach/Kumar, and Andrew Moore Introduction Foundations: Measuring

### K-Means Cluster Analysis. Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1

K-Means Cluster Analsis Chapter 3 PPDM Class Tan,Steinbach, Kumar Introduction to Data Mining 4/18/4 1 What is Cluster Analsis? Finding groups of objects such that the objects in a group will be similar

### Cluster Analysis: Basic Concepts and Algorithms

Cluster Analsis: Basic Concepts and Algorithms What does it mean clustering? Applications Tpes of clustering K-means Intuition Algorithm Choosing initial centroids Bisecting K-means Post-processing Strengths

### Data Mining Clustering. Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining

Data Mining Clustering Toon Calders Sheets are based on the those provided b Tan, Steinbach, and Kumar. Introduction to Data Mining What is Cluster Analsis? Finding groups of objects such that the objects

### Clustering. Data Mining. Abraham Otero. Data Mining. Agenda

Clustering 1/46 Agenda Introduction Distance K-nearest neighbors Hierarchical clustering Quick reference 2/46 1 Introduction It seems logical that in a new situation we should act in a similar way as in

### Clustering. Adrian Groza. Department of Computer Science Technical University of Cluj-Napoca

Clustering Adrian Groza Department of Computer Science Technical University of Cluj-Napoca Outline 1 Cluster Analysis What is Datamining? Cluster Analysis 2 K-means 3 Hierarchical Clustering What is Datamining?

### Clustering UE 141 Spring 2013

Clustering UE 141 Spring 013 Jing Gao SUNY Buffalo 1 Definition of Clustering Finding groups of obects such that the obects in a group will be similar (or related) to one another and different from (or

### CLASSIFICATION AND CLUSTERING. Anveshi Charuvaka

CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training

### Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

### Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototype-based Fuzzy c-means

### Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 by Tan, Steinbach, Kumar 1 What is Cluster Analysis? Finding groups of objects such that the objects in a group will

### Cluster Analysis. Alison Merikangas Data Analysis Seminar 18 November 2009

Cluster Analysis Alison Merikangas Data Analysis Seminar 18 November 2009 Overview What is cluster analysis? Types of cluster Distance functions Clustering methods Agglomerative K-means Density-based Interpretation

### Cluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico

Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from

### Data Mining. Cluster Analysis: Advanced Concepts and Algorithms

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototype-based clustering Density-based clustering Graph-based

### Unsupervised learning: Clustering

Unsupervised learning: Clustering Salissou Moutari Centre for Statistical Science and Operational Research CenSSOR 17 th September 2013 Unsupervised learning: Clustering 1/52 Outline 1 Introduction What

### SoSe 2014: M-TANI: Big Data Analytics

SoSe 2014: M-TANI: Big Data Analytics Lecture 4 21/05/2014 Sead Izberovic Dr. Nikolaos Korfiatis Agenda Recap from the previous session Clustering Introduction Distance mesures Hierarchical Clustering

### Clustering. Clustering. What is Clustering? What is Clustering? What is Clustering? Types of Data in Cluster Analysis

What is Clustering? Clustering Tpes of Data in Cluster Analsis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods What is Clustering? Clustering of data is

### Cluster Analysis: Basic Concepts and Algorithms

8 Cluster Analysis: Basic Concepts and Algorithms Cluster analysis divides data into groups (clusters) that are meaningful, useful, or both. If meaningful groups are the goal, then the clusters should

### Chapter 7. Hierarchical cluster analysis. Contents 7-1

7-1 Chapter 7 Hierarchical cluster analysis In Part 2 (Chapters 4 to 6) we defined several different ways of measuring distance (or dissimilarity as the case may be) between the rows or between the columns

### Clustering and Data Mining in R

Clustering and Data Mining in R Workshop Supplement Thomas Girke December 10, 2011 Introduction Data Preprocessing Data Transformations Distance Methods Cluster Linkage Hierarchical Clustering Approaches

### Clustering & Association

Clustering - Overview What is cluster analysis? Grouping data objects based only on information found in the data describing these objects and their relationships Maximize the similarity within objects

### Client Based Power Iteration Clustering Algorithm to Reduce Dimensionality in Big Data

Client Based Power Iteration Clustering Algorithm to Reduce Dimensionalit in Big Data Jaalatchum. D 1, Thambidurai. P 1, Department of CSE, PKIET, Karaikal, India Abstract - Clustering is a group of objects

### Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016

Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with

### Fig. 1 A typical Knowledge Discovery process [2]

Volume 4, Issue 7, July 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Review on Clustering

### Cluster Analysis using R

Cluster analysis or clustering is the task of assigning a set of objects into groups (called clusters) so that the objects in the same cluster are more similar (in some sense or another) to each other

### 10-810 /02-710 Computational Genomics. Clustering expression data

10-810 /02-710 Computational Genomics Clustering expression data What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally,

### Steven M. Ho!and. Department of Geology, University of Georgia, Athens, GA 30602-2501

CLUSTER ANALYSIS Steven M. Ho!and Department of Geology, University of Georgia, Athens, GA 30602-2501 January 2006 Introduction Cluster analysis includes a broad suite of techniques designed to find groups

### Distances, Clustering, and Classification. Heatmaps

Distances, Clustering, and Classification Heatmaps 1 Distance Clustering organizes things that are close into groups What does it mean for two genes to be close? What does it mean for two samples to be

### Neural Networks Lesson 5 - Cluster Analysis

Neural Networks Lesson 5 - Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt. - Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29

### There are a number of different methods that can be used to carry out a cluster analysis; these methods can be classified as follows:

Statistics: Rosie Cornish. 2007. 3.1 Cluster Analysis 1 Introduction This handout is designed to provide only a brief introduction to cluster analysis and how it is done. Books giving further details are

### Summary Data Mining & Process Mining (1BM46) Content. Made by S.P.T. Ariesen

Summary Data Mining & Process Mining (1BM46) Made by S.P.T. Ariesen Content Data Mining part... 2 Lecture 1... 2 Lecture 2:... 4 Lecture 3... 7 Lecture 4... 9 Process mining part... 13 Lecture 5... 13

### Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov

Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical Main

### 0.1 What is Cluster Analysis?

Cluster Analysis 1 2 0.1 What is Cluster Analysis? Cluster analysis is concerned with forming groups of similar objects based on several measurements of different kinds made on the objects. The key idea

### Cluster analysis Cosmin Lazar. COMO Lab VUB

Cluster analysis Cosmin Lazar COMO Lab VUB Introduction Cluster analysis foundations rely on one of the most fundamental, simple and very often unnoticed ways (or methods) of understanding and learning,

### Territorial Analysis for Ratemaking. Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs

Territorial Analysis for Ratemaking by Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs Department of Statistics and Applied Probability University

### Distance based clustering

// Distance based clustering Chapter ² ² Clustering Clustering is the art of finding groups in data (Kaufman and Rousseeuw, 99). What is a cluster? Group of objects separated from other clusters Means

### Data Mining Classification: Alternative Techniques. Instance-Based Classifiers. Lecture Notes for Chapter 5. Introduction to Data Mining

Data Mining Classification: Alternative Techniques Instance-Based Classifiers Lecture Notes for Chapter 5 Introduction to Data Mining by Tan, Steinbach, Kumar Set of Stored Cases Atr1... AtrN Class A B

### Decision Support System Methodology Using a Visual Approach for Cluster Analysis Problems

Decision Support System Methodology Using a Visual Approach for Cluster Analysis Problems Ran M. Bittmann School of Business Administration Ph.D. Thesis Submitted to the Senate of Bar-Ilan University Ramat-Gan,

### Hierarchical Cluster Analysis Some Basics and Algorithms

Hierarchical Cluster Analysis Some Basics and Algorithms Nethra Sambamoorthi CRMportals Inc., 11 Bartram Road, Englishtown, NJ 07726 (NOTE: Please use always the latest copy of the document. Click on this

### Clustering & Visualization

Chapter 5 Clustering & Visualization Clustering in high-dimensional databases is an important problem and there are a number of different clustering paradigms which are applicable to high-dimensional data.

### A Comparative Study of clustering algorithms Using weka tools

A Comparative Study of clustering algorithms Using weka tools Bharat Chaudhari 1, Manan Parikh 2 1,2 MECSE, KITRC KALOL ABSTRACT Data clustering is a process of putting similar data into groups. A clustering

### Exploratory data analysis (Chapter 2) Fall 2011

Exploratory data analysis (Chapter 2) Fall 2011 Data Examples Example 1: Survey Data 1 Data collected from a Stat 371 class in Fall 2005 2 They answered questions about their: gender, major, year in school,

### Data Mining Project Report. Document Clustering. Meryem Uzun-Per

Data Mining Project Report Document Clustering Meryem Uzun-Per 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. K-means algorithm...

### Data Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining

Data Mining: Exploring Data Lecture Notes for Chapter 3 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 1 What is data exploration? A preliminary

### An Introduction to Cluster Analysis for Data Mining

An Introduction to Cluster Analysis for Data Mining 10/02/2000 11:42 AM 1. INTRODUCTION... 4 1.1. Scope of This Paper... 4 1.2. What Cluster Analysis Is... 4 1.3. What Cluster Analysis Is Not... 5 2. OVERVIEW...

### Text Clustering. Clustering

Text Clustering 1 Clustering Partition unlabeled examples into disoint subsets of clusters, such that: Examples within a cluster are very similar Examples in different clusters are very different Discover

### Introduction to Clustering

Introduction to Clustering Yumi Kondo Student Seminar LSK301 Sep 25, 2010 Yumi Kondo (University of British Columbia) Introduction to Clustering Sep 25, 2010 1 / 36 Microarray Example N=65 P=1756 Yumi

### Chapter 10 - Practice Problems 2

Chapter - Practice Problems 2 MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. Determine the regression equation for the data. Round the final values

### Chapter 7. Cluster Analysis

Chapter 7. Cluster Analysis. What is Cluster Analysis?. A Categorization of Major Clustering Methods. Partitioning Methods. Hierarchical Methods 5. Density-Based Methods 6. Grid-Based Methods 7. Model-Based

### Social Media Mining. Data Mining Essentials

Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

### ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING)

ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Preliminaries Classification and Clustering Applications

### Data Mining and Visualization

Data Mining and Visualization Jeremy Walton NAG Ltd, Oxford Overview Data mining components Functionality Example application Quality control Visualization Use of 3D Example application Market research

### CLUSTER ANALYSIS FOR SEGMENTATION

CLUSTER ANALYSIS FOR SEGMENTATION Introduction We all understand that consumers are not all alike. This provides a challenge for the development and marketing of profitable products and services. Not every

### Solving Quadratic Equations by Graphing. Consider an equation of the form. y ax 2 bx c a 0. In an equation of the form

SECTION 11.3 Solving Quadratic Equations b Graphing 11.3 OBJECTIVES 1. Find an ais of smmetr 2. Find a verte 3. Graph a parabola 4. Solve quadratic equations b graphing 5. Solve an application involving

### Data Preprocessing. Week 2

Data Preprocessing Week 2 Topics Data Types Data Repositories Data Preprocessing Present homework assignment #1 Team Homework Assignment #2 Read pp. 227 240, pp. 250 250, and pp. 259 263 the text book.

### A Cluster Analysis Approach for Banks Risk Profile: The Romanian Evidence

109 European Research Studies, Volume XII, Issue (1) 2009 A Cluster Analysis Approach for Banks Risk Profile: The Romanian Evidence By Nicolae DARDAC 1 Iustina Alina BOITAN 2 Abstract: Cluster analysis,

### Performance Metrics for Graph Mining Tasks

Performance Metrics for Graph Mining Tasks 1 Outline Introduction to Performance Metrics Supervised Learning Performance Metrics Unsupervised Learning Performance Metrics Optimizing Metrics Statistical

### Clustering and Cluster Evaluation. Josh Stuart Tuesday, Feb 24, 2004 Read chap 4 in Causton

Clustering and Cluster Evaluation Josh Stuart Tuesday, Feb 24, 2004 Read chap 4 in Causton Clustering Methods Agglomerative Start with all separate, end with some connected Partitioning / Divisive Start

### Time series clustering and the analysis of film style

Time series clustering and the analysis of film style Nick Redfern Introduction Time series clustering provides a simple solution to the problem of searching a database containing time series data such

### Chapter ML:XI (continued)

Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained

### Data visualization and clustering. Genomics is to no small extend a data science

Data visualization and clustering Genomics is to no small extend a data science [www.data2discovery.org] Data visualization and clustering Genomics is to no small extend a data science [Andersson et al.,

### ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 http://intelligentoptimization.org/lionbook Roberto Battiti

### Classification Techniques (1)

10 10 Overview Classification Techniques (1) Today Classification Problem Classification based on Regression Distance-based Classification (KNN) Net Lecture Decision Trees Classification using Rules Quality

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### 7.2 Application to economics: Leontief Model

7 Application to economics: Leontief Model Wassil Leontief won the Nobel prize in economics in 97 The Leontief model is a model for the economics of a whole countr or region In the model there are n industries

### Visualizing non-hierarchical and hierarchical cluster analyses with clustergrams

Visualizing non-hierarchical and hierarchical cluster analyses with clustergrams Matthias Schonlau RAND 7 Main Street Santa Monica, CA 947 USA Summary In hierarchical cluster analysis dendrogram graphs

### Clustering. 15-381 Artificial Intelligence Henry Lin. Organizing data into clusters such that there is

Clustering 15-381 Artificial Intelligence Henry Lin Modified from excellent slides of Eamonn Keogh, Ziv Bar-Joseph, and Andrew Moore What is Clustering? Organizing data into clusters such that there is

### CELLULAR MANUFACTURING

CELLULAR MANUFACTURING Grouping Machines logically so that material handling (move time, wait time for moves and using smaller batch sizes) and setup (part family tooling and sequencing) can be minimized.

### Information Retrieval and Web Search Engines

Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 10 th, 2013 Wolf-Tilo Balke and Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig

### PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA

PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA Prakash Singh 1, Aarohi Surya 2 1 Department of Finance, IIM Lucknow, Lucknow, India 2 Department of Computer Science, LNMIIT, Jaipur,

### An Enhanced Clustering Algorithm to Analyze Spatial Data

International Journal of Engineering and Technical Research (IJETR) ISSN: 2321-0869, Volume-2, Issue-7, July 2014 An Enhanced Clustering Algorithm to Analyze Spatial Data Dr. Mahesh Kumar, Mr. Sachin Yadav

### L15: statistical clustering

Similarity measures Criterion functions Cluster validity Flat clustering algorithms k-means ISODATA L15: statistical clustering Hierarchical clustering algorithms Divisive Agglomerative CSCE 666 Pattern

### COM CO P 5318 Da t Da a t Explora Explor t a ion and Analysis y Chapte Chapt r e 3

COMP 5318 Data Exploration and Analysis Chapter 3 What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping

### Linköpings Universitet - ITN TNM033 2011-11-30 DBSCAN. A Density-Based Spatial Clustering of Application with Noise

DBSCAN A Density-Based Spatial Clustering of Application with Noise Henrik Bäcklund (henba892), Anders Hedblom (andh893), Niklas Neijman (nikne866) 1 1. Introduction Today data is received automatically

### Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu

Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?

### BIRCH: An Efficient Data Clustering Method For Very Large Databases

BIRCH: An Efficient Data Clustering Method For Very Large Databases Tian Zhang, Raghu Ramakrishnan, Miron Livny CPSC 504 Presenter: Discussion Leader: Sophia (Xueyao) Liang HelenJr, Birches. Online Image.

### Robotics 2 Clustering & EM. Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard

Robotics 2 Clustering & EM Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard 1 Clustering (1) Common technique for statistical data analysis to detect structure (machine learning,

### Clustering. Chapter 7. 7.1 Introduction to Clustering Techniques. 7.1.1 Points, Spaces, and Distances

240 Chapter 7 Clustering Clustering is the process of examining a collection of points, and grouping the points into clusters according to some distance measure. The goal is that points in the same cluster

### DHL Data Mining Project. Customer Segmentation with Clustering

DHL Data Mining Project Customer Segmentation with Clustering Timothy TAN Chee Yong Aditya Hridaya MISRA Jeffery JI Jun Yao 3/30/2010 DHL Data Mining Project Table of Contents Introduction to DHL and the

### COC131 Data Mining - Clustering

COC131 Data Mining - Clustering Martin D. Sykora m.d.sykora@lboro.ac.uk Tutorial 05, Friday 20th March 2009 1. Fire up Weka (Waikako Environment for Knowledge Analysis) software, launch the explorer window

### Concept of Cluster Analysis

RESEARCH PAPER ON CLUSTER TECHNIQUES OF DATA VARIATIONS Er. Arpit Gupta 1,Er.Ankit Gupta 2,Er. Amit Mishra 3 arpit_jp@yahoo.co.in, ank_mgcgv@yahoo.co.in,amitmishra.mtech@gmail.com Faculty Of Engineering

### The Graph of a Linear Equation

4.1 The Graph of a Linear Equation 4.1 OBJECTIVES 1. Find three ordered pairs for an equation in two variables 2. Graph a line from three points 3. Graph a line b the intercept method 4. Graph a line that

### Churn problem in retail banking Current methods in churn prediction models Fuzzy c-means clustering algorithm vs. classical k-means clustering

CHURN PREDICTION MODEL IN RETAIL BANKING USING FUZZY C- MEANS CLUSTERING Džulijana Popović Consumer Finance, Zagrebačka banka d.d. Bojana Dalbelo Bašić Faculty of Electrical Engineering and Computing University

### CLASSIFYING SERVICES USING A BINARY VECTOR CLUSTERING ALGORITHM: PRELIMINARY RESULTS

CLASSIFYING SERVICES USING A BINARY VECTOR CLUSTERING ALGORITHM: PRELIMINARY RESULTS Venkat Venkateswaran Department of Engineering and Science Rensselaer Polytechnic Institute 275 Windsor Street Hartford,

### 2 Basic Concepts and Techniques of Cluster Analysis

The Challenges of Clustering High Dimensional Data * Michael Steinbach, Levent Ertöz, and Vipin Kumar Abstract Cluster analysis divides data into groups (clusters) for the purposes of summarization or

### Mining Social-Network Graphs

342 Chapter 10 Mining Social-Network Graphs There is much information to be gained by analyzing the large-scale data that is derived from social networks. The best-known example of a social network is

### A comparison of various clustering methods and algorithms in data mining

Volume :2, Issue :5, 32-36 May 2015 www.allsubjectjournal.com e-issn: 2349-4182 p-issn: 2349-5979 Impact Factor: 3.762 R.Tamilselvi B.Sivasakthi R.Kavitha Assistant Professor A comparison of various clustering

### KNIME TUTORIAL. Anna Monreale KDD-Lab, University of Pisa Email: annam@di.unipi.it

KNIME TUTORIAL Anna Monreale KDD-Lab, University of Pisa Email: annam@di.unipi.it Outline Introduction on KNIME KNIME components Exercise: Market Basket Analysis Exercise: Customer Segmentation Exercise:

### CONTENTS PREFACE 1 INTRODUCTION 1 2 DATA VISUALIZATION 19

PREFACE xi 1 INTRODUCTION 1 1.1 Overview 1 1.2 Definition 1 1.3 Preparation 2 1.3.1 Overview 2 1.3.2 Accessing Tabular Data 3 1.3.3 Accessing Unstructured Data 3 1.3.4 Understanding the Variables and Observations

### Lecture 20: Clustering

Lecture 20: Clustering Wrap-up of neural nets (from last lecture Introduction to unsupervised learning K-means clustering COMP-424, Lecture 20 - April 3, 2013 1 Unsupervised learning In supervised learning,