Save this PDF as:

Size: px
Start display at page:

Transcription

1 Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/ Introduction to Data Mining 08/06/ Outline Prototype-based Fuzzy c-means Mixture Model Clustering Self-Organizing Maps Density-based Grid-based clustering Subspace clustering Graph-based Chameleon Jarvis-Patrick Shared Nearest Neighbor (SNN) Characteristics of Clustering Algorithms Introduction to Data Mining 08/06/2006 2

2 Hard (Crisp) vs Soft (Fuzzy) Clustering Hard (Crisp) vs. Soft (Fuzzy) clustering Generalize K-means objective function k N j= 1 i= 1 2 ( x c ), SSE = w w ij = 1 ij i j k j= 1 w ij : weight with which object x i belongs to cluster C j To minimize SSE, repeat the following steps: Fixed c j and determine w ij (cluster assignment) Fixed w ij and recompute c j Hard clustering: w ij {0,1} Introduction to Data Mining 08/06/ Hard (Crisp) vs Soft (Fuzzy) Clustering c 1 x c SSE( x) = w = w x1 x1 (2 1) + 9w 2 x2 + w x2 (5 2) 2 SSE(x) is minimized when w x1 = 1, w x2 = 0 Introduction to Data Mining 08/06/2006 4

3 Fuzzy C-means Objective function k p: fuzzifier (p > 1) 2 ( x c ), p SSE = w N j= 1 i= 1 ij i j k w ij j= 1 = 1 w ij : weight with which object x i belongs to cluster C j To minimize objective function, repeat the following: Fixed c j and determine w ij Fixed w ij and recompute c j Fuzzy clustering: w ij [0,1] Introduction to Data Mining 08/06/ Fuzzy C-means c 1 x c E( x) = w = w 2 x1 2 x1 (2 1) + 9w 2 2 x2 + w 2 x2 (5 2) 2 SSE(x) is minimized when w x1 = 0.9, w x2 = 0.1 Introduction to Data Mining 08/06/2006 6

4 Fuzzy C-means Objective function: k N 2 ( x c ), p SSE = w wij = 1 j= 1 i= 1 ij p: fuzzifier (p > 1) Initialization: choose the weights w ij randomly Repeat: Update centroids: Update weights: i j k j=1 Introduction to Data Mining 08/06/ Fuzzy K-means Applied to Sample Data maximum membership Introduction to Data Mining 08/06/2006 8

5 Hard (Crisp) vs Soft (Probabilistic) Clustering Idea is to model the set of data points as arising from a mixture of distributions Typically, y normal (Gaussian) distribution is used But other distributions have been very profitably used. Clusters are found by estimating the parameters of the statistical distributions Can use a k-means like algorithm, called the EM algorithm, to estimate these parameters Actually, k-means is a special case of this approach Provides a compact representation of clusters The probabilities with which point belongs to each cluster provide a functionality similar to fuzzy clustering. Introduction to Data Mining 08/06/ Probabilistic Clustering: Example Informal example: consider modeling the points that generate the following histogram. Looks like a combination of two normal distributions Suppose we can estimate the mean and standard deviation of each normal distribution. This completely describes the two clusters We can compute the probabilities with which each point belongs to each cluster Can assign each point to the cluster (distribution) in which it is most probable. Introduction to Data Mining 08/06/

6 Probabilistic Clustering: EM Algorithm Initialize the parameters Repeat For each point, compute its probability under each distribution Using these probabilities, update the parameters of each distribution Until there is not change Very similar to of K-means Consists of assignment and update steps Can use random initialization Problem of local minima For normal distributions, typically use K-means to initialize If using normal distributions, can find elliptical as well as spherical shapes. Introduction to Data Mining 08/06/ Probabilistic Clustering: Updating Centroids Update formula for the weights m c j = x i p( j x i ) / p( j x i ) Very similar to the fuzzy k-means formula Weights are not raised to a power Weights are probabilities These probabilities are calculated using Bayes rule Need to assign weights to each cluster Weights may not be equal Similar to prior probabilities Can be estimated, if desired m i= 1 i= 1 Introduction to Data Mining 08/06/

7 Probabilistic Clustering Applied to Sample Data maximum probability Introduction to Data Mining 08/06/ Probabilistic Clustering: Dense and Sparse Clusters 6 4? 2 0 y x Introduction to Data Mining 08/06/

8 SOM: Self-Organizing Maps Self-organizing maps (SOM) Centroid based clustering scheme Like K-means, a fixed number of clusters are specified However, the spatial relationship of clusters is also specified, typically as a grid Points are considered one by one Each point is assigned to the closest centroid Other centroids are updated based on their nearness to the closest centroid Introduction to Data Mining 08/06/ SOM: Self-Organizing Maps Updates are weighted by distance Centroids farther away are affected less The impact of the updates decreases with each time At some point the centroids will not change much Introduction to Data Mining 08/06/

9 SOM: Self-Organizing Maps SOM can be viewed as a type of dimensionality reduction If a two-dimensional i grid is used, the results can be visualized Introduction to Data Mining 08/06/ SOM Clusters of LA Times Document Data Introduction to Data Mining 08/06/

10 Grid-based Clustering A type of density-based clustering Introduction to Data Mining 08/06/ Subspace Clustering Until now, we found clusters by considering all of the attributes Some clusters may involve only a subset of attributes, i.e., subspaces of the data Example: When k-means is used to find document clusters, the resulting clusters can typically be characterized by 10 or so terms Introduction to Data Mining 08/06/

11 Example Introduction to Data Mining 08/06/ Example Introduction to Data Mining 08/06/

12 Example Introduction to Data Mining 08/06/ Example Introduction to Data Mining 08/06/

13 Clique Algorithm - Overview A grid-based clustering algorithm that methodically finds subspace clusters Partitions the data space into rectangular units of equal volume Measures the density of each unit by the fraction of points it contains A unit is dense if the fraction of overall points it contains is above a user specified threshold, τ A cluster is a group of collections of contiguous (touching) dense units Introduction to Data Mining 08/06/ Clique Algorithm It is impractical to check each volume unit to see if it is dense since there is exponential number of such units Monotone property of density-based clusters: If a set of points forms a density based cluster in k dimensions, then the same set of points is also part of a density based cluster in all possible subsets of those dimensions Very similar to Apriori algorithm Can find overlapping clusters Introduction to Data Mining 08/06/

14 Clique Algorithm Introduction to Data Mining 08/06/ Limitations of Clique Time complexity is exponential in number of dimensions Especially if too many dense units are generated at lower stages May fail if clusters are of widely differing densities, since the threshold is fixed Determining appropriate threshold and unit interval length can be challenging Introduction to Data Mining 08/06/

15 Denclue (DENsity CLUstering) Based on the notion of kernel-density estimation Contribution of each point to the density is given by a an influence or kernel function Formula and plot of Gaussian Kernel Overall density is the sum of the contributions of all points Introduction to Data Mining 08/06/ Example of Density from Gaussian Kernel Introduction to Data Mining 08/06/

16 DENCLUE Algorithm Introduction to Data Mining 08/06/ DENCLUE Algorithm Find the density function Identify local maxima (density attractors) Assign each point to the density attractor Follow direction of maximum increase in density Define clusters as groups consisting of points associated with density attractor Discard clusters whose density attractor has a density less than a user specified minimum, ξ Combine clusters connected by paths of points that are connected by points with density above ξ Introduction to Data Mining 08/06/

17 Graph-Based Clustering: General Concepts Graph-Based clustering uses the proximity graph Start with the proximity matrix Consider each point as a node in a graph Each edge between two nodes has a weight which is the proximity between the two points Initially the proximity graph is fully connected MIN (single-link) and MAX (complete-link) can be viewed as starting with this graph In the simplest case, clusters are connected components in the graph. Introduction to Data Mining 08/06/ CURE Algorithm: Graph-Based Clustering Agglomerative hierarchical clustering algorithms vary in terms of how the proximity of two clusters are computed MIN (single link) susceptible to noise/outliers MAX (complete link)/group AVERAGE/Centroid/Ward s: may not work well with non-globular clusters CURE algorithm tries to handle both problems Starts with a proximity matrix/proximity graph Introduction to Data Mining 08/06/

18 CURE Algorithm Represents a cluster using multiple representative points Representative points are found by selecting a constant number of points from a cluster First representative point is chosen to be the point furthest from the center of the cluster Remaining representative points are chosen so that they are farthest from all previously chosen points Introduction to Data Mining 08/06/ CURE Algorithm Shrink representative points toward the center of the cluster by a factor, α Shrinking representative points toward the center helps avoid problems with noise and outliers Cluster similarity is the similarity of the closest pair of representative points from different clusters Introduction to Data Mining 08/06/

19 CURE Algorithm Uses agglomerative hierarchical scheme to perform clustering; α = 0: similar to centroid-based α = 1: somewhat similar to single-link CURE is better able to handle clusters of arbitrary shapes and sizes Introduction to Data Mining 08/06/ Experimental Results: CURE Picture from CURE, Guha, Rastogi, Shim. Introduction to Data Mining 08/06/

20 Experimental Results: CURE (centroid) (single link) Picture from CURE, Guha, Rastogi, Shim. Introduction to Data Mining 08/06/ CURE Cannot Handle Differing Densities Original Points CURE Introduction to Data Mining 08/06/

21 Graph-Based Clustering: Chameleon Based on several key ideas Sparsification of the proximity it graph Partitioning the data into clusters that are relatively pure subclusters of the true clusters Merging based on preserving characteristics of clusters Introduction to Data Mining 08/06/ Graph-Based Clustering: Sparsification The amount of data that needs to be processed is drastically reduced Sparsification can eliminate more than 99% of the entries in a proximity matrix The amount of time required to cluster the data is drastically reduced The size of the problems that can be handled is increased Introduction to Data Mining 08/06/

22 Graph-Based Clustering: Sparsification Clustering may work better Sparsification techniques keep the connections to the most similar (nearest) neighbors of a point while breaking the connections to less similar il points. The nearest neighbors of a point tend to belong to the same class as the point itself. This reduces the impact of noise and outliers and sharpens the distinction between clusters. Sparsification facilitates the use of graph partitioning algorithms (or algorithms based on graph partitioning algorithms) Chameleon and Hypergraph-based Clustering Introduction to Data Mining 08/06/ Sparsification in the Clustering Process Introduction to Data Mining 08/06/

23 Limitations of Current Merging Schemes Existing merging schemes in hierarchical clustering algorithms are static in nature MIN or CURE: Merge two clusters based on their closeness (or minimum distance) GROUP-AVERAGE: Merge two clusters based on their average connectivity Introduction to Data Mining 08/06/ Limitations of Current Merging Schemes (a) (b) (c) (d) Closeness schemes will merge (a) and (b) Average connectivity schemes will merge (c) and (d) Introduction to Data Mining 08/06/

24 Chameleon: Clustering Using Dynamic Modeling Adapt to the characteristics of the data set to find the natural clusters Use a dynamic model to measure the similarity between clusters Main properties are the relative closeness and relative interconnectivity of the cluster Two clusters are combined if the resulting cluster shares certain properties with the constituent clusters The merging scheme preserves self-similarity Introduction to Data Mining 08/06/ Relative Interconnectivity Introduction to Data Mining 08/06/

25 Relative Closeness Introduction to Data Mining 08/06/ Chameleon: Steps Preprocessing Step: Represent the data by a Graph Given a set of points, construct the k-nearest- neighbor (k-nn) graph to capture the relationship between a point and its k nearest neighbors Concept of neighborhood is captured dynamically (even if region is sparse) Phase 1: Use a multilevel graph partitioning algorithm on the graph to find a large number of clusters of well-connected vertices Each cluster should contain mostly points from one true cluster, i.e., is a sub-cluster of a real cluster Introduction to Data Mining 08/06/

26 Chameleon: Steps Phase 2: Use Hierarchical Agglomerative Clustering to merge sub-clusters Two clusters are combined if the resulting cluster shares certain properties with the constituent clusters Two key properties used to model cluster similarity: Relative Interconnectivity: Absolute interconnectivity of two clusters normalized by the internal connectivity of the clusters Relative Closeness: Absolute closeness of two clusters normalized by the internal closeness of the clusters Introduction to Data Mining 08/06/ Experimental Results: CHAMELEON Introduction to Data Mining 08/06/

27 Experimental Results: CHAMELEON Introduction to Data Mining 08/06/ Experimental Results: CURE (10 clusters) Introduction to Data Mining 08/06/

28 Experimental Results: CURE (15 clusters) Introduction to Data Mining 08/06/ Experimental Results: CHAMELEON Introduction to Data Mining 08/06/

29 Experimental Results: CURE (9 clusters) Introduction to Data Mining 08/06/ Experimental Results: CURE (15 clusters) Introduction to Data Mining 08/06/

30 Graph-Based Clustering: SNN Approach Shared Nearest Neighbor (SNN) graph: the weight of an edge is the number of shared neighbors between vertices given that the vertices are connected i j i j 4 Introduction to Data Mining 08/06/ Creating the SNN Graph Sparse Graph Shared Near Neighbor Graph Link weights are similarities between neighboring points Link weights are number of Shared Nearest Neighbors Introduction to Data Mining 08/06/

31 Jarvis-Patrick Clustering First, the k-nearest neighbors of all points are found In graph terms this can be regarded as breaking all but the k strongest links from a point to other points in the proximity graph A pair of points is put in the same cluster if any two points share more than T neighbors and the two points are in each others k nearest neighbor list For instance, we might choose a nearest neighbor list of size 20 and put points in the same cluster if they share more than 10 near neighbors Jarvis-Patrick clustering is too brittle Introduction to Data Mining 08/06/ When Jarvis-Patrick Works Reasonably Well Original Points Jarvis Patrick Clustering 6 shared neighbors out of 20 Introduction to Data Mining 08/06/

32 When Jarvis-Patrick Does NOT Work Well Smallest threshold, T, that does not merge clusters. Threshold of T - 1 Introduction to Data Mining 08/06/ SNN Density-Based Clustering Combines: Graph based clustering (similarity definition based on number of shared nearest neighbors) Density based clustering (DBScan-like approach) SNN density measures whether a point is surrounded by similar points (with respect to its nearest neighbors) Introduction to Data Mining 08/06/

33 SNN Clustering Algorithm 1. Compute the similarity matrix This corresponds to a similarity graph with data points for nodes and edges whose weights are the similarities between data points 2. Sparsify the similarity il it matrix by keeping only the k most similar il neighbors This corresponds to only keeping the k strongest links of the similarity graph 3. Construct the shared nearest neighbor graph from the sparsified similarity matrix. At this point, we could apply a similarity threshold and find the connected components to obtain the clusters (Jarvis-Patrick algorithm) 4. Find the SNN density of each Point. Using a user specified parameters, Eps, find the number points that have an SNN similarity of Eps or greater to each point. This is the SNN density of the point Introduction to Data Mining 08/06/ SNN Clustering Algorithm 5. Find the core points Using a user specified parameter, MinPts, find the core points, i.e., all points that have an SNN density greater than MinPts 6. Form clusters from the core points If two core points are within a radius, Eps, of each other they are place in the same cluster 7. Discard all noise points All non-core points that are not within a radius of Eps of a core point are discarded 8. Assign all non-noise, non-core points to clusters This can be done by assigning such points to the nearest core point (Note that steps 4-8 are DBSCAN) Introduction to Data Mining 08/06/

34 SNN Density a) All Points b) High SNN Density c) Medium SNN Density d) Low SNN Density Introduction to Data Mining 08/06/ SNN Clustering Can Handle Differing Densities Original Points SNN Clustering Introduction to Data Mining 08/06/

35 SNN Clustering Can Handle Other Difficult Situations Introduction to Data Mining 08/06/ Finding Clusters of Time Series In Spatio-Temporal Data SLP Clusters via Shared Nearest Neighbor Clustering (100 NN, ) 90 SNN Density of SLP Time Series Data latitude longitude latitude longitude SNN Clusters of SLP. SNN Density of Points on the Globe. Introduction to Data Mining 08/06/

36 Limitations of SNN Clustering Does not cluster all the points Complexity of SNN Clustering is highh O( n * time to find numbers of neighbor within Eps) In worst case, this is O(n 2 ) For lower dimensions, there are more efficient ways to find the nearest neighbors R* Tree k-d dtrees Introduction to Data Mining 08/06/ Characteristics of Data, Clusters, and Clustering Algorithms A cluster analysis is affected by characteristics of Data Clusters Clustering algorithms Looking at these characteristics gives us a number of dimensions that you can use to describe clustering algorithms and the results that they produce Introduction to Data Mining 08/06/

37 Characteristics of Data High dimensionality Size of data set Sparsity of attribute values Noise and Outliers Types of attributes and type of data sets Differences in attribute scale Properties of the data space Can you define a meaningful centroid Introduction to Data Mining 08/06/ Characteristics of Clusters Data distribution Shape Differing sizes Differing densities Poor separation Relationship of clusters Subspace clusters Introduction to Data Mining 08/06/

38 Characteristics of Clustering Algorithms Order dependence Non-determinism Parameter selection Scalability Underlying model Optimization based approach Introduction to Data Mining 08/06/ Comparison of MIN and EM-Clustering We assume EM clustering using the Gaussian (normal) distribution. MIN is hierarchical, EM clustering is partitional. Both MIN and EM clustering are complete. MIN has a graph-based (contiguity-based) notion of a cluster, while EM clustering has a prototype (or model-based) notion of a cluster. MIN will not be able to distinguish poorly separated clusters, but EM can manage this in many situations. MIN can find clusters of different shapes and sizes; EM clustering prefers globular clusters and can have trouble with clusters of different sizes. Min has trouble with clusters of different densities, while EM can often handle this. Neither MIN nor EM clustering finds subspace clusters. Introduction to Data Mining 08/06/

39 Comparison of MIN and EM-Clustering MIN can handle outliers, but noise can join clusters; EM clustering can tolerate noise, but can be strongly affected by outliers. EM can only be applied to data for which a centroid is meaningful; MIN only requires a meaningful definition of proximity. EM will have trouble as dimensionality increases and the number of its parameters (the number of entries in the covariance matrix) increases as the square of the number of dimensions; MIN can work well with a suitable definition of proximity. EM is designed for Euclidean data, although versions of EM clustering have been developed for other types of data. MIN is shielded from the data type by the fact that it uses a similarity matrix. MIN makes not distribution assumptions; the version of EM we are considering assumes Gaussian distributions. Introduction to Data Mining 08/06/ Comparison of MIN and EM-Clustering EM has an O(n) time complexity; MIN is O(n 2 log(n)). Because of random initialization, the clusters found by EM can vary from one run to another; MIN produces the same clusters unless there are ties in the similarity matrix. Neither MIN nor EM automatically determine the number of clusters. MIN does not have any user-specified parameters; EM has the number of clusters and possibly the weights of the clusters. EM clustering can be viewed as an optimization problem; MIN uses a graph model of the data. Neither EM or MIN are order dependent. Introduction to Data Mining 08/06/

40 Comparison of DBSCAN and K-means Both are partitional. K-means is complete; DBSCAN is not. K-means has a prototype-based notion of a cluster; DB uses a density-based notion. K-means can find clusters that are not well-separated. DBSCAN will merge clusters that touch. DBSCAN handles clusters of different shapes and sizes; K-means prefers globular clusters. Introduction to Data Mining 08/06/ Comparison of DBSCAN and K-means DBSCAN can handle noise and outliers; K-means performs poorly in the presence of outliers K-means can only be applied to data for which a centroid is meaningful; DBSCAN requires a meaningful definition of density DBSCAN works poorly on high-dimensional data; K- means works well for some types of high-dimensional data Both techniques were designed for Euclidean data, but extended to other types of data DBSCAN makes not distribution assumptions; K-means is really assuming spherical Gaussian distributions Introduction to Data Mining 08/06/

41 Comparison of DBSCAN and K-means K-means has an O(n) time complexity; DBSCAN is O(n^2) Because of random initialization, the clusters found by K- means can vary from one run to another; DBSCAN always produces the same clusters DBSCAN automatically determines the number of cluster; K-means does not K-means has only one parameter, DBSCAN has two. K-means clustering can be viewed as an optimization problem and as a special case of EM clustering; DBSCAN is not based on a formal model. Introduction to Data Mining 08/06/

Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

Data Mining. Cluster Analysis: Advanced Concepts and Algorithms

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototype-based clustering Density-based clustering Graph-based

Clustering. Data Mining. Abraham Otero. Data Mining. Agenda

Clustering 1/46 Agenda Introduction Distance K-nearest neighbors Hierarchical clustering Quick reference 2/46 1 Introduction It seems logical that in a new situation we should act in a similar way as in

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDD-LAB ISTI- CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar

Clustering. Adrian Groza. Department of Computer Science Technical University of Cluj-Napoca

Clustering Adrian Groza Department of Computer Science Technical University of Cluj-Napoca Outline 1 Cluster Analysis What is Datamining? Cluster Analysis 2 K-means 3 Hierarchical Clustering What is Datamining?

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/2004 Hierarchical

Cluster Analysis: Basic Concepts and Algorithms

8 Cluster Analysis: Basic Concepts and Algorithms Cluster analysis divides data into groups (clusters) that are meaningful, useful, or both. If meaningful groups are the goal, then the clusters should

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 by Tan, Steinbach, Kumar 1 What is Cluster Analysis? Finding groups of objects such that the objects in a group will

Data Mining Clustering (2) Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining

Data Mining Clustering (2) Toon Calders Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining Outline Partitional Clustering Distance-based K-means, K-medoids,

An Introduction to Cluster Analysis for Data Mining

An Introduction to Cluster Analysis for Data Mining 10/02/2000 11:42 AM 1. INTRODUCTION... 4 1.1. Scope of This Paper... 4 1.2. What Cluster Analysis Is... 4 1.3. What Cluster Analysis Is Not... 5 2. OVERVIEW...

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining /8/ What is Cluster

Clustering & Association

Clustering - Overview What is cluster analysis? Grouping data objects based only on information found in the data describing these objects and their relationships Maximize the similarity within objects

Unsupervised Data Mining (Clustering)

Unsupervised Data Mining (Clustering) Javier Béjar KEMLG December 01 Javier Béjar (KEMLG) Unsupervised Data Mining (Clustering) December 01 1 / 51 Introduction Clustering in KDD One of the main tasks in

An Analysis on Density Based Clustering of Multi Dimensional Spatial Data

An Analysis on Density Based Clustering of Multi Dimensional Spatial Data K. Mumtaz 1 Assistant Professor, Department of MCA Vivekanandha Institute of Information and Management Studies, Tiruchengode,

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Clustering Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Clustering Algorithms K-means and its variants Hierarchical clustering

Clustering methods for Big data analysis

Clustering methods for Big data analysis Keshav Sanse, Meena Sharma Abstract Today s age is the age of data. Nowadays the data is being produced at a tremendous rate. In order to make use of this large-scale

Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 What is

2 Basic Concepts and Techniques of Cluster Analysis

The Challenges of Clustering High Dimensional Data * Michael Steinbach, Levent Ertöz, and Vipin Kumar Abstract Cluster analysis divides data into groups (clusters) for the purposes of summarization or

Chapter 7. Cluster Analysis

Chapter 7. Cluster Analysis. What is Cluster Analysis?. A Categorization of Major Clustering Methods. Partitioning Methods. Hierarchical Methods 5. Density-Based Methods 6. Grid-Based Methods 7. Model-Based

CLASSIFICATION AND CLUSTERING. Anveshi Charuvaka

CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training

Cluster Analysis. Alison Merikangas Data Analysis Seminar 18 November 2009

Cluster Analysis Alison Merikangas Data Analysis Seminar 18 November 2009 Overview What is cluster analysis? Types of cluster Distance functions Clustering methods Agglomerative K-means Density-based Interpretation

Clustering UE 141 Spring 2013

Clustering UE 141 Spring 013 Jing Gao SUNY Buffalo 1 Definition of Clustering Finding groups of obects such that the obects in a group will be similar (or related) to one another and different from (or

On Clustering Validation Techniques

Journal of Intelligent Information Systems, 17:2/3, 107 145, 2001 c 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques MARIA HALKIDI mhalk@aueb.gr YANNIS

A Novel Density based improved k-means Clustering Algorithm Dbkmeans

A Novel Density based improved k-means Clustering Algorithm Dbkmeans K. Mumtaz 1 and Dr. K. Duraiswamy 2, 1 Vivekanandha Institute of Information and Management Studies, Tiruchengode, India 2 KS Rangasamy

Data Clustering Techniques Qualifying Oral Examination Paper

Data Clustering Techniques Qualifying Oral Examination Paper Periklis Andritsos University of Toronto Department of Computer Science periklis@cs.toronto.edu March 11, 2002 1 Introduction During a cholera

Clustering Techniques: A Brief Survey of Different Clustering Algorithms

Clustering Techniques: A Brief Survey of Different Clustering Algorithms Deepti Sisodia Technocrates Institute of Technology, Bhopal, India Lokesh Singh Technocrates Institute of Technology, Bhopal, India

. Learn the number of classes and the structure of each class using similarity between unlabeled training patterns

Outline Part 1: of data clustering Non-Supervised Learning and Clustering : Problem formulation cluster analysis : Taxonomies of Clustering Techniques : Data types and Proximity Measures : Difficulties

K-Means Cluster Analysis. Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1

K-Means Cluster Analsis Chapter 3 PPDM Class Tan,Steinbach, Kumar Introduction to Data Mining 4/18/4 1 What is Cluster Analsis? Finding groups of objects such that the objects in a group will be similar

GraphZip: A Fast and Automatic Compression Method for Spatial Data Clustering

GraphZip: A Fast and Automatic Compression Method for Spatial Data Clustering Yu Qian Kang Zhang Department of Computer Science, The University of Texas at Dallas, Richardson, TX 75083-0688, USA {yxq012100,

Clustering & Visualization

Chapter 5 Clustering & Visualization Clustering in high-dimensional databases is an important problem and there are a number of different clustering paradigms which are applicable to high-dimensional data.

A comparison of various clustering methods and algorithms in data mining

Volume :2, Issue :5, 32-36 May 2015 www.allsubjectjournal.com e-issn: 2349-4182 p-issn: 2349-5979 Impact Factor: 3.762 R.Tamilselvi B.Sivasakthi R.Kavitha Assistant Professor A comparison of various clustering

A Method for Decentralized Clustering in Large Multi-Agent Systems

A Method for Decentralized Clustering in Large Multi-Agent Systems Elth Ogston, Benno Overeinder, Maarten van Steen, and Frances Brazier Department of Computer Science, Vrije Universiteit Amsterdam {elth,bjo,steen,frances}@cs.vu.nl

Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016

Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with

Clustering: Techniques & Applications. Nguyen Sinh Hoa, Nguyen Hung Son. 15 lutego 2006 Clustering 1

Clustering: Techniques & Applications Nguyen Sinh Hoa, Nguyen Hung Son 15 lutego 2006 Clustering 1 Agenda Introduction Clustering Methods Applications: Outlier Analysis Gene clustering Summary and Conclusions

Data Mining Clustering. Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining

Data Mining Clustering Toon Calders Sheets are based on the those provided b Tan, Steinbach, and Kumar. Introduction to Data Mining What is Cluster Analsis? Finding groups of objects such that the objects

Concept of Cluster Analysis

RESEARCH PAPER ON CLUSTER TECHNIQUES OF DATA VARIATIONS Er. Arpit Gupta 1,Er.Ankit Gupta 2,Er. Amit Mishra 3 arpit_jp@yahoo.co.in, ank_mgcgv@yahoo.co.in,amitmishra.mtech@gmail.com Faculty Of Engineering

Cluster Analysis: Basic Concepts and Algorithms

Cluster Analsis: Basic Concepts and Algorithms What does it mean clustering? Applications Tpes of clustering K-means Intuition Algorithm Choosing initial centroids Bisecting K-means Post-processing Strengths

Original Article Survey of Recent Clustering Techniques in Data Mining

International Archive of Applied Sciences and Technology Volume 3 [2] June 2012: 68-75 ISSN: 0976-4828 Society of Education, India Website: www.soeagra.com/iaast/iaast.htm Original Article Survey of Recent

Cluster Analysis: Basic Concepts and Methods

10 Cluster Analysis: Basic Concepts and Methods Imagine that you are the Director of Customer Relationships at AllElectronics, and you have five managers working for you. You would like to organize all

Example: Document Clustering. Clustering: Definition. Notion of a Cluster can be Ambiguous. Types of Clusterings. Hierarchical Clustering

Overview Prognostic Models and Data Mining in Medicine, part I Cluster Analsis What is Cluster Analsis? K-Means Clustering Hierarchical Clustering Cluster Validit Eample: Microarra data analsis 6 Summar

Clustering Hierarchical clustering and k-mean clustering

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein The clustering problem: A quick review partition genes into distinct sets with high homogeneity

Territorial Analysis for Ratemaking. Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs

Territorial Analysis for Ratemaking by Philip Begher, Dario Biasini, Filip Branitchev, David Graham, Erik McCracken, Rachel Rogers and Alex Takacs Department of Statistics and Applied Probability University

Outlier Detection in Clustering

Outlier Detection in Clustering Svetlana Cherednichenko 24.01.2005 University of Joensuu Department of Computer Science Master s Thesis TABLE OF CONTENTS 1. INTRODUCTION...1 1.1. BASIC DEFINITIONS... 1

Lecture 20: Clustering

Lecture 20: Clustering Wrap-up of neural nets (from last lecture Introduction to unsupervised learning K-means clustering COMP-424, Lecture 20 - April 3, 2013 1 Unsupervised learning In supervised learning,

A Comparative Analysis of Various Clustering Techniques used for Very Large Datasets

A Comparative Analysis of Various Clustering Techniques used for Very Large Datasets Preeti Baser, Assistant Professor, SJPIBMCA, Gandhinagar, Gujarat, India 382 007 Research Scholar, R. K. University,

The Role of Visualization in Effective Data Cleaning

The Role of Visualization in Effective Data Cleaning Yu Qian Dept. of Computer Science The University of Texas at Dallas Richardson, TX 75083-0688, USA qianyu@student.utdallas.edu Kang Zhang Dept. of Computer

Data Mining: Concepts and Techniques. Jiawei Han. Micheline Kamber. Simon Fräser University К MORGAN KAUFMANN PUBLISHERS. AN IMPRINT OF Elsevier

Data Mining: Concepts and Techniques Jiawei Han Micheline Kamber Simon Fräser University К MORGAN KAUFMANN PUBLISHERS AN IMPRINT OF Elsevier Contents Foreword Preface xix vii Chapter I Introduction I I.

An Enhanced Clustering Algorithm to Analyze Spatial Data

International Journal of Engineering and Technical Research (IJETR) ISSN: 2321-0869, Volume-2, Issue-7, July 2014 An Enhanced Clustering Algorithm to Analyze Spatial Data Dr. Mahesh Kumar, Mr. Sachin Yadav

Clustering in Ratemaking: Applications in Territories Clustering

Clustering in Ratemaking: Applications in Territories Clustering Ji Yao, Ph.D. Abstract: Clustering methods are briefly reviewed and their applications in insurance ratemaking are discussed in this paper.

Public Transportation BigData Clustering

Public Transportation BigData Clustering Preliminary Communication Tomislav Galba J.J. Strossmayer University of Osijek Faculty of Electrical Engineering Cara Hadriana 10b, 31000 Osijek, Croatia tomislav.galba@etfos.hr

Distance based clustering

// Distance based clustering Chapter ² ² Clustering Clustering is the art of finding groups in data (Kaufman and Rousseeuw, 99). What is a cluster? Group of objects separated from other clusters Means

Recent Advances in Clustering: A Brief Survey

Recent Advances in Clustering: A Brief Survey S.B. KOTSIANTIS, P. E. PINTELAS Department of Mathematics University of Patras Educational Software Development Laboratory Hellas {sotos, pintelas}@math.upatras.gr

Authors. Data Clustering: Algorithms and Applications

Authors Data Clustering: Algorithms and Applications 2 Contents 1 Grid-based Clustering 1 Wei Cheng, Wei Wang, and Sandra Batista 1.1 Introduction................................... 1 1.2 The Classical

On Data Clustering Analysis: Scalability, Constraints and Validation

On Data Clustering Analysis: Scalability, Constraints and Validation Osmar R. Zaïane, Andrew Foss, Chi-Hoon Lee, and Weinan Wang University of Alberta, Edmonton, Alberta, Canada Summary. Clustering is

Cluster Analysis Overview. Data Mining Techniques: Cluster Analysis. What is Cluster Analysis? What is Cluster Analysis?

Cluster Analsis Overview Data Mining Techniques: Cluster Analsis Mirek Riedewald Man slides based on presentations b Han/Kamber, Tan/Steinbach/Kumar, and Andrew Moore Introduction Foundations: Measuring

Data Mining Process Using Clustering: A Survey

Data Mining Process Using Clustering: A Survey Mohamad Saraee Department of Electrical and Computer Engineering Isfahan University of Techno1ogy, Isfahan, 84156-83111 saraee@cc.iut.ac.ir Najmeh Ahmadian

Data Mining Project Report. Document Clustering. Meryem Uzun-Per

Data Mining Project Report Document Clustering Meryem Uzun-Per 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. K-means algorithm...

Clustering and Outlier Detection

Clustering and Outlier Detection Application Examples Customer segmentation How to partition customers into groups so that customers in each group are similar, while customers in different groups are dissimilar?

Fig. 1 A typical Knowledge Discovery process [2]

Volume 4, Issue 7, July 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Review on Clustering

L15: statistical clustering

Similarity measures Criterion functions Cluster validity Flat clustering algorithms k-means ISODATA L15: statistical clustering Hierarchical clustering algorithms Divisive Agglomerative CSCE 666 Pattern

BIRCH: An Efficient Data Clustering Method For Very Large Databases

BIRCH: An Efficient Data Clustering Method For Very Large Databases Tian Zhang, Raghu Ramakrishnan, Miron Livny CPSC 504 Presenter: Discussion Leader: Sophia (Xueyao) Liang HelenJr, Birches. Online Image.

Data Mining K-Clustering Problem

Data Mining K-Clustering Problem Elham Karoussi Supervisor Associate Professor Noureddine Bouhmala This Master s Thesis is carried out as a part of the education at the University of Agder and is therefore

Data visualization and clustering. Genomics is to no small extend a data science

Data visualization and clustering Genomics is to no small extend a data science [www.data2discovery.org] Data visualization and clustering Genomics is to no small extend a data science [Andersson et al.,

Clustering. Chapter 7. 7.1 Introduction to Clustering Techniques. 7.1.1 Points, Spaces, and Distances

240 Chapter 7 Clustering Clustering is the process of examining a collection of points, and grouping the points into clusters according to some distance measure. The goal is that points in the same cluster

There are a number of different methods that can be used to carry out a cluster analysis; these methods can be classified as follows:

Statistics: Rosie Cornish. 2007. 3.1 Cluster Analysis 1 Introduction This handout is designed to provide only a brief introduction to cluster analysis and how it is done. Books giving further details are

Going Big in Data Dimensionality:

LUDWIG- MAXIMILIANS- UNIVERSITY MUNICH DEPARTMENT INSTITUTE FOR INFORMATICS DATABASE Going Big in Data Dimensionality: Challenges and Solutions for Mining High Dimensional Data Peer Kröger Lehrstuhl für

A Study on the Hierarchical Data Clustering Algorithm Based on Gravity Theory

A Study on the Hierarchical Data Clustering Algorithm Based on Gravity Theory Yen-Jen Oyang, Chien-Yu Chen, and Tsui-Wei Yang Department of Computer Science and Information Engineering National Taiwan

Cluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico

Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from

10-810 /02-710 Computational Genomics. Clustering expression data

10-810 /02-710 Computational Genomics Clustering expression data What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally,

SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING

AAS 07-228 SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING INTRODUCTION James G. Miller * Two historical uncorrelated track (UCT) processing approaches have been employed using general perturbations

Information Retrieval and Web Search Engines

Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 10 th, 2013 Wolf-Tilo Balke and Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig

Robust Outlier Detection Technique in Data Mining: A Univariate Approach

Robust Outlier Detection Technique in Data Mining: A Univariate Approach Singh Vijendra and Pathak Shivani Faculty of Engineering and Technology Mody Institute of Technology and Science Lakshmangarh, Sikar,

PROBABILISTIC DISTANCE CLUSTERING

PROBABILISTIC DISTANCE CLUSTERING BY CEM IYIGUN A dissertation submitted to the Graduate School New Brunswick Rutgers, The State University of New Jersey in partial fulfillment of the requirements for

Neural Networks Lesson 5 - Cluster Analysis

Neural Networks Lesson 5 - Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt. - Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29

OUTLIER ANALYSIS. Data Mining 1

OUTLIER ANALYSIS Data Mining 1 What Are Outliers? Outlier: A data object that deviates significantly from the normal objects as if it were generated by a different mechanism Ex.: Unusual credit card purchase,

Unsupervised Learning and Data Mining. Unsupervised Learning and Data Mining. Clustering. Supervised Learning. Supervised Learning

Unsupervised Learning and Data Mining Unsupervised Learning and Data Mining Clustering Decision trees Artificial neural nets K-nearest neighbor Support vectors Linear regression Logistic regression...

Clustering. 15-381 Artificial Intelligence Henry Lin. Organizing data into clusters such that there is

Clustering 15-381 Artificial Intelligence Henry Lin Modified from excellent slides of Eamonn Keogh, Ziv Bar-Joseph, and Andrew Moore What is Clustering? Organizing data into clusters such that there is

UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS

UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS Dwijesh C. Mishra I.A.S.R.I., Library Avenue, New Delhi-110 012 dcmishra@iasri.res.in What is Learning? "Learning denotes changes in a system that enable

Gene Expression Data Clustering Analysis: A Survey

Gene Expression Data Clustering Analysis: A Survey Sajid Nagi #, Dhruba K. Bhattacharyya *, Jugal K. Kalita # Department of Computer Science, St. Edmund s College, Shillong 793001, Meghalaya, India. *

Forschungskolleg Data Analytics Methods and Techniques

Forschungskolleg Data Analytics Methods and Techniques Martin Hahmann, Gunnar Schröder, Phillip Grosse Prof. Dr.-Ing. Wolfgang Lehner Why do we need it? We are drowning in data, but starving for knowledge!

A Survey of Kernel Clustering Methods

A Survey of Kernel Clustering Methods Maurizio Filippone, Francesco Camastra, Francesco Masulli and Stefano Rovetta Presented by: Kedar Grama Outline Unsupervised Learning and Clustering Types of clustering

Data Exploration Data Visualization

Data Exploration Data Visualization What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping to select

Comparison and Analysis of Various Clustering Methods in Data mining On Education data set Using the weak tool

Comparison and Analysis of Various Clustering Metho in Data mining On Education data set Using the weak tool Abstract:- Data mining is used to find the hidden information pattern and relationship between

PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA

PERFORMANCE ANALYSIS OF CLUSTERING ALGORITHMS IN DATA MINING IN WEKA Prakash Singh 1, Aarohi Surya 2 1 Department of Finance, IIM Lucknow, Lucknow, India 2 Department of Computer Science, LNMIIT, Jaipur,

Unsupervised learning: Clustering

Unsupervised learning: Clustering Salissou Moutari Centre for Statistical Science and Operational Research CenSSOR 17 th September 2013 Unsupervised learning: Clustering 1/52 Outline 1 Introduction What

Chapter ML:XI (continued)

Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained

Comparing large datasets structures through unsupervised learning

Comparing large datasets structures through unsupervised learning Guénaël Cabanes and Younès Bennani LIPN-CNRS, UMR 7030, Université de Paris 13 99, Avenue J-B. Clément, 93430 Villetaneuse, France cabanes@lipn.univ-paris13.fr

A Survey of Clustering Algorithms for Big Data: Taxonomy & Empirical Analysis

TRANSACTIONS ON EMERGING TOPICS IN COMPUTING, 2014 1 A Survey of Clustering Algorithms for Big Data: Taxonomy & Empirical Analysis A. Fahad, N. Alshatri, Z. Tari, Member, IEEE, A. Alamri, I. Khalil A.

Data Clustering Using Data Mining Techniques

Data Clustering Using Data Mining Techniques S.R.Pande 1, Ms. S.S.Sambare 2, V.M.Thakre 3 Department of Computer Science, SSES Amti's Science College, Congressnagar, Nagpur, India 1 Department of Computer

Distances, Clustering, and Classification. Heatmaps

Distances, Clustering, and Classification Heatmaps 1 Distance Clustering organizes things that are close into groups What does it mean for two genes to be close? What does it mean for two samples to be

Classifying Large Data Sets Using SVMs with Hierarchical Clusters. Presented by :Limou Wang

Classifying Large Data Sets Using SVMs with Hierarchical Clusters Presented by :Limou Wang Overview SVM Overview Motivation Hierarchical micro-clustering algorithm Clustering-Based SVM (CB-SVM) Experimental

Clustering and Data Mining in R

Clustering and Data Mining in R Workshop Supplement Thomas Girke December 10, 2011 Introduction Data Preprocessing Data Transformations Distance Methods Cluster Linkage Hierarchical Clustering Approaches

Outlier Ensembles. Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598. Keynote, Outlier Detection and Description Workshop, 2013

Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598 Outlier Ensembles Keynote, Outlier Detection and Description Workshop, 2013 Based on the ACM SIGKDD Explorations Position Paper: Outlier

Machine Learning and Data Analysis overview. Department of Cybernetics, Czech Technical University in Prague. http://ida.felk.cvut.

Machine Learning and Data Analysis overview Jiří Kléma Department of Cybernetics, Czech Technical University in Prague http://ida.felk.cvut.cz psyllabus Lecture Lecturer Content 1. J. Kléma Introduction,

Data Mining: Exploring Data. Lecture Notes for Chapter 3. Slides by Tan, Steinbach, Kumar adapted by Michael Hahsler

Data Mining: Exploring Data Lecture Notes for Chapter 3 Slides by Tan, Steinbach, Kumar adapted by Michael Hahsler Topics Exploratory Data Analysis Summary Statistics Visualization What is data exploration?

Data Mining and Knowledge Discovery in Databases (KDD) State of the Art. Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland

Data Mining and Knowledge Discovery in Databases (KDD) State of the Art Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland 1 Conference overview 1. Overview of KDD and data mining 2. Data