Descriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications

Size: px

Start display at page:

Download "Descriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications"

Roy Hunt
8 years ago
Views:

1 CMSC828G Prncples of Data Mnng Lecture #9 Today s Readng: HMS, chapter 9 Today s Lecture: Descrptve Modelng Clusterng Algorthms Descrptve Models model presents the man features of the data, a global summary of the data cluster analyss densty estmaton Upcomng: hw2 avalable on web page ths evenng proect proposals due 3/12 Cluster Analyss decomposng or parttonng a data set nto groups so that the ponts n one group are smlar to each other and are as dfferent as possble from the ponts n other groups General Applcatons of Clusterng Pattern Recognton Spatal Data Analyss create thematc maps n GIS by clusterng feature spaces detect spatal clusters and explan them n spatal data mnng Image Processng Economc Scence (especally maret research) WWW Document classfcaton Cluster Weblog data to dscover groups of smlar access patterns Ths sn t really clusterng, t s ust bnnng the obects Examples of Clusterng Applcatons Maretng: Help mareters dscover dstnct groups n ther customer bases, and then use ths nowledge to develop targeted maretng programs Land use: Identfcaton of areas of smlar land use n an earth observaton database Insurance: Identfyng groups of motor nsurance polcy holders wth a hgh average clam cost Cty-plannng: Identfyng groups of houses accordng to ther house type, value, and geographcal locaton Earth-quae studes: Observed earth quae epcenters should be clustered along contnent faults Example households: locaton, ncome, number of chldren, rent/own, crme rate, number of cars The approprate clusterng wll depend on goals: mnmze delvery tme cluster by locaton others?

ponts n one group are smlar to each other and are as dfferent as possble from the ponts n other groups General Applcatons of Clusterng Pattern Recognton Spatal Data Analyss create thematc maps n GIS

2 Clusterng decomposng or parttonng a data set nto groups so that the ponts n one group are smlar to each other and are as dfferent as possble from the ponts n other groups Measure of dstance s fundamental Explct representaton: D(x(),x()) for each x only feasble for small domans Measurement: dstance computed from features we saw a number of dfferent ways of dong ths n ch. 2 Clusterng Huge body of wor (aa unsupervsed learnng, segmentaton, ) One of the maor dffcultes s n evaluatng the success of a method valdty depends on goals f goal s to fnd nterestng clusters, ths s rather dffcult to quantfy however, for our probablstc methods, we wll present some tools for valdatng our models Choosng an Algorthm As we wll see, dfferent algorthms wll result n clusters of dfferent shapes The approprate shape wll depend on the applcaton and should be consder when choosng an algorthm match method to obectves Famles of Clusterng Algorthms Partton-based methods e.g., K-means Herarchcal clusterng e.g., herarchcal agglomeratve clusterng Probablstc model-based clusterng e.g., mxture models Partton-based Clusterng Algorthms Gven set of n data ponts D={x(1),, x(n)} partton data nto clusters C = {C 1,, C } such that each x() s assgned to a unque C and Score(C,D) s mnmzed/maxmzed combnatoral optmzaton: searchng for allocaton of n obects nto classes that maxmzes score functon Number of possble allocatons n exhaustve typcally fndng the optmal soluton s ntractable Resort to teratve mprovement Score Functon Score functon: clusters compact mnmze wthn cluster dstance, wc(c) clusters should be far apart maxmze dstance between clusters, bc(c) Gven a clusterng C, assgn cluster centers, c f ponts belong to space where means mae sense, we can use the centrod of the ponts n the cluster: 1 c = x n x C wc(c) = sum-of-squares wthn cluster dstance wc ( C ) = K = 1 wc ( C ) = K = 1 x bc(c) = dstance between clusters bc ( C ) = 1 < d ( c K C d ( x,c 2, c ) ) Score(C,D) = f(wc(c), bc(c))

2 Clusterng Huge body of wor (aa unsupervsed learnng, segmentaton, ) One of the maor dffcultes s n evaluatng the success of a method valdty depends on goals f goal s to fnd nterestng clusters, ths s

3 K-means Idea: Start wth randomly chosen cluster centers Assgn ponts to gve greatest ncrease n score Recompute cluster centers Reassgn ponts Repeat untl no changes

4 #2 #2 #2 Demos Complexty -means applet another demo mage example Does algorthm termnate? Does algorthm converge to optmal soluton? Tme complexty one teraton? n

5 Algorthm Varatons recompute centrod as soon as a pont s reassgned allow merge and splt of clusters methods for mprovng soluton accuracy? n cases where means do not mae sense -medods use one of the data ponts as center categorcal data - what f data set s too large for algorthm to be tractable? compress data by replacng groups of obects by condensed representaton Bnary Varables A contngency table for bnary data Obect 1 0 sum 1 a b a+ b Obect 0 c d c+ d sum a+ c b+ d p Smple matchng coeffcent (nvarant, f the bnary varable s symmetrc): d (, ) = b + c a + b + c + d Jaccard coeffcent (nonnvarant f the bnary varable s asymmetrc): d (, ) = b c a + + b + c Dssmlarty between Bnary Varables Nomnal Varables Example Name Fever Cough Test-1 Test-2 Test-3 Test-4 Jac Y N P N N N Mary Y N P N P N Jm Y P N N N N attrbutes are asymmetrc bnary let the values Y and P be set to 1, and the value N be set to d ( ac, mary ) = = d ( ac, m ) = = d ( m, mary ) = = A generalzaton of the bnary varable n that t can tae more than 2 states, e.g., red, yellow, blue, green Method 1: Smple matchng m: # of matches, p: total # of varables d (, ) = p p m Method 2: use a large number of bnary varables creatng a new bnary varable for each of the M nomnal states Herarchcal Clusterng Dendogram rather than decdng the number of clusters K at the start, buld a herarchy of nested clusters ether gradually merge ponts (agglomeratve) dvde superclusters (dvsve) result of ether approach can be shown as a dendogram whch depcts the sequence of merges or splts

compress data by replacng groups of obects by condensed representaton Bnary Varables A contngency table for bnary data Obect 1 0 sum 1 a b a+ b Obect 0 c d c+ d sum a+ c b+ d p Smple matchng

6 tme complexty? space complexty? Agglomeratve Methods based on measures of dstance between clusters for = 1 to n let C = {x()} whle there s more than one cluster left do let C and C be the par of clusters wth mnmum D(C, C ) C = C C remove C end Measurng Dstances between Clusters sngle ln/nearest neghbor method: D(C,C ) = mn{d(x, y) x C, y C } complete ln/furthest neghbor method: D(C,C ) = max{d(x, y) x C, y C } average ln: D(C,C ) = avg{d(x, y) x C, y C } centrod measure: D(C,C) = d(c,c ) where c and c are centrods Ward s measure: dfference between total wthn cluster sum of squares for the two clusters separately and the sum of squares error n the merged cluster Dvsve Methods Begn wth a sngle cluster, consstng of all the data ponts splt nto components ultmately ends wth a partton n whch each cluster has a sngle pont monolthc methods splt cluster usng one varable at a tme polythetc methods mae splts based on all of the varables together; dffculty comes n how to choose potental splts n general, dvsve methods are less wdely used than agglomeratve methods Demos ClusterCalc Readng: HMS, chapter 9 cont. Next Tme References Prncples of Data Mnng, Hand, Mannla, Smyth. MIT Press, Data Mnng, Jawe Han and Mchelne Kamber. Morgan Kaufmann, sldes:

remove C end Measurng Dstances between Clusters sngle ln/nearest neghbor method: D(C,C ) = mn{d(x, y) x C, y C } complete ln/furthest neghbor method: D(C,C ) = max{d(x, y) x C, y C } average ln:

Cluster Analysis. Cluster Analysis

Cluster Analysis. Cluster Analysis Cluster Analyss Cluster Analyss What s Cluster Analyss? Types of Data n Cluster Analyss A Categorzaton of Maor Clusterng Methos Parttonng Methos Herarchcal Methos Densty-Base Methos Gr-Base Methos Moel-Base