B490 Mining the Big Data. 2 Clustering


 Gwenda Horton
 1 years ago
 Views:
Transcription
1 B490 Mining the Big Data 2 Clustering Qin Zhang 11
2 Motivations Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Cluster topic/articles from the web by same story. Google image Many others
3 Motivations 22 Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Cluster topic/articles from the web by same story. Google image Many others... Given a cloud of data points we want to understand their structures Community discovery in social networks
4 Motivations 23 Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Cluster topic/articles from the web by same story. Google image Many others... Given a cloud of data points we want to understand their structures Community discovery in social networks
5 The plan Clustering is an extremely broad and illdefined topic. There are many many variants; we will not try to cover all of them. Instead we will focus on three broad classes of algorithms Hierachical Clustering Assignmentbased Clustering Spectural Clustering 31
6 The plan Clustering is an extremely broad and illdefined topic. There are many many variants; we will not try to cover all of them. Instead we will focus on three broad classes of algorithms Hierachical Clustering Assignmentbased Clustering Spectural Clustering There is a saying When data is easily clusterable, most clustering algorithms work quickly and well. When is not easily clusterable, then no algorithm will find good clusters. 32
7 41 Hierachical Clustering
8 Hard clustering Problem: Start with a set X R d, and a metric d : R d R d R +. A cluster C is a subset of X. A clustering is a partition ρ(x ) = {C 1, C 2,..., C k }, where 1. Each C i X. 2. Each pair C i C j =. 3. Each k i=1 C i = X. 51
9 Hard clustering Problem: Start with a set X R d, and a metric d : R d R d R +. A cluster C is a subset of X. A clustering is a partition ρ(x ) = {C 1, C 2,..., C k }, where 1. Each C i X. 2. Each pair C i C j =. 3. Each k i=1 C i = X. Goal: 1. (close intra) For each C ρ(x ) for all x, x C, d(x, x ) is small. 2. (far inter) For each C i, C j ρ(x ) (i j), for most x i C i and x j C j, d(x i, x j ) is large. 52
10 Hierarchical clustering Hierarchical Clustering 1. Each x i X forms a cluster C i 2. While exists two clusters close enough (a) Find the closest two clusters C i, C j. (b) Merge C i, C j into a single cluster 61
11 Hierarchical clustering Hierarchical Clustering 1. Each x i X forms a cluster C i 2. While exists two clusters close enough (a) Find the closest two clusters C i, C j. (b) Merge C i, C j into a single cluster Two things to define 1. Define a distance between clusters. 2. Define close enough 62
12 Definition of distance between clusters Distance between center of clusters. What does center mean? 71
13 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. 72
14 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) 73
15 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) 74
16 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point 75
17 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points 76
18 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points Distance between furthest pair of points 77
19 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points Distance between furthest pair of points Average distance between all pairs of points in different clusters 78
20 Definition of distance between clusters 79 Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points Distance between furthest pair of points Average distance between all pairs of points in different clusters Radius of minimum enclosing ball of joined cluster
21 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. 81
22 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. 82
23 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d 83
24 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d If the joint density decrease too much from single cluster density. Variations of this are called the elbow technique. 84
25 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d If the joint density decrease too much from single cluster density. Variations of this are called the elbow technique. When the number of clusters is k (for some magical value k). 85
26 Definition of close enough 86 If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d If the joint density decrease too much from single cluster density. Variations of this are called the elbow technique. When the number of clusters is k (for some magical value k). Example (on board): Distance is distance between centroids. Stop when there is 1 cluster left.
27 Assignmentbased Clustering (kcenter, kmean, kmedian) 91
28 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). 101
29 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). The kcenter cluster problem is to find the set C of k clusters (often, but not always as a subset of X ) to minimize max x X d(φ C (x), x). 102
30 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). The kcenter cluster problem is to find the set C of k clusters (often, but not always as a subset of X ) to minimize max x X d(φ C (x), x). The kmeans cluster problem: minimize x X d(φ C (x), x)
31 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). The kcenter cluster problem is to find the set C of k clusters (often, but not always as a subset of X ) to minimize max x X d(φ C (x), x). The kmeans cluster problem: minimize x X d(φ C (x), x) 2 The kmedian cluster problem: minimize x X d(φ C (x), x) 104
32 Gonzalez Algorithm Gonzalez Algorithm A simple greedy algorithm for kcenter 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Set c i = arg max x X d(x, φ Si 1 (x)) (b) Let S i = {c 1,..., c i } 111
33 Gonzalez Algorithm Gonzalez Algorithm A simple greedy algorithm for kcenter 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Set c i = arg max x X d(x, φ Si 1 (x)) (b) Let S i = {c 1,..., c i } In the worst case, it gives a 2approximation to the optimal solution. 112
34 Gonzalez Algorithm Gonzalez Algorithm A simple greedy algorithm for kcenter 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Set c i = arg max x X d(x, φ Si 1 (x)) (b) Let S i = {c 1,..., c i } In the worst case, it gives a 2approximation to the optimal solution. Analysis (on board) 113
35 Parallel Guessing Algorithm Parallel Guessing Algorithm for kcenter Run for R = (1 + ɛ/2), (1 + ɛ/2) 2,..., 1. We pick an arbitrary point as the first center, C = {c 1 }. 2. For each point p P compute r p = min c C d(p, c) and test whether r p > R. If it is, then set C = C {p}. 3. Abort at C > k Pick the minimum R that does not abort. 121
36 Parallel Guessing Algorithm Parallel Guessing Algorithm for kcenter Run for R = (1 + ɛ/2), (1 + ɛ/2) 2,..., 1. We pick an arbitrary point as the first center, C = {c 1 }. 2. For each point p P compute r p = min c C d(p, c) and test whether r p > R. If it is, then set C = C {p}. 3. Abort at C > k Pick the minimum R that does not abort. In the worst case, it gives a (2 + ɛ)approximation to the optimal solution (on board). 122
37 Parallel Guessing Algorithm Parallel Guessing Algorithm for kcenter Run for R = (1 + ɛ/2), (1 + ɛ/2) 2,..., 1. We pick an arbitrary point as the first center, C = {c 1 }. 2. For each point p P compute r p = min c C d(p, c) and test whether r p > R. If it is, then set C = C {p}. 3. Abort at C > k Pick the minimum R that does not abort. In the worst case, it gives a (2 + ɛ)approximation to the optimal solution (on board). Can be implemented as a streaming algorithm performs a linear scan of the data, uses space Õ(k) and time Õ(nk). 123
38 Lloyd s Algorithm Lloyd s Algorithm The wellknown algorithm for kmeans 1. Choose k points C X (arbitrarily?) 2. Repeat (a) For all x X, find φ C (x) (closest center c C to x) (b) For all i [k], let c i = average{x X φ C (x) = c i } 3. until the set C is unchanged Kmeans_Kmedoids.html 131
39 Lloyd s Algorithm Lloyd s Algorithm The wellknown algorithm for kmeans 1. Choose k points C X (arbitrarily?) 2. Repeat (a) For all x X, find φ C (x) (closest center c C to x) (b) For all i [k], let c i = average{x X φ C (x) = c i } 3. until the set C is unchanged Kmeans_Kmedoids.html If the main loop has R rounds, then the running time is O(Rnk) 132
40 Lloyd s Algorithm 133 Lloyd s Algorithm The wellknown algorithm for kmeans 1. Choose k points C X (arbitrarily?) 2. Repeat (a) For all x X, find φ C (x) (closest center c C to x) (b) For all i [k], let c i = average{x X φ C (x) = c i } 3. until the set C is unchanged Kmeans_Kmedoids.html If the main loop has R rounds, then the running time is O(Rnk) But 1. What is R? Converge in finite number of steps? 2. How do we initialize C? 3. How accurate is this algorithm?
41 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) 141
42 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. 142
43 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. Smoothed analysis: if data perturbed randomly slightly, then R = O(n 35 k 34 d 8 ). This is polynomial, but still ridiculous. 143
44 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. Smoothed analysis: if data perturbed randomly slightly, then R = O(n 35 k 34 d 8 ). This is polynomial, but still ridiculous. If all points are on a grid of length M, then R = O(dn 4 M 2 ). But thats still way too big. 144
45 Value of R 145 It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. Smoothed analysis: if data perturbed randomly slightly, then R = O(n 35 k 34 d 8 ). This is polynomial, but still ridiculous. If all points are on a grid of length M, then R = O(dn 4 M 2 ). But thats still way too big. Conclusion: There are crazy special cases that can take a long time, but usually it works.
46 Initialize C Random set of points. By coupon collectors, we know that we need about O(k log k) to get one in each cluster, given that each cluster contains equal number of points. We can later reduce to k clusters, by merging extra clusters. 151
47 Initialize C Random set of points. By coupon collectors, we know that we need about O(k log k) to get one in each cluster, given that each cluster contains equal number of points. We can later reduce to k clusters, by merging extra clusters. Randomly partition X = {X 1, X 2,..., X k } and take c i = average(x i ). This biases towards center of X. 152
48 Initialize C Random set of points. By coupon collectors, we know that we need about O(k log k) to get one in each cluster, given that each cluster contains equal number of points. We can later reduce to k clusters, by merging extra clusters. Randomly partition X = {X 1, X 2,..., X k } and take c i = average(x i ). This biases towards center of X. Gonzalez algorithm (for kcenter). This biases too much to outlier points. 153
49 Accuracy Can be arbitrarily bad. (Give an example?) 161
50 Accuracy Can be arbitrarily bad. (Give an example?) 4 vertices of a rectangle (width height) 162
51 Accuracy Can be arbitrarily bad. (Give an example?) Theory algorithm: Gets (1 + ɛ)approximation for kmeans in 2 (k/ɛ)o(1) nd time. (Kumar, Sabharwal, Sen 2004) 163
52 Accuracy 164 Can be arbitrarily bad. (Give an example?) Theory algorithm: Gets (1 + ɛ)approximation for kmeans in 2 (k/ɛ)o(1) nd time. (Kumar, Sabharwal, Sen 2004) The following kmeans++ is O(log k)approximation a. a Ref: kmeans++: The Advantages of Careful Seeding kmeans++ 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Choose c i from X with probability proportional to d(x, φ Si 1 (x)) 2 (b) Let S i = {c 1,..., c i }
53 Other issues The key step that makes Lloyds algorithm so cool is average{x X } = arg min c x 2 c R d 2. x X But it only works for distance function d(x, c) = x c 2 Is effected by outliers more than kmedian clustering. 171
54 Local Search Algorithm Local Search Algorithm Designed for kmedian 1. Choose K centers at random from X. Call this set C. 2. Repeat For each u in X and c in C, compute Cost(C {c} + {u}). Find (u, c) for which this cost is smallest, and replace C with C {c} + {u}. 3. Stop when Cost(C) does not change significantly 181
55 Local Search Algorithm Local Search Algorithm Designed for kmedian 1. Choose K centers at random from X. Call this set C. 2. Repeat For each u in X and c in C, compute Cost(C {c} + {u}). Find (u, c) for which this cost is smallest, and replace C with C {c} + {u}. 3. Stop when Cost(C) does not change significantly Can be used to design a 5approximation algorithm a. a Ref: Local Search Heuristics for kmedian and Facility Location Problems Q: How can you speed up the swap operation by avoiding searching over all (u, c) pairs? 182
56 191 Spectural Clustering
57 Topdown clustering Topdown Clustering Framework 1. Find the best cut of the data into two pieces. 2. Recur on both pieces until that data should not be split anymore. 201
58 Topdown clustering Topdown Clustering Framework 1. Find the best cut of the data into two pieces. 2. Recur on both pieces until that data should not be split anymore. What is the best way to partition the data? This is a big question. Let s first talk about graphs. 202
59 Graphs Graph: G = (V, E) is defined by a set of vertices V = {v 1, v 2,..., v n } and a set of edges E = {e 1, e 2,..., e m } where each edge e j is an unordered (or ordered in a directed graph) pair of edges e j = {v i, v i } (or e j = (v i, v i )). Example: e g a b c d f h 211
60 Graphs Graph: G = (V, E) is defined by a set of vertices V = {v 1, v 2,..., v n } and a set of edges E = {e 1, e 2,..., e m } where each edge e j is an unordered (or ordered in a directed graph) pair of edges e j = {v i, v i } (or e j = (v i, v i )). Example: e c a d b f g h Adjacent list representation A =
61 From point sets to graphs ɛneighborhood graph: Connect all points whose pairwise distance are smaller than ɛ. An unweighted graph 221
62 From point sets to graphs ɛneighborhood graph: Connect all points whose pairwise distance are smaller than ɛ. An unweighted graph knearest neighbor graphs: Connect v i with v j if v j is among the knearest neighbors of v i. Naturally a directed graph. Two ways to make it undirected: (1) neglect the direction; (2) mutual knearest neighbor. 222
63 From point sets to graphs ɛneighborhood graph: Connect all points whose pairwise distance are smaller than ɛ. An unweighted graph knearest neighbor graphs: Connect v i with v j if v j is among the knearest neighbors of v i. Naturally a directed graph. Two ways to make it undirected: (1) neglect the direction; (2) mutual knearest neighbor. The fully connected graph: connect all points with finite distances with each other, and weight all edges by d(v i, v j ), where d(v i, v j ) is the distance between v i and v j. 223
64 A good cut a b c d e f g h Rule of the thumb 1. Many edges in a cluster. Volume if a cluster is Vol(S) = the number of edges with at least one vertex in V. 2. Few edges between clusters. Cut between two clusters S, T is Cut(S, T ) = the number of edges with one vertex in S and the other in T. 231
65 A good cut a b c d e f g h Rule of the thumb 1. Many edges in a cluster. Volume if a cluster is Vol(S) = the number of edges with at least one vertex in V. 2. Few edges between clusters. Cut between two clusters S, T is Cut(S, T ) = the number of edges with one vertex in S and the other in T Ratio cut: RatioCut(S, T ) = Cut(S,T ) S + Cut(S,T ) T Normalized cut: NCut(S, T ) = Cut(S,T ) Vol(S) + Cut(S,T ) Vol(T )
66 A good cut a b c Rule of the thumb 1. Many edges in a cluster. 2. Few edges between clusters. d e f g h Can be generalize to general k Volume if a cluster is Vol(S) = the number of edges with at least one vertex in V. Cut between two clusters S, T is Cut(S, T ) = the number of edges with one vertex in S and the other in T Ratio cut: RatioCut(S, T ) = Cut(S,T ) S + Cut(S,T ) T Normalized cut: NCut(S, T ) = Cut(S,T ) Vol(S) + Cut(S,T ) Vol(T )
67 Laplacian matrix and eigenvector 1. Adjacent matrix of the graph A 2. Degree (diagonal) matrix D 3. Laplacian matrix L = D A 4. Eigenvector of a matrix M is the vector v such that Mv = λv, where λ is a scalar, called the corresponding eigenvalue. a b c e d f g h 241
68 Laplacian matrix and eigenvector 1. Adjacent matrix of the graph A 2. Degree (diagonal) matrix D 3. Laplacian matrix L = D A 4. Eigenvector of a matrix M is the vector v such that Mv = λv, where λ is a scalar, called the corresponding eigenvalue. a b c e d f g h Adjacent list matrix A = Degree matrix D =
69 Laplacian matrix and eigenvector (cont.) L = D A =
70 Laplacian matrix and eigenvector (cont.) L = D A = Eigenvalues and eigenvectors (after normalization) of Laplacian matrix 252 λ / / 2 1/ / V 1/ / 2 1/ / / /
71 Laplacian matrix and eigenvector (cont.) L = D A = Eigenvalues and eigenvectors (after normalization) of Laplacian matrix 253 λ / / 2 1/ / V 1/ / 2 1/ / / / (Graph embedding on board)
72 Laplacian matrix and eigenvector (cont.) L = D A = Eigenvalues and eigenvectors (after normalization) of Laplacian matrix 254 λ / / 2 1/ / V 1/ / 2 1/ / / / (Graph embedding on board)
73 Spectral clustering algorithm Spectral Clustering 1. Compute the Laplacian L 2. Compute the first k eigenvectors u 1,..., u k of L 3. Let U R n k be the matrix containing the vectors u 1,..., u k as columns 4. For i = 1,..., n, let y i R k be the vector corresponding to the ith row of U. 5. Cluster the points {y 1,..., y n } in R k with kmeans algorithm into clusters C 1,..., C k 261
74 The connections When k = 2, we want to compute min S V RatioCut(S, T ) 271
75 The connections When k = 2, we want to compute min S V RatioCut(S, T ) We can prove that this is equivalent to min f T Lf s.t. f 1, f = 1, and f i is defined as S V f i = 1 n T S if v i S 1 n S T if v i T = V \S 272
76 The connections When k = 2, we want to compute min S V RatioCut(S, T ) We can prove that this is equivalent to min f T Lf s.t. f 1, f = 1, and f i is defined as S V f i = 1 n T S if v i S 1 n S T if v i T = V \S This is NPhard. And we solve the following relaxed optimization problem instead. min f R n f T Lf s.t. f 1, f = The best f is the second smallest eigenvalue of L.
77 281 Clustering Social Network
78 Social network and communities Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. 291
79 Social network and communities Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. networks 292
80 Social network and communities Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. networks Collaboration networks: E.g., two people publish paper together are connected. Communities: authors working on a particular topic (Wikipedia articles), people working on the same project in Google. 293
81 Social network and communities 294 Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. networks Collaboration networks: E.g., two people publish paper together are connected. Communities: authors working on a particular topic (Wikipedia articles), people working on the same project in Google. Goal: To find the communities. Similar to normal clustering but we sometimes allow overlaps (will not discuss here).
82 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied 301
83 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied Hierarchical clustering: may combine c, e first. 302
84 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied Hierarchical clustering: may combine c, e first. Assignmentbased clustering, e.g., kmean: if two random seeds are chosen to be b, e, then c may be assigned to e 303
85 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied Hierarchical clustering: may combine c, e first. Assignmentbased clustering, e.g., kmean: if two random seeds are chosen to be b, e, then c may be assigned to e Spectral clustering: may work. Try it. 304
86 Betweenness A new distance measure in social network: Betweenness Betweenness(A, B): number of shortest paths that use edge (A, B). 311
87 Betweenness A new distance measure in social network: Betweenness Betweenness(A, B): number of shortest paths that use edge (A, B). Intuition: If to get between two communities you need to take this edge, its betweenness score will be high. An edge with a high betweenness may be a facilitator edge, not a community edge. 312
88 GirvanNewman Algorithm GirvanNewman Algorithm 1. For each v V (a) Run BFS on v to build a directed acyclic graph (DAG) on the entire graph. Give 1 credit to each node except the root. (b) Walk back up the DAG and add a credit to each edge. When paths merge, the credits adds up, and when paths split, the credit splits as well. 2. The final score of each edge is the summation of all credits of the n runs, divided by Remove edges with high betweenness, and the remaining connected components are communities. Example (on board) 321
89 GirvanNewman Algorithm GirvanNewman Algorithm For each v V (a) Run BFS on v to build a directed acyclic graph (DAG) on the entire graph. Give 1 credit to each node except the root. (b) Walk back up the DAG and add a credit to each edge. When paths merge, the credits adds up, and when paths split, the credit splits as well. 2. The final score of each edge is the summation of all credits of the n runs, divided by Remove edges with high betweenness, and the remaining connected components are communities. Example (on board) What s the running time? Can we speed it?
90 331 What do the real graphs look like?
91 Powerlaw distributions Powerlaw distributions A nonnegative random variable X is said to have a power law distribution if Pr[X = x] cx α, for constants c > 0 and α > Zipf s Law: states that the frequency of the j the most common word in English (or other common languages) is proportional to j 1 A city grows in proportion to its current size as a result of people having children. Price studied the network of citations between scientific papers and found that the in degrees (number of times a paper has been cited) have power law distributions. WWW,... the rich get richer, heavy tails
92 Powerlaw graphs 351
93 Preferential attachment 361 BarabasiAlbert model When a new node is added to the network at each time t N. 1. With a probability p [0, 1], this new node connects to m existing nodes uniformly at random. 2. With a probability 1 p, this new node connects to m existing nodes with a probability proportional to the degree of node which it will be connected to. After some math, we get m(k) k β, where m(k) is the number of nodes whose degree is k, and β = 3 p 1 p. (on board, for p = 0)
94 Thank you! Some slides are based on the MMDS book and Jeff Phillips lecture notes 371
Part 2: Community Detection
Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection  Social networks 
More informationDATA ANALYSIS II. Matrix Algorithms
DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where
More informationMining SocialNetwork Graphs
342 Chapter 10 Mining SocialNetwork Graphs There is much information to be gained by analyzing the largescale data that is derived from social networks. The bestknown example of a social network is
More informationWhy graph clustering is useful?
Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as any other clustering Identification of communities in social networks Webpage clustering for better data management
More information! Solve problem to optimality. ! Solve problem in polytime. ! Solve arbitrary instances of the problem. #approximation algorithm.
Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NPhard problem What should I do? A Theory says you're unlikely to find a polytime algorithm Must sacrifice one of three
More information! Solve problem to optimality. ! Solve problem in polytime. ! Solve arbitrary instances of the problem. !approximation algorithm.
Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NPhard problem What should I do? A Theory says you're unlikely to find a polytime algorithm Must sacrifice one of
More informationNeural Networks Lesson 5  Cluster Analysis
Neural Networks Lesson 5  Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt.  Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29
More informationChapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling
Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NPhard problem. What should I do? A. Theory says you're unlikely to find a polytime algorithm. Must sacrifice one
More informationApproximation Algorithms. Scheduling. Approximation algorithms. Scheduling jobs on a single machine
Approximation algorithms Approximation Algorithms Fast. Cheap. Reliable. Choose two. NPhard problems: choose 2 of optimal polynomial time all instances Approximation algorithms. Tradeoff between time
More informationUSING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE FREE NETWORKS AND SMALLWORLD NETWORKS
USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE FREE NETWORKS AND SMALLWORLD NETWORKS Natarajan Meghanathan Jackson State University, 1400 Lynch St, Jackson, MS, USA natarajan.meghanathan@jsums.edu
More informationClustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016
Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with
More informationDistance based clustering
// Distance based clustering Chapter ² ² Clustering Clustering is the art of finding groups in data (Kaufman and Rousseeuw, 99). What is a cluster? Group of objects separated from other clusters Means
More informationApplied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
More informationNimble Algorithms for Cloud Computing. Ravi Kannan, Santosh Vempala and David Woodruff
Nimble Algorithms for Cloud Computing Ravi Kannan, Santosh Vempala and David Woodruff Cloud computing Data is distributed arbitrarily on many servers Parallel algorithms: time Streaming algorithms: sublinear
More informationData Mining. Cluster Analysis: Advanced Concepts and Algorithms
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototypebased clustering Densitybased clustering Graphbased
More informationCMPSCI611: Approximating MAXCUT Lecture 20
CMPSCI611: Approximating MAXCUT Lecture 20 For the next two lectures we ll be seeing examples of approximation algorithms for interesting NPhard problems. Today we consider MAXCUT, which we proved to
More informationWellSeparated Pair Decomposition for the Unitdisk Graph Metric and its Applications
WellSeparated Pair Decomposition for the Unitdisk Graph Metric and its Applications Jie Gao Department of Computer Science Stanford University Joint work with Li Zhang Systems Research Center HewlettPackard
More informationApproximation Algorithms
Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NPCompleteness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms
More informationLecture 9. 1 Introduction. 2 Random Walks in Graphs. 1.1 How To Explore a Graph? CS621 Theory Gems October 17, 2012
CS62 Theory Gems October 7, 202 Lecture 9 Lecturer: Aleksander Mądry Scribes: Dorina Thanou, Xiaowen Dong Introduction Over the next couple of lectures, our focus will be on graphs. Graphs are one of
More informationClustering and Data Mining in R
Clustering and Data Mining in R Workshop Supplement Thomas Girke December 10, 2011 Introduction Data Preprocessing Data Transformations Distance Methods Cluster Linkage Hierarchical Clustering Approaches
More informationARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING)
ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Preliminaries Classification and Clustering Applications
More informationMining Social Network Graphs
Mining Social Network Graphs Debapriyo Majumdar Data Mining Fall 2014 Indian Statistical Institute Kolkata November 13, 17, 2014 Social Network No introduc+on required Really? We s7ll need to understand
More informationMedical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu
Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?
More information6.042/18.062J Mathematics for Computer Science October 3, 2006 Tom Leighton and Ronitt Rubinfeld. Graph Theory III
6.04/8.06J Mathematics for Computer Science October 3, 006 Tom Leighton and Ronitt Rubinfeld Lecture Notes Graph Theory III Draft: please check back in a couple of days for a modified version of these
More informationCLASSIFICATION AND CLUSTERING. Anveshi Charuvaka
CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training
More information2.3 Scheduling jobs on identical parallel machines
2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed
More informationB490 Mining the Big Data. 0 Introduction
B490 Mining the Big Data 0 Introduction Qin Zhang 11 Data Mining What is Data Mining? A definition : Discovery of useful, possibly unexpected, patterns in data. 21 Data Mining What is Data Mining? A
More informationLecture 4 Online and streaming algorithms for clustering
CSE 291: Geometric algorithms Spring 2013 Lecture 4 Online and streaming algorithms for clustering 4.1 Online kclustering To the extent that clustering takes place in the brain, it happens in an online
More informationSocial Media Mining. Network Measures
Klout Measures and Metrics 22 Why Do We Need Measures? Who are the central figures (influential individuals) in the network? What interaction patterns are common in friends? Who are the likeminded users
More informationCluster Analysis: Advanced Concepts
Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototypebased Fuzzy cmeans
More informationCluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico
Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from
More informationSocial Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers
More informationData Mining Project Report. Document Clustering. Meryem UzunPer
Data Mining Project Report Document Clustering Meryem UzunPer 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. Kmeans algorithm...
More informationAlgorithmic Aspects of Big Data. Nikhil Bansal (TU Eindhoven)
Algorithmic Aspects of Big Data Nikhil Bansal (TU Eindhoven) Algorithm design Algorithm: Set of steps to solve a problem (by a computer) Studied since 1950 s. Given a problem: Find (i) best solution (ii)
More informationDATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS
DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDDLAB ISTI CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar
More informationPrivate Approximation of Clustering and Vertex Cover
Private Approximation of Clustering and Vertex Cover Amos Beimel, Renen Hallak, and Kobbi Nissim Department of Computer Science, BenGurion University of the Negev Abstract. Private approximation of search
More informationInformation Retrieval and Web Search Engines
Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 10 th, 2013 WolfTilo Balke and Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig
More informationAn Enhanced Clustering Algorithm to Analyze Spatial Data
International Journal of Engineering and Technical Research (IJETR) ISSN: 23210869, Volume2, Issue7, July 2014 An Enhanced Clustering Algorithm to Analyze Spatial Data Dr. Mahesh Kumar, Mr. Sachin Yadav
More informationData Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004
More informationGeometrical Segmentation of Point Cloud Data using Spectral Clustering
Geometrical Segmentation of Point Cloud Data using Spectral Clustering Sergey Alexandrov and Rainer Herpers University of Applied Sciences BonnRheinSieg 15th July 2014 1/29 Addressed Problem Given: a
More informationGraph Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar
Graph Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text Introduction to Parallel Computing, Addison Wesley, 3. Topic Overview Definitions and Representation Minimum
More informationLesson 3. Algebraic graph theory. Sergio Barbarossa. Rome  February 2010
Lesson 3 Algebraic graph theory Sergio Barbarossa Basic notions Definition: A directed graph (or digraph) composed by a set of vertices and a set of edges We adopt the convention that the information flows
More informationClustering Hierarchical clustering and kmean clustering
Clustering Hierarchical clustering and kmean clustering Genome 373 Genomic Informatics Elhanan Borenstein The clustering problem: A quick review partition genes into distinct sets with high homogeneity
More informationChapter 4: NonParametric Classification
Chapter 4: NonParametric Classification Introduction Density Estimation Parzen Windows KnNearest Neighbor Density Estimation KNearest Neighbor (KNN) Decision Rule Gaussian Mixture Model A weighted combination
More informationSome questions... Graphs
Uni Innsbruck Informatik  1 Uni Innsbruck Informatik  2 Some questions... Peerto topeer Systems Analysis of unstructured P2P systems How scalable is Gnutella? How robust is Gnutella? Why does FreeNet
More informationprinceton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora
princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of
More informationLecture 20: Clustering
Lecture 20: Clustering Wrapup of neural nets (from last lecture Introduction to unsupervised learning Kmeans clustering COMP424, Lecture 20  April 3, 2013 1 Unsupervised learning In supervised learning,
More informationClustering & Association
Clustering  Overview What is cluster analysis? Grouping data objects based only on information found in the data describing these objects and their relationships Maximize the similarity within objects
More informationA Survey of Kernel Clustering Methods
A Survey of Kernel Clustering Methods Maurizio Filippone, Francesco Camastra, Francesco Masulli and Stefano Rovetta Presented by: Kedar Grama Outline Unsupervised Learning and Clustering Types of clustering
More informationCMSC 451: Graph Properties, DFS, BFS, etc.
CMSC 451: Graph Properties, DFS, BFS, etc. Slides By: Carl Kingsford Department of Computer Science University of Maryland, College Park Based on Chapter 3 of Algorithm Design by Kleinberg & Tardos. Graphs
More informationGraph Theory and Complex Networks: An Introduction. Chapter 06: Network analysis. Contents. Introduction. Maarten van Steen. Version: April 28, 2014
Graph Theory and Complex Networks: An Introduction Maarten van Steen VU Amsterdam, Dept. Computer Science Room R.0, steen@cs.vu.nl Chapter 0: Version: April 8, 0 / Contents Chapter Description 0: Introduction
More information8.1 Makespan Scheduling
600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programing: MinMakespan and Bin Packing Date: 2/19/15 Scribe: Gabriel Kaptchuk 8.1 Makespan Scheduling Consider an instance
More informationChapter ML:XI (continued)
Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis DensityBased Cluster Analysis Cluster Evaluation Constrained
More informationAIMA Chapter 3: Solving problems by searching
AIMA Chapter 3: Solving problems by searching Ana Huaman January 13, 2011 1. Define in your own words the following terms: state, state space, search tree, search node, goal, action, transition model and
More informationGuessing Game: NPComplete?
Guessing Game: NPComplete? 1. LONGESTPATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTESTPATH: Given a graph G = (V, E), does there exists a simple
More informationCSC2420 Fall 2012: Algorithm Design, Analysis and Theory
CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms
More informationUNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS
UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS Dwijesh C. Mishra I.A.S.R.I., Library Avenue, New Delhi110 012 dcmishra@iasri.res.in What is Learning? "Learning denotes changes in a system that enable
More informationCSE 494 CSE/CBS 598 (Fall 2007): Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye
CSE 494 CSE/CBS 598 Fall 2007: Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye 1 Introduction One important method for data compression and classification is to organize
More informationApproximated Distributed Minimum Vertex Cover Algorithms for Bounded Degree Graphs
Approximated Distributed Minimum Vertex Cover Algorithms for Bounded Degree Graphs Yong Zhang 1.2, Francis Y.L. Chin 2, and HingFung Ting 2 1 College of Mathematics and Computer Science, Hebei University,
More information10810 /02710 Computational Genomics. Clustering expression data
10810 /02710 Computational Genomics Clustering expression data What is Clustering? Organizing data into clusters such that there is high intracluster similarity low intercluster similarity Informally,
More informationGRAPH data are important data types in many scientific
Limited Random Walk Algorithm for Big Graph Data Clustering Honglei Zhang, Member, IEEE, Jenni Raitoharju, Member, IEEE, Serkan Kiranyaz, Senior Member, IEEE, and Moncef Gabbouj, Fellow, IEEE arxiv:66.645v
More informationGRAPH THEORY and APPLICATIONS. Trees
GRAPH THEORY and APPLICATIONS Trees Properties Tree: a connected graph with no cycle (acyclic) Forest: a graph with no cycle Paths are trees. Star: A tree consisting of one vertex adjacent to all the others.
More informationLecture 7: Approximation via Randomized Rounding
Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining
More informationA Tutorial on Spectral Clustering
A Tutorial on Spectral Clustering Ulrike von Luxburg Max Planck Institute for Biological Cybernetics Spemannstr. 38, 7276 Tübingen, Germany ulrike.luxburg@tuebingen.mpg.de This article appears in Statistics
More informationSocial Networks and Social Media
Social Networks and Social Media Social Media: ManytoMany Social Networking Content Sharing Social Media Blogs Microblogging Wiki Forum 2 Characteristics of Social Media Consumers become Producers Rich
More informationIntroduction to Segmentation
Lecture 2: Introduction to Segmentation Jonathan Krause 1 Goal Goal: Identify groups of pixels that go together image credit: Steve Seitz, Kristen Grauman 2 Types of Segmentation Semantic Segmentation:
More informationConductance, the Normalized Laplacian, and Cheeger s Inequality
Spectral Graph Theory Lecture 6 Conductance, the Normalized Laplacian, and Cheeger s Inequality Daniel A. Spielman September 21, 2015 Disclaimer These notes are not necessarily an accurate representation
More informationChapter 6. Cuboids. and. vol(conv(p ))
Chapter 6 Cuboids We have already seen that we can efficiently find the bounding box Q(P ) and an arbitrarily good approximation to the smallest enclosing ball B(P ) of a set P R d. Unfortunately, both
More informationVector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a nonempty
More informationDistances, Clustering, and Classification. Heatmaps
Distances, Clustering, and Classification Heatmaps 1 Distance Clustering organizes things that are close into groups What does it mean for two genes to be close? What does it mean for two samples to be
More informationDistance Metric Learning in Data Mining (Part I) Fei Wang and Jimeng Sun IBM TJ Watson Research Center
Distance Metric Learning in Data Mining (Part I) Fei Wang and Jimeng Sun IBM TJ Watson Research Center 1 Outline Part I  Applications Motivation and Introduction Patient similarity application Part II
More informationIEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 20, NO. 7, JULY 2009 1181
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 20, NO. 7, JULY 2009 1181 The Global Kernel kmeans Algorithm for Clustering in Feature Space Grigorios F. Tzortzis and Aristidis C. Likas, Senior Member, IEEE
More informationAlgorithm Design and Analysis
Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;
More informationEfficient Load Balancing by Adaptive Bypasses for the Migration on the Internet
Efficient Load Balancing by Adaptive Bypasses for the Migration on the Internet Yukio Hayashi yhayashi@jaist.ac.jp Japan Advanced Institute of Science and Technology ICCS 03 Workshop on Grid Computing,
More information1 Basic Definitions and Concepts in Graph Theory
CME 305: Discrete Mathematics and Algorithms 1 Basic Definitions and Concepts in Graph Theory A graph G(V, E) is a set V of vertices and a set E of edges. In an undirected graph, an edge is an unordered
More informationUnsupervised Learning: Clustering with DBSCAN Mat Kallada
Unsupervised Learning: Clustering with DBSCAN Mat Kallada STAT 2450  Introduction to Data Mining Supervised Data Mining: Predicting a column called the label The domain of data mining focused on prediction:
More informationSegmentation & Clustering
EECS 442 Computer vision Segmentation & Clustering Segmentation in human vision Kmean clustering Meanshift Graphcut Reading: Chapters 14 [FP] Some slides of this lectures are courtesy of prof F. Li,
More informationFlat Clustering KMeans Algorithm
Flat Clustering KMeans Algorithm 1. Purpose. Clustering algorithms group a set of documents into subsets or clusters. The cluster algorithms goal is to create clusters that are coherent internally, but
More informationMapReduce Algorithms. Sergei Vassilvitskii. Saturday, August 25, 12
MapReduce Algorithms A Sense of Scale At web scales... Mail: Billions of messages per day Search: Billions of searches per day Social: Billions of relationships 2 A Sense of Scale At web scales... Mail:
More informationSimilarity and Diagonalization. Similar Matrices
MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that
More informationMachine Learning using MapReduce
Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous
More informationData Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004
More informationLecture Topic: LowRank Approximations
Lecture Topic: LowRank Approximations LowRank Approximations We have seen principal component analysis. The extraction of the first principle eigenvalue could be seen as an approximation of the original
More informationComplex Networks Analysis: Clustering Methods
Complex Networks Analysis: Clustering Methods Nikolai Nefedov Spring 2013 ISI ETH Zurich nefedov@isi.ee.ethz.ch 1 Outline Purpose to give an overview of modern graphclustering methods and their applications
More informationData visualization and clustering. Genomics is to no small extend a data science
Data visualization and clustering Genomics is to no small extend a data science [www.data2discovery.org] Data visualization and clustering Genomics is to no small extend a data science [Andersson et al.,
More informationClustering. Chapter 7. 7.1 Introduction to Clustering Techniques. 7.1.1 Points, Spaces, and Distances
240 Chapter 7 Clustering Clustering is the process of examining a collection of points, and grouping the points into clusters according to some distance measure. The goal is that points in the same cluster
More informationFig. 1 A typical Knowledge Discovery process [2]
Volume 4, Issue 7, July 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Review on Clustering
More informationClosest Pair Problem
Closest Pair Problem Given n points in ddimensions, find two whose mutual distance is smallest. Fundamental problem in many applications as well as a key step in many algorithms. p q A naive algorithm
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/2004 Hierarchical
More informationMATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).
MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0
More informationApproximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques. My T. Thai
Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques My T. Thai 1 Overview An overview of LP relaxation and rounding method is as follows: 1. Formulate an optimization
More informationDistance Degree Sequences for Network Analysis
Universität Konstanz Computer & Information Science Algorithmics Group 15 Mar 2005 based on Palmer, Gibbons, and Faloutsos: ANF A Fast and Scalable Tool for Data Mining in Massive Graphs, SIGKDD 02. Motivation
More informationLABEL PROPAGATION ON GRAPHS. SEMISUPERVISED LEARNING. Changsheng Liu 10302014
LABEL PROPAGATION ON GRAPHS. SEMISUPERVISED LEARNING Changsheng Liu 10302014 Agenda Semi Supervised Learning Topics in Semi Supervised Learning Label Propagation Local and global consistency Graph
More informationLecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method
Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming
More informationIn the following we will only consider undirected networks.
Roles in Networks Roles in Networks Motivation for work: Let topology define network roles. Work by Kleinberg on directed graphs, used topology to define two types of roles: authorities and hubs. (Each
More informationRandom Map Generator v1.0 User s Guide
Random Map Generator v1.0 User s Guide Jonathan Teutenberg 2003 1 Map Generation Overview...4 1.1 Command Line...4 1.2 Operation Flow...4 2 Map Initialisation...5 2.1 Initialisation Parameters...5 w xxxxxxx...5
More informationClustering. Data Mining. Abraham Otero. Data Mining. Agenda
Clustering 1/46 Agenda Introduction Distance Knearest neighbors Hierarchical clustering Quick reference 2/46 1 Introduction It seems logical that in a new situation we should act in a similar way as in
More informationGoing Big in Data Dimensionality:
LUDWIG MAXIMILIANS UNIVERSITY MUNICH DEPARTMENT INSTITUTE FOR INFORMATICS DATABASE Going Big in Data Dimensionality: Challenges and Solutions for Mining High Dimensional Data Peer Kröger Lehrstuhl für
More informationDynamic Programming. Lecture 11. 11.1 Overview. 11.2 Introduction
Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach
More informationComplexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar
Complexity Theory IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Outline Goals Computation of Problems Concepts and Definitions Complexity Classes and Problems Polynomial Time Reductions Examples
More information