B490 Mining the Big Data. 2 Clustering


 Gwenda Horton
 1 years ago
 Views:
Transcription
1 B490 Mining the Big Data 2 Clustering Qin Zhang 11
2 Motivations Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Cluster topic/articles from the web by same story. Google image Many others
3 Motivations 22 Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Cluster topic/articles from the web by same story. Google image Many others... Given a cloud of data points we want to understand their structures Community discovery in social networks
4 Motivations 23 Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Cluster topic/articles from the web by same story. Google image Many others... Given a cloud of data points we want to understand their structures Community discovery in social networks
5 The plan Clustering is an extremely broad and illdefined topic. There are many many variants; we will not try to cover all of them. Instead we will focus on three broad classes of algorithms Hierachical Clustering Assignmentbased Clustering Spectural Clustering 31
6 The plan Clustering is an extremely broad and illdefined topic. There are many many variants; we will not try to cover all of them. Instead we will focus on three broad classes of algorithms Hierachical Clustering Assignmentbased Clustering Spectural Clustering There is a saying When data is easily clusterable, most clustering algorithms work quickly and well. When is not easily clusterable, then no algorithm will find good clusters. 32
7 41 Hierachical Clustering
8 Hard clustering Problem: Start with a set X R d, and a metric d : R d R d R +. A cluster C is a subset of X. A clustering is a partition ρ(x ) = {C 1, C 2,..., C k }, where 1. Each C i X. 2. Each pair C i C j =. 3. Each k i=1 C i = X. 51
9 Hard clustering Problem: Start with a set X R d, and a metric d : R d R d R +. A cluster C is a subset of X. A clustering is a partition ρ(x ) = {C 1, C 2,..., C k }, where 1. Each C i X. 2. Each pair C i C j =. 3. Each k i=1 C i = X. Goal: 1. (close intra) For each C ρ(x ) for all x, x C, d(x, x ) is small. 2. (far inter) For each C i, C j ρ(x ) (i j), for most x i C i and x j C j, d(x i, x j ) is large. 52
10 Hierarchical clustering Hierarchical Clustering 1. Each x i X forms a cluster C i 2. While exists two clusters close enough (a) Find the closest two clusters C i, C j. (b) Merge C i, C j into a single cluster 61
11 Hierarchical clustering Hierarchical Clustering 1. Each x i X forms a cluster C i 2. While exists two clusters close enough (a) Find the closest two clusters C i, C j. (b) Merge C i, C j into a single cluster Two things to define 1. Define a distance between clusters. 2. Define close enough 62
12 Definition of distance between clusters Distance between center of clusters. What does center mean? 71
13 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. 72
14 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) 73
15 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) 74
16 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point 75
17 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points 76
18 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points Distance between furthest pair of points 77
19 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points Distance between furthest pair of points Average distance between all pairs of points in different clusters 78
20 Definition of distance between clusters 79 Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the centerpoint (like a highdimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points Distance between furthest pair of points Average distance between all pairs of points in different clusters Radius of minimum enclosing ball of joined cluster
21 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. 81
22 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. 82
23 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d 83
24 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d If the joint density decrease too much from single cluster density. Variations of this are called the elbow technique. 84
25 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d If the joint density decrease too much from single cluster density. Variations of this are called the elbow technique. When the number of clusters is k (for some magical value k). 85
26 Definition of close enough 86 If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d If the joint density decrease too much from single cluster density. Variations of this are called the elbow technique. When the number of clusters is k (for some magical value k). Example (on board): Distance is distance between centroids. Stop when there is 1 cluster left.
27 Assignmentbased Clustering (kcenter, kmean, kmedian) 91
28 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). 101
29 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). The kcenter cluster problem is to find the set C of k clusters (often, but not always as a subset of X ) to minimize max x X d(φ C (x), x). 102
30 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). The kcenter cluster problem is to find the set C of k clusters (often, but not always as a subset of X ) to minimize max x X d(φ C (x), x). The kmeans cluster problem: minimize x X d(φ C (x), x)
31 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). The kcenter cluster problem is to find the set C of k clusters (often, but not always as a subset of X ) to minimize max x X d(φ C (x), x). The kmeans cluster problem: minimize x X d(φ C (x), x) 2 The kmedian cluster problem: minimize x X d(φ C (x), x) 104
32 Gonzalez Algorithm Gonzalez Algorithm A simple greedy algorithm for kcenter 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Set c i = arg max x X d(x, φ Si 1 (x)) (b) Let S i = {c 1,..., c i } 111
33 Gonzalez Algorithm Gonzalez Algorithm A simple greedy algorithm for kcenter 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Set c i = arg max x X d(x, φ Si 1 (x)) (b) Let S i = {c 1,..., c i } In the worst case, it gives a 2approximation to the optimal solution. 112
34 Gonzalez Algorithm Gonzalez Algorithm A simple greedy algorithm for kcenter 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Set c i = arg max x X d(x, φ Si 1 (x)) (b) Let S i = {c 1,..., c i } In the worst case, it gives a 2approximation to the optimal solution. Analysis (on board) 113
35 Parallel Guessing Algorithm Parallel Guessing Algorithm for kcenter Run for R = (1 + ɛ/2), (1 + ɛ/2) 2,..., 1. We pick an arbitrary point as the first center, C = {c 1 }. 2. For each point p P compute r p = min c C d(p, c) and test whether r p > R. If it is, then set C = C {p}. 3. Abort at C > k Pick the minimum R that does not abort. 121
36 Parallel Guessing Algorithm Parallel Guessing Algorithm for kcenter Run for R = (1 + ɛ/2), (1 + ɛ/2) 2,..., 1. We pick an arbitrary point as the first center, C = {c 1 }. 2. For each point p P compute r p = min c C d(p, c) and test whether r p > R. If it is, then set C = C {p}. 3. Abort at C > k Pick the minimum R that does not abort. In the worst case, it gives a (2 + ɛ)approximation to the optimal solution (on board). 122
37 Parallel Guessing Algorithm Parallel Guessing Algorithm for kcenter Run for R = (1 + ɛ/2), (1 + ɛ/2) 2,..., 1. We pick an arbitrary point as the first center, C = {c 1 }. 2. For each point p P compute r p = min c C d(p, c) and test whether r p > R. If it is, then set C = C {p}. 3. Abort at C > k Pick the minimum R that does not abort. In the worst case, it gives a (2 + ɛ)approximation to the optimal solution (on board). Can be implemented as a streaming algorithm performs a linear scan of the data, uses space Õ(k) and time Õ(nk). 123
38 Lloyd s Algorithm Lloyd s Algorithm The wellknown algorithm for kmeans 1. Choose k points C X (arbitrarily?) 2. Repeat (a) For all x X, find φ C (x) (closest center c C to x) (b) For all i [k], let c i = average{x X φ C (x) = c i } 3. until the set C is unchanged Kmeans_Kmedoids.html 131
39 Lloyd s Algorithm Lloyd s Algorithm The wellknown algorithm for kmeans 1. Choose k points C X (arbitrarily?) 2. Repeat (a) For all x X, find φ C (x) (closest center c C to x) (b) For all i [k], let c i = average{x X φ C (x) = c i } 3. until the set C is unchanged Kmeans_Kmedoids.html If the main loop has R rounds, then the running time is O(Rnk) 132
40 Lloyd s Algorithm 133 Lloyd s Algorithm The wellknown algorithm for kmeans 1. Choose k points C X (arbitrarily?) 2. Repeat (a) For all x X, find φ C (x) (closest center c C to x) (b) For all i [k], let c i = average{x X φ C (x) = c i } 3. until the set C is unchanged Kmeans_Kmedoids.html If the main loop has R rounds, then the running time is O(Rnk) But 1. What is R? Converge in finite number of steps? 2. How do we initialize C? 3. How accurate is this algorithm?
41 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) 141
42 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. 142
43 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. Smoothed analysis: if data perturbed randomly slightly, then R = O(n 35 k 34 d 8 ). This is polynomial, but still ridiculous. 143
44 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. Smoothed analysis: if data perturbed randomly slightly, then R = O(n 35 k 34 d 8 ). This is polynomial, but still ridiculous. If all points are on a grid of length M, then R = O(dn 4 M 2 ). But thats still way too big. 144
45 Value of R 145 It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. Smoothed analysis: if data perturbed randomly slightly, then R = O(n 35 k 34 d 8 ). This is polynomial, but still ridiculous. If all points are on a grid of length M, then R = O(dn 4 M 2 ). But thats still way too big. Conclusion: There are crazy special cases that can take a long time, but usually it works.
46 Initialize C Random set of points. By coupon collectors, we know that we need about O(k log k) to get one in each cluster, given that each cluster contains equal number of points. We can later reduce to k clusters, by merging extra clusters. 151
47 Initialize C Random set of points. By coupon collectors, we know that we need about O(k log k) to get one in each cluster, given that each cluster contains equal number of points. We can later reduce to k clusters, by merging extra clusters. Randomly partition X = {X 1, X 2,..., X k } and take c i = average(x i ). This biases towards center of X. 152
48 Initialize C Random set of points. By coupon collectors, we know that we need about O(k log k) to get one in each cluster, given that each cluster contains equal number of points. We can later reduce to k clusters, by merging extra clusters. Randomly partition X = {X 1, X 2,..., X k } and take c i = average(x i ). This biases towards center of X. Gonzalez algorithm (for kcenter). This biases too much to outlier points. 153
49 Accuracy Can be arbitrarily bad. (Give an example?) 161
50 Accuracy Can be arbitrarily bad. (Give an example?) 4 vertices of a rectangle (width height) 162
51 Accuracy Can be arbitrarily bad. (Give an example?) Theory algorithm: Gets (1 + ɛ)approximation for kmeans in 2 (k/ɛ)o(1) nd time. (Kumar, Sabharwal, Sen 2004) 163
52 Accuracy 164 Can be arbitrarily bad. (Give an example?) Theory algorithm: Gets (1 + ɛ)approximation for kmeans in 2 (k/ɛ)o(1) nd time. (Kumar, Sabharwal, Sen 2004) The following kmeans++ is O(log k)approximation a. a Ref: kmeans++: The Advantages of Careful Seeding kmeans++ 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Choose c i from X with probability proportional to d(x, φ Si 1 (x)) 2 (b) Let S i = {c 1,..., c i }
53 Other issues The key step that makes Lloyds algorithm so cool is average{x X } = arg min c x 2 c R d 2. x X But it only works for distance function d(x, c) = x c 2 Is effected by outliers more than kmedian clustering. 171
54 Local Search Algorithm Local Search Algorithm Designed for kmedian 1. Choose K centers at random from X. Call this set C. 2. Repeat For each u in X and c in C, compute Cost(C {c} + {u}). Find (u, c) for which this cost is smallest, and replace C with C {c} + {u}. 3. Stop when Cost(C) does not change significantly 181
55 Local Search Algorithm Local Search Algorithm Designed for kmedian 1. Choose K centers at random from X. Call this set C. 2. Repeat For each u in X and c in C, compute Cost(C {c} + {u}). Find (u, c) for which this cost is smallest, and replace C with C {c} + {u}. 3. Stop when Cost(C) does not change significantly Can be used to design a 5approximation algorithm a. a Ref: Local Search Heuristics for kmedian and Facility Location Problems Q: How can you speed up the swap operation by avoiding searching over all (u, c) pairs? 182
56 191 Spectural Clustering
57 Topdown clustering Topdown Clustering Framework 1. Find the best cut of the data into two pieces. 2. Recur on both pieces until that data should not be split anymore. 201
58 Topdown clustering Topdown Clustering Framework 1. Find the best cut of the data into two pieces. 2. Recur on both pieces until that data should not be split anymore. What is the best way to partition the data? This is a big question. Let s first talk about graphs. 202
59 Graphs Graph: G = (V, E) is defined by a set of vertices V = {v 1, v 2,..., v n } and a set of edges E = {e 1, e 2,..., e m } where each edge e j is an unordered (or ordered in a directed graph) pair of edges e j = {v i, v i } (or e j = (v i, v i )). Example: e g a b c d f h 211
60 Graphs Graph: G = (V, E) is defined by a set of vertices V = {v 1, v 2,..., v n } and a set of edges E = {e 1, e 2,..., e m } where each edge e j is an unordered (or ordered in a directed graph) pair of edges e j = {v i, v i } (or e j = (v i, v i )). Example: e c a d b f g h Adjacent list representation A =
61 From point sets to graphs ɛneighborhood graph: Connect all points whose pairwise distance are smaller than ɛ. An unweighted graph 221
62 From point sets to graphs ɛneighborhood graph: Connect all points whose pairwise distance are smaller than ɛ. An unweighted graph knearest neighbor graphs: Connect v i with v j if v j is among the knearest neighbors of v i. Naturally a directed graph. Two ways to make it undirected: (1) neglect the direction; (2) mutual knearest neighbor. 222
63 From point sets to graphs ɛneighborhood graph: Connect all points whose pairwise distance are smaller than ɛ. An unweighted graph knearest neighbor graphs: Connect v i with v j if v j is among the knearest neighbors of v i. Naturally a directed graph. Two ways to make it undirected: (1) neglect the direction; (2) mutual knearest neighbor. The fully connected graph: connect all points with finite distances with each other, and weight all edges by d(v i, v j ), where d(v i, v j ) is the distance between v i and v j. 223
64 A good cut a b c d e f g h Rule of the thumb 1. Many edges in a cluster. Volume if a cluster is Vol(S) = the number of edges with at least one vertex in V. 2. Few edges between clusters. Cut between two clusters S, T is Cut(S, T ) = the number of edges with one vertex in S and the other in T. 231
65 A good cut a b c d e f g h Rule of the thumb 1. Many edges in a cluster. Volume if a cluster is Vol(S) = the number of edges with at least one vertex in V. 2. Few edges between clusters. Cut between two clusters S, T is Cut(S, T ) = the number of edges with one vertex in S and the other in T Ratio cut: RatioCut(S, T ) = Cut(S,T ) S + Cut(S,T ) T Normalized cut: NCut(S, T ) = Cut(S,T ) Vol(S) + Cut(S,T ) Vol(T )
66 A good cut a b c Rule of the thumb 1. Many edges in a cluster. 2. Few edges between clusters. d e f g h Can be generalize to general k Volume if a cluster is Vol(S) = the number of edges with at least one vertex in V. Cut between two clusters S, T is Cut(S, T ) = the number of edges with one vertex in S and the other in T Ratio cut: RatioCut(S, T ) = Cut(S,T ) S + Cut(S,T ) T Normalized cut: NCut(S, T ) = Cut(S,T ) Vol(S) + Cut(S,T ) Vol(T )
67 Laplacian matrix and eigenvector 1. Adjacent matrix of the graph A 2. Degree (diagonal) matrix D 3. Laplacian matrix L = D A 4. Eigenvector of a matrix M is the vector v such that Mv = λv, where λ is a scalar, called the corresponding eigenvalue. a b c e d f g h 241
68 Laplacian matrix and eigenvector 1. Adjacent matrix of the graph A 2. Degree (diagonal) matrix D 3. Laplacian matrix L = D A 4. Eigenvector of a matrix M is the vector v such that Mv = λv, where λ is a scalar, called the corresponding eigenvalue. a b c e d f g h Adjacent list matrix A = Degree matrix D =
69 Laplacian matrix and eigenvector (cont.) L = D A =
70 Laplacian matrix and eigenvector (cont.) L = D A = Eigenvalues and eigenvectors (after normalization) of Laplacian matrix 252 λ / / 2 1/ / V 1/ / 2 1/ / / /
71 Laplacian matrix and eigenvector (cont.) L = D A = Eigenvalues and eigenvectors (after normalization) of Laplacian matrix 253 λ / / 2 1/ / V 1/ / 2 1/ / / / (Graph embedding on board)
72 Laplacian matrix and eigenvector (cont.) L = D A = Eigenvalues and eigenvectors (after normalization) of Laplacian matrix 254 λ / / 2 1/ / V 1/ / 2 1/ / / / (Graph embedding on board)
73 Spectral clustering algorithm Spectral Clustering 1. Compute the Laplacian L 2. Compute the first k eigenvectors u 1,..., u k of L 3. Let U R n k be the matrix containing the vectors u 1,..., u k as columns 4. For i = 1,..., n, let y i R k be the vector corresponding to the ith row of U. 5. Cluster the points {y 1,..., y n } in R k with kmeans algorithm into clusters C 1,..., C k 261
74 The connections When k = 2, we want to compute min S V RatioCut(S, T ) 271
75 The connections When k = 2, we want to compute min S V RatioCut(S, T ) We can prove that this is equivalent to min f T Lf s.t. f 1, f = 1, and f i is defined as S V f i = 1 n T S if v i S 1 n S T if v i T = V \S 272
76 The connections When k = 2, we want to compute min S V RatioCut(S, T ) We can prove that this is equivalent to min f T Lf s.t. f 1, f = 1, and f i is defined as S V f i = 1 n T S if v i S 1 n S T if v i T = V \S This is NPhard. And we solve the following relaxed optimization problem instead. min f R n f T Lf s.t. f 1, f = The best f is the second smallest eigenvalue of L.
77 281 Clustering Social Network
78 Social network and communities Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. 291
79 Social network and communities Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. networks 292
80 Social network and communities Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. networks Collaboration networks: E.g., two people publish paper together are connected. Communities: authors working on a particular topic (Wikipedia articles), people working on the same project in Google. 293
81 Social network and communities 294 Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. networks Collaboration networks: E.g., two people publish paper together are connected. Communities: authors working on a particular topic (Wikipedia articles), people working on the same project in Google. Goal: To find the communities. Similar to normal clustering but we sometimes allow overlaps (will not discuss here).
82 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied 301
83 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied Hierarchical clustering: may combine c, e first. 302
84 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied Hierarchical clustering: may combine c, e first. Assignmentbased clustering, e.g., kmean: if two random seeds are chosen to be b, e, then c may be assigned to e 303
85 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied Hierarchical clustering: may combine c, e first. Assignmentbased clustering, e.g., kmean: if two random seeds are chosen to be b, e, then c may be assigned to e Spectral clustering: may work. Try it. 304
86 Betweenness A new distance measure in social network: Betweenness Betweenness(A, B): number of shortest paths that use edge (A, B). 311
87 Betweenness A new distance measure in social network: Betweenness Betweenness(A, B): number of shortest paths that use edge (A, B). Intuition: If to get between two communities you need to take this edge, its betweenness score will be high. An edge with a high betweenness may be a facilitator edge, not a community edge. 312
88 GirvanNewman Algorithm GirvanNewman Algorithm 1. For each v V (a) Run BFS on v to build a directed acyclic graph (DAG) on the entire graph. Give 1 credit to each node except the root. (b) Walk back up the DAG and add a credit to each edge. When paths merge, the credits adds up, and when paths split, the credit splits as well. 2. The final score of each edge is the summation of all credits of the n runs, divided by Remove edges with high betweenness, and the remaining connected components are communities. Example (on board) 321
89 GirvanNewman Algorithm GirvanNewman Algorithm For each v V (a) Run BFS on v to build a directed acyclic graph (DAG) on the entire graph. Give 1 credit to each node except the root. (b) Walk back up the DAG and add a credit to each edge. When paths merge, the credits adds up, and when paths split, the credit splits as well. 2. The final score of each edge is the summation of all credits of the n runs, divided by Remove edges with high betweenness, and the remaining connected components are communities. Example (on board) What s the running time? Can we speed it?
90 331 What do the real graphs look like?
91 Powerlaw distributions Powerlaw distributions A nonnegative random variable X is said to have a power law distribution if Pr[X = x] cx α, for constants c > 0 and α > Zipf s Law: states that the frequency of the j the most common word in English (or other common languages) is proportional to j 1 A city grows in proportion to its current size as a result of people having children. Price studied the network of citations between scientific papers and found that the in degrees (number of times a paper has been cited) have power law distributions. WWW,... the rich get richer, heavy tails
92 Powerlaw graphs 351
93 Preferential attachment 361 BarabasiAlbert model When a new node is added to the network at each time t N. 1. With a probability p [0, 1], this new node connects to m existing nodes uniformly at random. 2. With a probability 1 p, this new node connects to m existing nodes with a probability proportional to the degree of node which it will be connected to. After some math, we get m(k) k β, where m(k) is the number of nodes whose degree is k, and β = 3 p 1 p. (on board, for p = 0)
94 Thank you! Some slides are based on the MMDS book and Jeff Phillips lecture notes 371
Part 2: Community Detection
Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection  Social networks 
More informationDATA ANALYSIS II. Matrix Algorithms
DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where
More informationLecture # 24: Clustering
COMPSCI 634: Geometric Algorithms April 10, 2014 Lecturer: Pankaj Agarwal Lecture # 24: Clustering Scribe: Nat Kell 1 Overview In this lecture, we briefly survey problems in geometric clustering. We define
More informationMining SocialNetwork Graphs
342 Chapter 10 Mining SocialNetwork Graphs There is much information to be gained by analyzing the largescale data that is derived from social networks. The bestknown example of a social network is
More informationWhy graph clustering is useful?
Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as any other clustering Identification of communities in social networks Webpage clustering for better data management
More informationParallel Graph Algorithms (Chapter 10)
Parallel Graph Algorithms (Chapter 10) Vivek Sarkar Department of Computer Science Rice University vsarkar@cs.rice.edu COMP 422 Lecture 24 10 April 2008 Schedule for rest of semester 4/11/08  Deadline
More informationCSE 100 Minimum Spanning Trees Unionfind
CSE 100 Minimum Spanning Trees Unionfind Announcements Midterm grades posted on gradesource Collect your paper at the end of class or during the times mentioned on Piazza. Regrades Submit your regrade
More informationChapter 11. Approximation Algorithms. Slides by Kevin Wayne PearsonAddison Wesley. All rights reserved.
Chapter 11 Approximation Algorithms Slides by Kevin Wayne. Copyright @ 2005 PearsonAddison Wesley. All rights reserved. 1 Approximation Algorithms Q. Suppose I need to solve an NPhard problem. What should
More informationSemisupervised learning
Semisupervised learning Learning from both labeled and unlabeled data Semisupervised learning Learning from both labeled and unlabeled data Motivation: labeled data may be hard/expensive to get, but
More informationPart II: Web Content Mining Chapter 3: Clustering
Part II: Web Content Mining Chapter 3: Clustering Learning by Example and Clustering Hierarchical Agglomerative Clustering KMeans Clustering ProbabilityBased Clustering Collaborative Filtering Slides
More informationPrototype Methods: KMeans
Prototype Methods: KMeans Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Prototype Methods Essentially modelfree. Not useful for understanding the nature of the
More informationSignal Processing over Graphs: Distributed Optimization and BioInspired Mechanisms. Sergio Barbarossa
Signal Processing over Graphs: Distributed Optimization and BioInspired Mechanisms Sergio Barbarossa 1 Overall summary 1. Algebraic graph theory 2. Signal Processing over Graphs 3. Distributed OpFmizaFon
More informationLecture 7,8 (Sep 22 & 24, 2015):Facility Location, KCenter
,8 CMPUT 675: Approximation Algorithms Fall 2015 Lecture 7,8 (Sep 22 & 24, 2015):Facility Location, KCenter Lecturer: Mohammad R. Salavatipour Scribe: Based on older notes 7.1 Uncapacitated Facility Location
More informationNeural Networks Lesson 5  Cluster Analysis
Neural Networks Lesson 5  Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt.  Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29
More information! Solve problem to optimality. ! Solve problem in polytime. ! Solve arbitrary instances of the problem. !approximation algorithm.
Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NPhard problem What should I do? A Theory says you're unlikely to find a polytime algorithm Must sacrifice one of
More information11 A: Algorithm Design Techniques
11 A: Algorithm Design Techniques CS1102S: Data Structures and Algorithms Martin Henz March 31, 2010 Generated on Tuesday 6 th April, 2010, 14:22 CS1102S: Data Structures and Algorithms 11 A: Algorithm
More information! Solve problem to optimality. ! Solve problem in polytime. ! Solve arbitrary instances of the problem. #approximation algorithm.
Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NPhard problem What should I do? A Theory says you're unlikely to find a polytime algorithm Must sacrifice one of three
More informationEdge Separators :Algorithms in the Real World. Compared with Mincut. Vertex Separators
Edge Separators 9.:Algorithms in the Real World Graph Separators Introduction Applications An edge separator : a set of edges E E which partitions V into V and V Criteria: V, V balanced E is small V E
More informationLecture 13: Clustering and Kmeans
Lecture 13: Clustering and Kmeans CSE 6040 Fall 2014 CSE 6040 Kmeans 1 / 22 What is clustering? Unsupervised learning: Given an unlabeled data set (no prior knowledge), how can we find structures/patterns
More informationSpectral graph theory
Spectral graph theory Uri Feige January 2010 1 Background With every graph (or digraph) one can associate several different matrices. We have already seen the vertexedge incidence matrix, the Laplacian
More informationBIRCH: Balanced Iterative Reducing and Clustering using Hierarchies
T61.6020 Popular Algorithms in Data Mining and Machine Learning BIRCH: Balanced Iterative Reducing and Clustering using Hierarchies Sami Virpioja Adaptive Informatics Research Centre Helsinki University
More informationCS 598: Communication Cost Analysis of Algorithms Lecture 12: Bitonic sort revisited and singlesource shortest path graph algorithms
CS 598: Communication Cost Analysis of Algorithms Lecture 12: Bitonic sort revisited and singlesource shortest path graph algorithms Edgar Solomonik University of Illinois at UrbanaChampaign October
More informationGeneralizing Vertex Cover, an Introduction to LPs
CS5234: Combinatorial and Graph Algorithms Lecture 2 Generalizing Vertex Cover, an Introduction to LPs Lecturer: Seth Gilbert August 19, 2014 Abstract Today we consider two generalizations of vertex cover:
More informationChapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling
Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NPhard problem. What should I do? A. Theory says you're unlikely to find a polytime algorithm. Must sacrifice one
More informationData Mining Cluster Analysis: Advanced Concepts and Algorithms. ref. Chapter 9. Introduction to Data Mining
Data Mining Cluster Analysis: Advanced Concepts and Algorithms ref. Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar 1 Outline Prototypebased Fuzzy cmeans Mixture Model Clustering Densitybased
More informationQuiz 2 Solutions. Solution: FALSE. If the graph contains a negativeweight cycle, then no shortest path exists.
Introduction to Algorithms April 14, 2010 Massachusetts Institute of Technology 6.006 Spring 2010 Professors Piotr Indyk and David Karger Quiz 2 Solutions Quiz 2 Solutions Problem 1. True or False [30
More information13.1 SemiDefinite Programming
CS787: Advanced Algorithms Lecture 13: Semidefinite programming In this lecture we will study an extension of linear programming, namely semidefinite programming. Semidefinite programs are much more
More informationClustering (K Means)
10601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Clustering (K Means) Clustering Readings: Murphy 25.5 Bishop 12.1, 12.3 HTF 14.3.0
More informationApplied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
More informationApproximation Algorithms. Scheduling. Approximation algorithms. Scheduling jobs on a single machine
Approximation algorithms Approximation Algorithms Fast. Cheap. Reliable. Choose two. NPhard problems: choose 2 of optimal polynomial time all instances Approximation algorithms. Tradeoff between time
More informationClustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016
Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with
More informationClustering: Partition Clustering. Wednesday, October 2, 13
Clustering: Partition Clustering Lecture outline Distance/Similarity between data objects Data objects as geometric data points Clustering problems and algorithms Kmeans Kmedian Kcenter What is clustering?
More informationDominating Set. Chapter 7
Chapter 7 Dominating Set In this chapter we present another randomized algorithm that demonstrates the power of randomization to break symmetries. We study the problem of finding a small dominating set
More informationCSE 373 Analysis of Algorithms April 6, Solutions to HW 3
CSE 373 Analysis of Algorithms April 6, 2014 Solutions to HW 3 PROBLEM 1 [5 4] Let T be the BFStree of the graph G. For any e in G and e / T, we have to show that e is a cross edge. Suppose not, suppose
More informationChapter 23 Minimum Spanning Trees
Chapter 23 Kruskal Prim [Source: Cormen et al. textbook except where noted] This refers to a number of problems (many of practical importance) that can be reduced to the following abstraction: let G =
More informationDistance based clustering
// Distance based clustering Chapter ² ² Clustering Clustering is the art of finding groups in data (Kaufman and Rousseeuw, 99). What is a cluster? Group of objects separated from other clusters Means
More informationApplications of Spectral Matrix Theory in Computational Biology
Applications of Spectral Matrix Theory in Computational Biology Soheil Feizi Computer Science and Artificial Intelligence Laboratory Research Laboratory for Electronics Massachusetts Institute of Technology
More informationWellSeparated Pair Decomposition for the Unitdisk Graph Metric and its Applications
WellSeparated Pair Decomposition for the Unitdisk Graph Metric and its Applications Jie Gao Department of Computer Science Stanford University Joint work with Li Zhang Systems Research Center HewlettPackard
More informationData Mining. Cluster Analysis: Advanced Concepts and Algorithms
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototypebased clustering Densitybased clustering Graphbased
More informationUSING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE FREE NETWORKS AND SMALLWORLD NETWORKS
USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE FREE NETWORKS AND SMALLWORLD NETWORKS Natarajan Meghanathan Jackson State University, 1400 Lynch St, Jackson, MS, USA natarajan.meghanathan@jsums.edu
More informationGive an efficient greedy algorithm that finds an optimal vertex cover of a tree in linear time.
Give an efficient greedy algorithm that finds an optimal vertex cover of a tree in linear time. In a vertex cover we need to have at least one vertex for each edge. Every tree has at least two leaves,
More information6. If there is no improvement of the categories after several steps, then choose new seeds using another criterion (e.g. the objects near the edge of
Clustering Clustering is an unsupervised learning method: there is no target value (class label) to be predicted, the goal is finding common patterns or grouping similar examples. Differences between models/algorithms
More informationUSING SPECTRAL RADIUS RATIO FOR NODE DEGREE
USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF COMPLEX NETWORKS Natarajan Meghanathan Jackson State University, 1400 Lynch St, Jackson, MS, USA ABSTRACT In this paper, we show
More informationApproximation Algorithms
Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NPCompleteness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms
More information1 Shortest Paths. 1.1 Breadth First Search (BFS) CS 124 Section #3 Shortest Paths and MST s 2/13/2017
CS 4 Section # Shortest Paths and MST s //07 Shortest Paths There are types of shortest paths problems: Single source single destination Single source to all destinations All pairs shortest path In today
More informationImage Segmentation using kmeans clustering, EM and Normalized Cuts
Image Segmentation using kmeans clustering, EM and Normalized Cuts Suman Tatiraju Department of EECS University Of California  Irvine Irvine, CA 92612 statiraj@uci.edu Avi Mehta Department of EECS University
More informationPath Problems and Complexity Classes
Path Problems and omplexity lasses Henry Z. Lo July 19, 2014 1 Path Problems 1.1 Euler paths 1.1.1 escription Problem 1. Find a walk which crosses each of the seven bridges in figure 1 exactly once. The
More informationHierarchical vs. Partitional Clustering
Hierarchical vs. Partitional Clustering A distinction among different types of clusterings is whether the set of clusters is nested or unnested. A partitional clustering a simply a division of the set
More informationNimble Algorithms for Cloud Computing. Ravi Kannan, Santosh Vempala and David Woodruff
Nimble Algorithms for Cloud Computing Ravi Kannan, Santosh Vempala and David Woodruff Cloud computing Data is distributed arbitrarily on many servers Parallel algorithms: time Streaming algorithms: sublinear
More informationSPANNING TREES 11/3/16. Spanning trees. Dijkstra s algorithm using class SFdata. Backpointers. Backpointers
// Spanning trees What we do today: Talk about modifying an existing algorithm Calculating the shortest path in Dijkstra s algorithm Minimum spanning trees greedy algorithms (including Kruskal & Prim)
More informationOn the WorstCase Complexity of the kmeans Method
On the WorstCase Complexity of the kmeans Method Sergei Vassilvitskii David Arthur (Stanford University) Clustering Given npoints in split them into k similar groups. R d 2 Clustering Objectives Let
More informationMA2223: NORMED VECTOR SPACES
MA2223: NORMED VECTOR SPACES Contents 1. Normed vector spaces 2 1.1. Examples of normed vector spaces 3 1.2. Continuous linear operators 6 1.3. Equivalent norms 9 1.4. Matrix norms 11 1.5. Banach spaces
More informationSpectral Clustering. Aarti Singh. Machine Learning / Nov 22, Slides Courtesy: Eric Xing, M. Hein & U.V. Luxburg
Spectral Clustering Aarti Singh Machine Learning 10701/15781 Nov 22, 2010 Slides Courtesy: Eric Xing, M. Hein & U.V. Luxburg 1 Data Clustering Graph Clustering Goal: Given data points X1,, Xn and similarities
More informationprinceton university F 02 cos 597D: a theorist s toolkit Lecture 7: Markov Chains and Random Walks
princeton university F 02 cos 597D: a theorist s toolkit Lecture 7: Markov Chains and Random Walks Lecturer: Sanjeev Arora Scribe:Elena Nabieva 1 Basics A Markov chain is a discretetime stochastic process
More informationMining Social Network Graphs
Mining Social Network Graphs Debapriyo Majumdar Data Mining Fall 2014 Indian Statistical Institute Kolkata November 13, 17, 2014 Social Network No introduc+on required Really? We s7ll need to understand
More informationLecture 9. 1 Introduction. 2 Random Walks in Graphs. 1.1 How To Explore a Graph? CS621 Theory Gems October 17, 2012
CS62 Theory Gems October 7, 202 Lecture 9 Lecturer: Aleksander Mądry Scribes: Dorina Thanou, Xiaowen Dong Introduction Over the next couple of lectures, our focus will be on graphs. Graphs are one of
More information11. APPROXIMATION ALGORITHMS
Coping with NPcompleteness 11. APPROXIMATION ALGORITHMS load balancing center selection pricing method: vertex cover LP rounding: vertex cover generalized load balancing knapsack problem Q. Suppose I
More information(where C, X R and are column vectors)
CS 05: Algorithms (Grad) What is a Linear Programming Problem? A linear program (LP) is a minimization problem where we are asked to minimize a given linear function subject to one or more linear inequality
More informationGeometrical Segmentation of Point Cloud Data using Spectral Clustering
Geometrical Segmentation of Point Cloud Data using Spectral Clustering Sergey Alexandrov and Rainer Herpers University of Applied Sciences BonnRheinSieg 15th July 2014 1/29 Addressed Problem Given: a
More informationSocial Media Mining. Network Measures
Klout Measures and Metrics 22 Why Do We Need Measures? Who are the central figures (influential individuals) in the network? What interaction patterns are common in friends? Who are the likeminded users
More informationLecture More common geometric view of the simplex method
18.409 The Behavior of Algorithms in Practice 9 Apr. 2002 Lecture 14 Lecturer: Dan Spielman Scribe: Brian Sutton In this lecture, we study the worstcase complexity of the simplex method. We conclude by
More informationPTAS for Euclidean TSP
Sanjeev Arora George Mason University November 30, 2010 Outline 1 Traveling Salesman Problem Variants 2 3 Problem Description Intuition 4 Classical Formulation Traveling Salesman Problem Variants Traveling
More informationMarkov Chain Fundamentals (Contd.)
Markov Chain Fundamentals (Contd.) Recap from last lecture:  A random walk on a graph is a special type of Markov chain used for analyzing algorithms.  An irreducible and aperiodic Markov chain does
More informationCommunity Detection in Networks: The LeaderFollower Algorithm
Community Detection in Networks: The LeaderFollower Algorithm Devavrat Shah and Tauhid Zaman* Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology Cambridge,
More informationFunctional Analysis Review
9.520 Math Camp 2010 Functional Analysis Review Andre Wibisono February 8, 2010 These notes present a brief summary of some of the basic definitions from functional analysis that we will need in this class.
More informationClustering. Unsupervised learning Generating classes Distance/similarity measures Agglomerative methods Divisive methods. CS 5751 Machine Learning
Clustering Unsupervised learning Generating classes Distance/similarity measures Agglomerative methods Divisive methods Data Clustering 1 What is Clustering? Form of unsupervised learning  no information
More information6.042/18.062J Mathematics for Computer Science October 3, 2006 Tom Leighton and Ronitt Rubinfeld. Graph Theory III
6.04/8.06J Mathematics for Computer Science October 3, 006 Tom Leighton and Ronitt Rubinfeld Lecture Notes Graph Theory III Draft: please check back in a couple of days for a modified version of these
More informationCluster Analysis: Advanced Concepts
Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototypebased Fuzzy cmeans
More informationLecture 6: Approximation via LP Rounding
Lecture 6: Approximation via LP Rounding Let G = (V, E) be an (undirected) graph. A subset C V is called a vertex cover for G if for every edge (v i, v j ) E we have v i C or v j C (or both). In other
More informationThe kcenter problem February 14, 2007
Sanders/van Stee: Approximations und OnlineAlgorithmen 1 The kcenter problem February 14, 2007 Input is set of cities with intercity distances (G = (V,V V )) Select k cities to place warehouses Goal:
More informationPrivate Approximation of Clustering and Vertex Cover
Private Approximation of Clustering and Vertex Cover Amos Beimel, Renen Hallak, and Kobbi Nissim Department of Computer Science, BenGurion University of the Negev Abstract. Private approximation of search
More informationAlgorithmic Aspects of Big Data. Nikhil Bansal (TU Eindhoven)
Algorithmic Aspects of Big Data Nikhil Bansal (TU Eindhoven) Goal: Look from the lens of theoretical CS Theory of Algorithms: Past, present and future? Some cool ideas Some work we have done. Algorithm
More informationDATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS
DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDDLAB ISTI CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar
More informationCLASSIFICATION AND CLUSTERING. Anveshi Charuvaka
CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training
More informationCluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico
Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from
More informationCMPSCI611: Approximating MAXCUT Lecture 20
CMPSCI611: Approximating MAXCUT Lecture 20 For the next two lectures we ll be seeing examples of approximation algorithms for interesting NPhard problems. Today we consider MAXCUT, which we proved to
More informationVoronoi Diagrams Robust and Efficient implementation
Robust and Efficient implementation Master s Thesis Defense Binghamton University Department of Computer Science Thomas J. Watson School of Engineering and Applied Science May 5th, 2005 Motivation The
More informationClustering and Data Mining in R
Clustering and Data Mining in R Workshop Supplement Thomas Girke December 10, 2011 Introduction Data Preprocessing Data Transformations Distance Methods Cluster Linkage Hierarchical Clustering Approaches
More informationLecture 4 Online and streaming algorithms for clustering
CSE 291: Geometric algorithms Spring 2013 Lecture 4 Online and streaming algorithms for clustering 4.1 Online kclustering To the extent that clustering takes place in the brain, it happens in an online
More informationIntroduction to Segmentation
Lecture 2: Introduction to Segmentation Jonathan Krause 1 Goal Goal: Identify groups of pixels that go together image credit: Steve Seitz, Kristen Grauman 2 Types of Segmentation Semantic Segmentation:
More informationAlgorithmic Aspects of Big Data. Nikhil Bansal (TU Eindhoven)
Algorithmic Aspects of Big Data Nikhil Bansal (TU Eindhoven) Algorithm design Algorithm: Set of steps to solve a problem (by a computer) Studied since 1950 s. Given a problem: Find (i) best solution (ii)
More informationprinceton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora
princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of
More informationDynamic programming CS325
Dynamic programming CS325 Shortest path in DAG DAG: directed acyclic graph A DAG can be linearized: all the nodes arranged in a line and the edges only goes from left to right (Chapter 3.3.2) Consider
More informationARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING)
ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Preliminaries Classification and Clustering Applications
More informationCS261: A Second Course in Algorithms Lecture #13: Online Scheduling and Online Steiner Tree
CS6: A Second Course in Algorithms Lecture #3: Online Scheduling and Online Steiner Tree Tim Roughgarden February 6, 06 Preamble Last week we began our study of online algorithms with the multiplicative
More informationProblem Set 1 Solutions
Design and Analysis of Algorithms February 15, 2015 Massachusetts Institute of Technology 6.046J/18.410J Profs. Erik Demaine, Srini Devadas, and Nancy Lynch Problem Set 1 Solutions Problem Set 1 Solutions
More informationA Summary: Fastest Mixing Markov Chain on a Graph
OR 79K: Convex Optimization and Interior Point Methods December 2007 A Summary: Fastest Mixing Markov Chain on a Graph Prepared for: Dr.Kartik Sivaramakrishnan Prepared by: Susan Osborne Introduction.
More informationLecture and notes by: Luke Postle, Linji Yang Nov 29th, Undirected STConnectivity in Log Space
Lecture and notes by: Luke Postle, Linji Yang Nov 9th, 007 Undirected STConnectivity in Log Space 1 History The problem of interest in this lecture is undirected st connectivity. For a given graph G and
More informationGraph Theory and Metric
Chapter 2 Graph Theory and Metric Dimension 2.1 Science and Engineering Multi discipline teams and multi discipline areas are words that now a days seem to be important in the scientific research. Indeed
More informationB490 Mining the Big Data. 0 Introduction
B490 Mining the Big Data 0 Introduction Qin Zhang 11 Data Mining What is Data Mining? A definition : Discovery of useful, possibly unexpected, patterns in data. 21 Data Mining What is Data Mining? A
More informationCS 583: Approximation Algorithms Lecture date: February 3, 2016 Instructor: Chandra Chekuri
CS 583: Approximation Algorithms Lecture date: February 3, 2016 Instructor: Chandra Chekuri Scribe: CC Notes updated from Spring 2011. In the previous lecture we discussed the Knapsack problem. In this
More informationClustering. Clustering. Clustering Vs. Classification. What is Cluster Analysis? General Applications of Clustering
Clustering Clustering Recall Supervised Vs. Unsupervised Learning Unsupervised Learning What have we studied? Clustering is another form of unsupervised Learning Clustering Vs. Classification Similarity
More informationGraph Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar
Graph Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text Introduction to Parallel Computing, Addison Wesley, 3. Topic Overview Definitions and Representation Minimum
More informationLecture 11: Constructing Expanders, Undirected Connectivity in log space
Lecture 11: Constructing Expanders, Undirected Connectivity in log space opics in Complexity heory and Pseudorandomness (Spring 01) Rutgers University Swastik Kopparty Scribes: Abdul Basit, Erick Chastain
More informationChapter 6: Random rounding of semidefinite programs
Chapter 6: Random rounding of semidefinite programs Section 6.1: Semidefinite Programming Section 6.: Finding large cuts Extra (not in book): The max sat problem. Section 6.5: Coloring 3colorabale graphs.
More informationGraph. Graph Theory. Adjacent, Nonadjacent, Incident. Degree of Graph
Graph Graph Theory Peter Lo A Graph (or undirected graph) G consists of a set V of vertices (or nodes) and a set E of edges (or arcs) such that each edge e E is associated with an unordered pair of vertices.
More informationSimilarity and Diagonalization. Similar Matrices
MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that
More informationBindel, Fall 2012 Matrix Computations (CS 6210) Week 8: Friday, Oct 12
Why eigenvalues? Week 8: Friday, Oct 12 I spend a lot of time thinking about eigenvalue problems. In part, this is because I look for problems that can be solved via eigenvalues. But I might have fewer
More informationData Analysis and Manifold Learning Lecture 1: Introduction to spectral and graphbased methods
Data Analysis and Manifold Learning Lecture 1: Introduction to spectral and graphbased methods Radu Horaud INRIA Grenoble RhoneAlpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Introduction
More information