# B490 Mining the Big Data. 2 Clustering

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 B490 Mining the Big Data 2 Clustering Qin Zhang 1-1

2 Motivations Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Cluster topic/articles from the web by same story. Google image Many others

3 Motivations 2-2 Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Cluster topic/articles from the web by same story. Google image Many others... Given a cloud of data points we want to understand their structures Community discovery in social networks

4 Motivations 2-3 Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Cluster topic/articles from the web by same story. Google image Many others... Given a cloud of data points we want to understand their structures Community discovery in social networks

5 The plan Clustering is an extremely broad and ill-defined topic. There are many many variants; we will not try to cover all of them. Instead we will focus on three broad classes of algorithms Hierachical Clustering Assignment-based Clustering Spectural Clustering 3-1

6 The plan Clustering is an extremely broad and ill-defined topic. There are many many variants; we will not try to cover all of them. Instead we will focus on three broad classes of algorithms Hierachical Clustering Assignment-based Clustering Spectural Clustering There is a saying When data is easily cluster-able, most clustering algorithms work quickly and well. When is not easily cluster-able, then no algorithm will find good clusters. 3-2

7 4-1 Hierachical Clustering

8 Hard clustering Problem: Start with a set X R d, and a metric d : R d R d R +. A cluster C is a subset of X. A clustering is a partition ρ(x ) = {C 1, C 2,..., C k }, where 1. Each C i X. 2. Each pair C i C j =. 3. Each k i=1 C i = X. 5-1

9 Hard clustering Problem: Start with a set X R d, and a metric d : R d R d R +. A cluster C is a subset of X. A clustering is a partition ρ(x ) = {C 1, C 2,..., C k }, where 1. Each C i X. 2. Each pair C i C j =. 3. Each k i=1 C i = X. Goal: 1. (close intra) For each C ρ(x ) for all x, x C, d(x, x ) is small. 2. (far inter) For each C i, C j ρ(x ) (i j), for most x i C i and x j C j, d(x i, x j ) is large. 5-2

10 Hierarchical clustering Hierarchical Clustering 1. Each x i X forms a cluster C i 2. While exists two clusters close enough (a) Find the closest two clusters C i, C j. (b) Merge C i, C j into a single cluster 6-1

11 Hierarchical clustering Hierarchical Clustering 1. Each x i X forms a cluster C i 2. While exists two clusters close enough (a) Find the closest two clusters C i, C j. (b) Merge C i, C j into a single cluster Two things to define 1. Define a distance between clusters. 2. Define close enough 6-2

12 Definition of distance between clusters Distance between center of clusters. What does center mean? 7-1

13 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. 7-2

14 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the center-point (like a high-dimensional median) 7-3

15 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the center-point (like a high-dimensional median) center of the MEB (minimum enclosing ball) 7-4

16 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the center-point (like a high-dimensional median) center of the MEB (minimum enclosing ball) some (random) representative point 7-5

17 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the center-point (like a high-dimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points 7-6

18 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the center-point (like a high-dimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points Distance between furthest pair of points 7-7

19 Definition of distance between clusters Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the center-point (like a high-dimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points Distance between furthest pair of points Average distance between all pairs of points in different clusters 7-8

20 Definition of distance between clusters 7-9 Distance between center of clusters. What does center mean? the mean (average) point. Called the centroid. the center-point (like a high-dimensional median) center of the MEB (minimum enclosing ball) some (random) representative point Distance between closest pair of points Distance between furthest pair of points Average distance between all pairs of points in different clusters Radius of minimum enclosing ball of joined cluster

21 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. 8-1

22 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. 8-2

23 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d 8-3

24 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d If the joint density decrease too much from single cluster density. Variations of this are called the elbow technique. 8-4

25 Definition of close enough If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d If the joint density decrease too much from single cluster density. Variations of this are called the elbow technique. When the number of clusters is k (for some magical value k). 8-5

26 Definition of close enough 8-6 If information is known about the data, perhaps some absolute value τ can be given. If the { diameter, or radius of MEB, or average distance from center point } is beneath some threshold. This fixes the scale of a cluster, which could be good or bad. If the density is beneath some threshold; where density is # points/volume or # points/radius d If the joint density decrease too much from single cluster density. Variations of this are called the elbow technique. When the number of clusters is k (for some magical value k). Example (on board): Distance is distance between centroids. Stop when there is 1 cluster left.

27 Assignment-based Clustering (k-center, k-mean, k-median) 9-1

28 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). 10-1

29 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). The k-center cluster problem is to find the set C of k clusters (often, but not always as a subset of X ) to minimize max x X d(φ C (x), x). 10-2

30 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). The k-center cluster problem is to find the set C of k clusters (often, but not always as a subset of X ) to minimize max x X d(φ C (x), x). The k-means cluster problem: minimize x X d(φ C (x), x)

31 Assignment based clustering Definitions For a set X, and distance d : X X R +, the output is a set of points C = {c 1,..., c k }, which implicitly defines a set of clusters where φ C (x) = arg min c C d(x, c). The k-center cluster problem is to find the set C of k clusters (often, but not always as a subset of X ) to minimize max x X d(φ C (x), x). The k-means cluster problem: minimize x X d(φ C (x), x) 2 The k-median cluster problem: minimize x X d(φ C (x), x) 10-4

32 Gonzalez Algorithm Gonzalez Algorithm A simple greedy algorithm for k-center 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Set c i = arg max x X d(x, φ Si 1 (x)) (b) Let S i = {c 1,..., c i } 11-1

33 Gonzalez Algorithm Gonzalez Algorithm A simple greedy algorithm for k-center 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Set c i = arg max x X d(x, φ Si 1 (x)) (b) Let S i = {c 1,..., c i } In the worst case, it gives a 2-approximation to the optimal solution. 11-2

34 Gonzalez Algorithm Gonzalez Algorithm A simple greedy algorithm for k-center 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Set c i = arg max x X d(x, φ Si 1 (x)) (b) Let S i = {c 1,..., c i } In the worst case, it gives a 2-approximation to the optimal solution. Analysis (on board) 11-3

35 Parallel Guessing Algorithm Parallel Guessing Algorithm for k-center Run for R = (1 + ɛ/2), (1 + ɛ/2) 2,..., 1. We pick an arbitrary point as the first center, C = {c 1 }. 2. For each point p P compute r p = min c C d(p, c) and test whether r p > R. If it is, then set C = C {p}. 3. Abort at C > k Pick the minimum R that does not abort. 12-1

36 Parallel Guessing Algorithm Parallel Guessing Algorithm for k-center Run for R = (1 + ɛ/2), (1 + ɛ/2) 2,..., 1. We pick an arbitrary point as the first center, C = {c 1 }. 2. For each point p P compute r p = min c C d(p, c) and test whether r p > R. If it is, then set C = C {p}. 3. Abort at C > k Pick the minimum R that does not abort. In the worst case, it gives a (2 + ɛ)-approximation to the optimal solution (on board). 12-2

37 Parallel Guessing Algorithm Parallel Guessing Algorithm for k-center Run for R = (1 + ɛ/2), (1 + ɛ/2) 2,..., 1. We pick an arbitrary point as the first center, C = {c 1 }. 2. For each point p P compute r p = min c C d(p, c) and test whether r p > R. If it is, then set C = C {p}. 3. Abort at C > k Pick the minimum R that does not abort. In the worst case, it gives a (2 + ɛ)-approximation to the optimal solution (on board). Can be implemented as a streaming algorithm performs a linear scan of the data, uses space Õ(k) and time Õ(nk). 12-3

38 Lloyd s Algorithm Lloyd s Algorithm The well-known algorithm for k-means 1. Choose k points C X (arbitrarily?) 2. Repeat (a) For all x X, find φ C (x) (closest center c C to x) (b) For all i [k], let c i = average{x X φ C (x) = c i } 3. until the set C is unchanged Kmeans_Kmedoids.html 13-1

39 Lloyd s Algorithm Lloyd s Algorithm The well-known algorithm for k-means 1. Choose k points C X (arbitrarily?) 2. Repeat (a) For all x X, find φ C (x) (closest center c C to x) (b) For all i [k], let c i = average{x X φ C (x) = c i } 3. until the set C is unchanged Kmeans_Kmedoids.html If the main loop has R rounds, then the running time is O(Rnk) 13-2

40 Lloyd s Algorithm 13-3 Lloyd s Algorithm The well-known algorithm for k-means 1. Choose k points C X (arbitrarily?) 2. Repeat (a) For all x X, find φ C (x) (closest center c C to x) (b) For all i [k], let c i = average{x X φ C (x) = c i } 3. until the set C is unchanged Kmeans_Kmedoids.html If the main loop has R rounds, then the running time is O(Rnk) But 1. What is R? Converge in finite number of steps? 2. How do we initialize C? 3. How accurate is this algorithm?

41 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) 14-1

42 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. 14-2

43 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. Smoothed analysis: if data perturbed randomly slightly, then R = O(n 35 k 34 d 8 ). This is polynomial, but still ridiculous. 14-3

44 Value of R It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. Smoothed analysis: if data perturbed randomly slightly, then R = O(n 35 k 34 d 8 ). This is polynomial, but still ridiculous. If all points are on a grid of length M, then R = O(dn 4 M 2 ). But thats still way too big. 14-4

45 Value of R 14-5 It is finite. The cost x X d(x, φ C (x)) 2 is always decreasing, and there are a finite number of possible distinct cluster centers. But it could be exponential in k and d (the dimension when Euclidean distance used): n O(kd) (number of voronoi cell) However, usually R = 10 is fine. Smoothed analysis: if data perturbed randomly slightly, then R = O(n 35 k 34 d 8 ). This is polynomial, but still ridiculous. If all points are on a grid of length M, then R = O(dn 4 M 2 ). But thats still way too big. Conclusion: There are crazy special cases that can take a long time, but usually it works.

46 Initialize C Random set of points. By coupon collectors, we know that we need about O(k log k) to get one in each cluster, given that each cluster contains equal number of points. We can later reduce to k clusters, by merging extra clusters. 15-1

47 Initialize C Random set of points. By coupon collectors, we know that we need about O(k log k) to get one in each cluster, given that each cluster contains equal number of points. We can later reduce to k clusters, by merging extra clusters. Randomly partition X = {X 1, X 2,..., X k } and take c i = average(x i ). This biases towards center of X. 15-2

48 Initialize C Random set of points. By coupon collectors, we know that we need about O(k log k) to get one in each cluster, given that each cluster contains equal number of points. We can later reduce to k clusters, by merging extra clusters. Randomly partition X = {X 1, X 2,..., X k } and take c i = average(x i ). This biases towards center of X. Gonzalez algorithm (for k-center). This biases too much to outlier points. 15-3

49 Accuracy Can be arbitrarily bad. (Give an example?) 16-1

50 Accuracy Can be arbitrarily bad. (Give an example?) 4 vertices of a rectangle (width height) 16-2

51 Accuracy Can be arbitrarily bad. (Give an example?) Theory algorithm: Gets (1 + ɛ)-approximation for k-means in 2 (k/ɛ)o(1) nd time. (Kumar, Sabharwal, Sen 2004) 16-3

52 Accuracy 16-4 Can be arbitrarily bad. (Give an example?) Theory algorithm: Gets (1 + ɛ)-approximation for k-means in 2 (k/ɛ)o(1) nd time. (Kumar, Sabharwal, Sen 2004) The following k-means++ is O(log k)-approximation a. a Ref: k-means++: The Advantages of Careful Seeding k-means++ 1. Choose c 1 X arbitrarily. Let S 1 = {c 1 } 2. For i = 2 to k do (a) Choose c i from X with probability proportional to d(x, φ Si 1 (x)) 2 (b) Let S i = {c 1,..., c i }

53 Other issues The key step that makes Lloyds algorithm so cool is average{x X } = arg min c x 2 c R d 2. x X But it only works for distance function d(x, c) = x c 2 Is effected by outliers more than k-median clustering. 17-1

54 Local Search Algorithm Local Search Algorithm Designed for k-median 1. Choose K centers at random from X. Call this set C. 2. Repeat For each u in X and c in C, compute Cost(C {c} + {u}). Find (u, c) for which this cost is smallest, and replace C with C {c} + {u}. 3. Stop when Cost(C) does not change significantly 18-1

55 Local Search Algorithm Local Search Algorithm Designed for k-median 1. Choose K centers at random from X. Call this set C. 2. Repeat For each u in X and c in C, compute Cost(C {c} + {u}). Find (u, c) for which this cost is smallest, and replace C with C {c} + {u}. 3. Stop when Cost(C) does not change significantly Can be used to design a 5-approximation algorithm a. a Ref: Local Search Heuristics for k-median and Facility Location Problems Q: How can you speed up the swap operation by avoiding searching over all (u, c) pairs? 18-2

56 19-1 Spectural Clustering

57 Top-down clustering Top-down Clustering Framework 1. Find the best cut of the data into two pieces. 2. Recur on both pieces until that data should not be split anymore. 20-1

58 Top-down clustering Top-down Clustering Framework 1. Find the best cut of the data into two pieces. 2. Recur on both pieces until that data should not be split anymore. What is the best way to partition the data? This is a big question. Let s first talk about graphs. 20-2

59 Graphs Graph: G = (V, E) is defined by a set of vertices V = {v 1, v 2,..., v n } and a set of edges E = {e 1, e 2,..., e m } where each edge e j is an unordered (or ordered in a directed graph) pair of edges e j = {v i, v i } (or e j = (v i, v i )). Example: e g a b c d f h 21-1

60 Graphs Graph: G = (V, E) is defined by a set of vertices V = {v 1, v 2,..., v n } and a set of edges E = {e 1, e 2,..., e m } where each edge e j is an unordered (or ordered in a directed graph) pair of edges e j = {v i, v i } (or e j = (v i, v i )). Example: e c a d b f g h Adjacent list representation A =

61 From point sets to graphs ɛ-neighborhood graph: Connect all points whose pairwise distance are smaller than ɛ. An unweighted graph 22-1

62 From point sets to graphs ɛ-neighborhood graph: Connect all points whose pairwise distance are smaller than ɛ. An unweighted graph k-nearest neighbor graphs: Connect v i with v j if v j is among the k-nearest neighbors of v i. Naturally a directed graph. Two ways to make it undirected: (1) neglect the direction; (2) mutual k-nearest neighbor. 22-2

63 From point sets to graphs ɛ-neighborhood graph: Connect all points whose pairwise distance are smaller than ɛ. An unweighted graph k-nearest neighbor graphs: Connect v i with v j if v j is among the k-nearest neighbors of v i. Naturally a directed graph. Two ways to make it undirected: (1) neglect the direction; (2) mutual k-nearest neighbor. The fully connected graph: connect all points with finite distances with each other, and weight all edges by d(v i, v j ), where d(v i, v j ) is the distance between v i and v j. 22-3

64 A good cut a b c d e f g h Rule of the thumb 1. Many edges in a cluster. Volume if a cluster is Vol(S) = the number of edges with at least one vertex in V. 2. Few edges between clusters. Cut between two clusters S, T is Cut(S, T ) = the number of edges with one vertex in S and the other in T. 23-1

65 A good cut a b c d e f g h Rule of the thumb 1. Many edges in a cluster. Volume if a cluster is Vol(S) = the number of edges with at least one vertex in V. 2. Few edges between clusters. Cut between two clusters S, T is Cut(S, T ) = the number of edges with one vertex in S and the other in T Ratio cut: RatioCut(S, T ) = Cut(S,T ) S + Cut(S,T ) T Normalized cut: NCut(S, T ) = Cut(S,T ) Vol(S) + Cut(S,T ) Vol(T )

66 A good cut a b c Rule of the thumb 1. Many edges in a cluster. 2. Few edges between clusters. d e f g h Can be generalize to general k Volume if a cluster is Vol(S) = the number of edges with at least one vertex in V. Cut between two clusters S, T is Cut(S, T ) = the number of edges with one vertex in S and the other in T Ratio cut: RatioCut(S, T ) = Cut(S,T ) S + Cut(S,T ) T Normalized cut: NCut(S, T ) = Cut(S,T ) Vol(S) + Cut(S,T ) Vol(T )

67 Laplacian matrix and eigenvector 1. Adjacent matrix of the graph A 2. Degree (diagonal) matrix D 3. Laplacian matrix L = D A 4. Eigenvector of a matrix M is the vector v such that Mv = λv, where λ is a scalar, called the corresponding eigenvalue. a b c e d f g h 24-1

68 Laplacian matrix and eigenvector 1. Adjacent matrix of the graph A 2. Degree (diagonal) matrix D 3. Laplacian matrix L = D A 4. Eigenvector of a matrix M is the vector v such that Mv = λv, where λ is a scalar, called the corresponding eigenvalue. a b c e d f g h Adjacent list matrix A = Degree matrix D =

69 Laplacian matrix and eigenvector (cont.) L = D A =

70 Laplacian matrix and eigenvector (cont.) L = D A = Eigenvalues and eigenvectors (after normalization) of Laplacian matrix 25-2 λ / / 2 1/ / V 1/ / 2 1/ / / /

71 Laplacian matrix and eigenvector (cont.) L = D A = Eigenvalues and eigenvectors (after normalization) of Laplacian matrix 25-3 λ / / 2 1/ / V 1/ / 2 1/ / / / (Graph embedding on board)

72 Laplacian matrix and eigenvector (cont.) L = D A = Eigenvalues and eigenvectors (after normalization) of Laplacian matrix 25-4 λ / / 2 1/ / V 1/ / 2 1/ / / / (Graph embedding on board)

73 Spectral clustering algorithm Spectral Clustering 1. Compute the Laplacian L 2. Compute the first k eigenvectors u 1,..., u k of L 3. Let U R n k be the matrix containing the vectors u 1,..., u k as columns 4. For i = 1,..., n, let y i R k be the vector corresponding to the i-th row of U. 5. Cluster the points {y 1,..., y n } in R k with k-means algorithm into clusters C 1,..., C k 26-1

74 The connections When k = 2, we want to compute min S V RatioCut(S, T ) 27-1

75 The connections When k = 2, we want to compute min S V RatioCut(S, T ) We can prove that this is equivalent to min f T Lf s.t. f 1, f = 1, and f i is defined as S V f i = 1 n T S if v i S 1 n S T if v i T = V \S 27-2

76 The connections When k = 2, we want to compute min S V RatioCut(S, T ) We can prove that this is equivalent to min f T Lf s.t. f 1, f = 1, and f i is defined as S V f i = 1 n T S if v i S 1 n S T if v i T = V \S This is NP-hard. And we solve the following relaxed optimization problem instead. min f R n f T Lf s.t. f 1, f = The best f is the second smallest eigenvalue of L.

77 28-1 Clustering Social Network

78 Social network and communities Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. 29-1

79 Social network and communities Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. networks 29-2

80 Social network and communities Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. networks Collaboration networks: E.g., two people publish paper together are connected. Communities: authors working on a particular topic (Wikipedia articles), people working on the same project in Google. 29-3

81 Social network and communities 29-4 Telephone networks Communities: groups of people that communicate frequently, e.g., groups of friends, members of a club, etc. networks Collaboration networks: E.g., two people publish paper together are connected. Communities: authors working on a particular topic (Wikipedia articles), people working on the same project in Google. Goal: To find the communities. Similar to normal clustering but we sometimes allow overlaps (will not discuss here).

82 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied 30-1

83 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied Hierarchical clustering: may combine c, e first. 30-2

84 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied Hierarchical clustering: may combine c, e first. Assignment-based clustering, e.g., k-mean: if two random seeds are chosen to be b, e, then c may be assigned to e 30-3

85 Previous clustering algorithms a b c d e f g Difficulty: Hard to define the distance. E.g., if we assign 1 to (x, y) if x, y are friends and 0 otherwise, then the triangle inequality is not satisfied Hierarchical clustering: may combine c, e first. Assignment-based clustering, e.g., k-mean: if two random seeds are chosen to be b, e, then c may be assigned to e Spectral clustering: may work. Try it. 30-4

86 Betweenness A new distance measure in social network: Betweenness Betweenness(A, B): number of shortest paths that use edge (A, B). 31-1

87 Betweenness A new distance measure in social network: Betweenness Betweenness(A, B): number of shortest paths that use edge (A, B). Intuition: If to get between two communities you need to take this edge, its betweenness score will be high. An edge with a high betweenness may be a facilitator edge, not a community edge. 31-2

88 Girvan-Newman Algorithm Girvan-Newman Algorithm 1. For each v V (a) Run BFS on v to build a directed acyclic graph (DAG) on the entire graph. Give 1 credit to each node except the root. (b) Walk back up the DAG and add a credit to each edge. When paths merge, the credits adds up, and when paths split, the credit splits as well. 2. The final score of each edge is the summation of all credits of the n runs, divided by Remove edges with high betweenness, and the remaining connected components are communities. Example (on board) 32-1

89 Girvan-Newman Algorithm Girvan-Newman Algorithm For each v V (a) Run BFS on v to build a directed acyclic graph (DAG) on the entire graph. Give 1 credit to each node except the root. (b) Walk back up the DAG and add a credit to each edge. When paths merge, the credits adds up, and when paths split, the credit splits as well. 2. The final score of each edge is the summation of all credits of the n runs, divided by Remove edges with high betweenness, and the remaining connected components are communities. Example (on board) What s the running time? Can we speed it?

90 33-1 What do the real graphs look like?

91 Power-law distributions Power-law distributions A nonnegative random variable X is said to have a power law distribution if Pr[X = x] cx α, for constants c > 0 and α > Zipf s Law: states that the frequency of the j the most common word in English (or other common languages) is proportional to j 1 A city grows in proportion to its current size as a result of people having children. Price studied the network of citations between scientific papers and found that the in degrees (number of times a paper has been cited) have power law distributions. WWW,... the rich get richer, heavy tails

92 Power-law graphs 35-1

93 Preferential attachment 36-1 Barabasi-Albert model When a new node is added to the network at each time t N. 1. With a probability p [0, 1], this new node connects to m existing nodes uniformly at random. 2. With a probability 1 p, this new node connects to m existing nodes with a probability proportional to the degree of node which it will be connected to. After some math, we get m(k) k β, where m(k) is the number of nodes whose degree is k, and β = 3 p 1 p. (on board, for p = 0)

94 Thank you! Some slides are based on the MMDS book and Jeff Phillips lecture notes

### Part 2: Community Detection

Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection - Social networks -

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### Mining Social-Network Graphs

342 Chapter 10 Mining Social-Network Graphs There is much information to be gained by analyzing the large-scale data that is derived from social networks. The best-known example of a social network is

### Why graph clustering is useful?

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as any other clustering Identification of communities in social networks Webpage clustering for better data management

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### Neural Networks Lesson 5 - Cluster Analysis

Neural Networks Lesson 5 - Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt. - Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### Approximation Algorithms. Scheduling. Approximation algorithms. Scheduling jobs on a single machine

Approximation algorithms Approximation Algorithms Fast. Cheap. Reliable. Choose two. NP-hard problems: choose 2 of optimal polynomial time all instances Approximation algorithms. Trade-off between time

### USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE- FREE NETWORKS AND SMALL-WORLD NETWORKS

USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE- FREE NETWORKS AND SMALL-WORLD NETWORKS Natarajan Meghanathan Jackson State University, 1400 Lynch St, Jackson, MS, USA natarajan.meghanathan@jsums.edu

### Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016

Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with

### Distance based clustering

// Distance based clustering Chapter ² ² Clustering Clustering is the art of finding groups in data (Kaufman and Rousseeuw, 99). What is a cluster? Group of objects separated from other clusters Means

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### Nimble Algorithms for Cloud Computing. Ravi Kannan, Santosh Vempala and David Woodruff

Nimble Algorithms for Cloud Computing Ravi Kannan, Santosh Vempala and David Woodruff Cloud computing Data is distributed arbitrarily on many servers Parallel algorithms: time Streaming algorithms: sublinear

### Data Mining. Cluster Analysis: Advanced Concepts and Algorithms

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototype-based clustering Density-based clustering Graph-based

### CMPSCI611: Approximating MAX-CUT Lecture 20

CMPSCI611: Approximating MAX-CUT Lecture 20 For the next two lectures we ll be seeing examples of approximation algorithms for interesting NP-hard problems. Today we consider MAX-CUT, which we proved to

### Well-Separated Pair Decomposition for the Unit-disk Graph Metric and its Applications

Well-Separated Pair Decomposition for the Unit-disk Graph Metric and its Applications Jie Gao Department of Computer Science Stanford University Joint work with Li Zhang Systems Research Center Hewlett-Packard

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Lecture 9. 1 Introduction. 2 Random Walks in Graphs. 1.1 How To Explore a Graph? CS-621 Theory Gems October 17, 2012

CS-62 Theory Gems October 7, 202 Lecture 9 Lecturer: Aleksander Mądry Scribes: Dorina Thanou, Xiaowen Dong Introduction Over the next couple of lectures, our focus will be on graphs. Graphs are one of

### Clustering and Data Mining in R

Clustering and Data Mining in R Workshop Supplement Thomas Girke December 10, 2011 Introduction Data Preprocessing Data Transformations Distance Methods Cluster Linkage Hierarchical Clustering Approaches

### ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING)

ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Preliminaries Classification and Clustering Applications

### Mining Social Network Graphs

Mining Social Network Graphs Debapriyo Majumdar Data Mining Fall 2014 Indian Statistical Institute Kolkata November 13, 17, 2014 Social Network No introduc+on required Really? We s7ll need to understand

### Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu

Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?

### 6.042/18.062J Mathematics for Computer Science October 3, 2006 Tom Leighton and Ronitt Rubinfeld. Graph Theory III

6.04/8.06J Mathematics for Computer Science October 3, 006 Tom Leighton and Ronitt Rubinfeld Lecture Notes Graph Theory III Draft: please check back in a couple of days for a modified version of these

### CLASSIFICATION AND CLUSTERING. Anveshi Charuvaka

CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### B490 Mining the Big Data. 0 Introduction

B490 Mining the Big Data 0 Introduction Qin Zhang 1-1 Data Mining What is Data Mining? A definition : Discovery of useful, possibly unexpected, patterns in data. 2-1 Data Mining What is Data Mining? A

### Lecture 4 Online and streaming algorithms for clustering

CSE 291: Geometric algorithms Spring 2013 Lecture 4 Online and streaming algorithms for clustering 4.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

### Social Media Mining. Network Measures

Klout Measures and Metrics 22 Why Do We Need Measures? Who are the central figures (influential individuals) in the network? What interaction patterns are common in friends? Who are the like-minded users

Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototype-based Fuzzy c-means

### Cluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico

Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from

### Social Media Mining. Data Mining Essentials

Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

### Data Mining Project Report. Document Clustering. Meryem Uzun-Per

Data Mining Project Report Document Clustering Meryem Uzun-Per 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. K-means algorithm...

### Algorithmic Aspects of Big Data. Nikhil Bansal (TU Eindhoven)

Algorithmic Aspects of Big Data Nikhil Bansal (TU Eindhoven) Algorithm design Algorithm: Set of steps to solve a problem (by a computer) Studied since 1950 s. Given a problem: Find (i) best solution (ii)

### DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDD-LAB ISTI- CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar

### Private Approximation of Clustering and Vertex Cover

Private Approximation of Clustering and Vertex Cover Amos Beimel, Renen Hallak, and Kobbi Nissim Department of Computer Science, Ben-Gurion University of the Negev Abstract. Private approximation of search

### Information Retrieval and Web Search Engines

Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 10 th, 2013 Wolf-Tilo Balke and Kinda El Maarry Institut für Informationssysteme Technische Universität Braunschweig

### An Enhanced Clustering Algorithm to Analyze Spatial Data

International Journal of Engineering and Technical Research (IJETR) ISSN: 2321-0869, Volume-2, Issue-7, July 2014 An Enhanced Clustering Algorithm to Analyze Spatial Data Dr. Mahesh Kumar, Mr. Sachin Yadav

### Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

### Geometrical Segmentation of Point Cloud Data using Spectral Clustering

Geometrical Segmentation of Point Cloud Data using Spectral Clustering Sergey Alexandrov and Rainer Herpers University of Applied Sciences Bonn-Rhein-Sieg 15th July 2014 1/29 Addressed Problem Given: a

### Graph Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar

Graph Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text Introduction to Parallel Computing, Addison Wesley, 3. Topic Overview Definitions and Representation Minimum

### Lesson 3. Algebraic graph theory. Sergio Barbarossa. Rome - February 2010

Lesson 3 Algebraic graph theory Sergio Barbarossa Basic notions Definition: A directed graph (or digraph) composed by a set of vertices and a set of edges We adopt the convention that the information flows

### Clustering Hierarchical clustering and k-mean clustering

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein The clustering problem: A quick review partition genes into distinct sets with high homogeneity

### Chapter 4: Non-Parametric Classification

Chapter 4: Non-Parametric Classification Introduction Density Estimation Parzen Windows Kn-Nearest Neighbor Density Estimation K-Nearest Neighbor (KNN) Decision Rule Gaussian Mixture Model A weighted combination

### Some questions... Graphs

Uni Innsbruck Informatik - 1 Uni Innsbruck Informatik - 2 Some questions... Peer-to to-peer Systems Analysis of unstructured P2P systems How scalable is Gnutella? How robust is Gnutella? Why does FreeNet

### princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

### Lecture 20: Clustering

Lecture 20: Clustering Wrap-up of neural nets (from last lecture Introduction to unsupervised learning K-means clustering COMP-424, Lecture 20 - April 3, 2013 1 Unsupervised learning In supervised learning,

### Clustering & Association

Clustering - Overview What is cluster analysis? Grouping data objects based only on information found in the data describing these objects and their relationships Maximize the similarity within objects

### A Survey of Kernel Clustering Methods

A Survey of Kernel Clustering Methods Maurizio Filippone, Francesco Camastra, Francesco Masulli and Stefano Rovetta Presented by: Kedar Grama Outline Unsupervised Learning and Clustering Types of clustering

### CMSC 451: Graph Properties, DFS, BFS, etc.

CMSC 451: Graph Properties, DFS, BFS, etc. Slides By: Carl Kingsford Department of Computer Science University of Maryland, College Park Based on Chapter 3 of Algorithm Design by Kleinberg & Tardos. Graphs

### Graph Theory and Complex Networks: An Introduction. Chapter 06: Network analysis. Contents. Introduction. Maarten van Steen. Version: April 28, 2014

Graph Theory and Complex Networks: An Introduction Maarten van Steen VU Amsterdam, Dept. Computer Science Room R.0, steen@cs.vu.nl Chapter 0: Version: April 8, 0 / Contents Chapter Description 0: Introduction

### 8.1 Makespan Scheduling

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programing: Min-Makespan and Bin Packing Date: 2/19/15 Scribe: Gabriel Kaptchuk 8.1 Makespan Scheduling Consider an instance

### Chapter ML:XI (continued)

Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained

### AIMA Chapter 3: Solving problems by searching

AIMA Chapter 3: Solving problems by searching Ana Huaman January 13, 2011 1. Define in your own words the following terms: state, state space, search tree, search node, goal, action, transition model and

### Guessing Game: NP-Complete?

Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple

### CSC2420 Fall 2012: Algorithm Design, Analysis and Theory

CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms

### UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS

UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS Dwijesh C. Mishra I.A.S.R.I., Library Avenue, New Delhi-110 012 dcmishra@iasri.res.in What is Learning? "Learning denotes changes in a system that enable

### CSE 494 CSE/CBS 598 (Fall 2007): Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye

CSE 494 CSE/CBS 598 Fall 2007: Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye 1 Introduction One important method for data compression and classification is to organize

### Approximated Distributed Minimum Vertex Cover Algorithms for Bounded Degree Graphs

Approximated Distributed Minimum Vertex Cover Algorithms for Bounded Degree Graphs Yong Zhang 1.2, Francis Y.L. Chin 2, and Hing-Fung Ting 2 1 College of Mathematics and Computer Science, Hebei University,

### 10-810 /02-710 Computational Genomics. Clustering expression data

10-810 /02-710 Computational Genomics Clustering expression data What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally,

### GRAPH data are important data types in many scientific

Limited Random Walk Algorithm for Big Graph Data Clustering Honglei Zhang, Member, IEEE, Jenni Raitoharju, Member, IEEE, Serkan Kiranyaz, Senior Member, IEEE, and Moncef Gabbouj, Fellow, IEEE arxiv:66.645v

### GRAPH THEORY and APPLICATIONS. Trees

GRAPH THEORY and APPLICATIONS Trees Properties Tree: a connected graph with no cycle (acyclic) Forest: a graph with no cycle Paths are trees. Star: A tree consisting of one vertex adjacent to all the others.

### Lecture 7: Approximation via Randomized Rounding

Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

### A Tutorial on Spectral Clustering

A Tutorial on Spectral Clustering Ulrike von Luxburg Max Planck Institute for Biological Cybernetics Spemannstr. 38, 7276 Tübingen, Germany ulrike.luxburg@tuebingen.mpg.de This article appears in Statistics

### Social Networks and Social Media

Social Networks and Social Media Social Media: Many-to-Many Social Networking Content Sharing Social Media Blogs Microblogging Wiki Forum 2 Characteristics of Social Media Consumers become Producers Rich

### Introduction to Segmentation

Lecture 2: Introduction to Segmentation Jonathan Krause 1 Goal Goal: Identify groups of pixels that go together image credit: Steve Seitz, Kristen Grauman 2 Types of Segmentation Semantic Segmentation:

### Conductance, the Normalized Laplacian, and Cheeger s Inequality

Spectral Graph Theory Lecture 6 Conductance, the Normalized Laplacian, and Cheeger s Inequality Daniel A. Spielman September 21, 2015 Disclaimer These notes are not necessarily an accurate representation

### Chapter 6. Cuboids. and. vol(conv(p ))

Chapter 6 Cuboids We have already seen that we can efficiently find the bounding box Q(P ) and an arbitrarily good approximation to the smallest enclosing ball B(P ) of a set P R d. Unfortunately, both

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### Distances, Clustering, and Classification. Heatmaps

Distances, Clustering, and Classification Heatmaps 1 Distance Clustering organizes things that are close into groups What does it mean for two genes to be close? What does it mean for two samples to be

### Distance Metric Learning in Data Mining (Part I) Fei Wang and Jimeng Sun IBM TJ Watson Research Center

Distance Metric Learning in Data Mining (Part I) Fei Wang and Jimeng Sun IBM TJ Watson Research Center 1 Outline Part I - Applications Motivation and Introduction Patient similarity application Part II

### IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 20, NO. 7, JULY 2009 1181

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 20, NO. 7, JULY 2009 1181 The Global Kernel k-means Algorithm for Clustering in Feature Space Grigorios F. Tzortzis and Aristidis C. Likas, Senior Member, IEEE

### Algorithm Design and Analysis

Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;

### Efficient Load Balancing by Adaptive Bypasses for the Migration on the Internet

Efficient Load Balancing by Adaptive Bypasses for the Migration on the Internet Yukio Hayashi yhayashi@jaist.ac.jp Japan Advanced Institute of Science and Technology ICCS 03 Workshop on Grid Computing,

### 1 Basic Definitions and Concepts in Graph Theory

CME 305: Discrete Mathematics and Algorithms 1 Basic Definitions and Concepts in Graph Theory A graph G(V, E) is a set V of vertices and a set E of edges. In an undirected graph, an edge is an unordered

### Unsupervised Learning: Clustering with DBSCAN Mat Kallada

Unsupervised Learning: Clustering with DBSCAN Mat Kallada STAT 2450 - Introduction to Data Mining Supervised Data Mining: Predicting a column called the label The domain of data mining focused on prediction:

### Segmentation & Clustering

EECS 442 Computer vision Segmentation & Clustering Segmentation in human vision K-mean clustering Mean-shift Graph-cut Reading: Chapters 14 [FP] Some slides of this lectures are courtesy of prof F. Li,

### Flat Clustering K-Means Algorithm

Flat Clustering K-Means Algorithm 1. Purpose. Clustering algorithms group a set of documents into subsets or clusters. The cluster algorithms goal is to create clusters that are coherent internally, but

### MapReduce Algorithms. Sergei Vassilvitskii. Saturday, August 25, 12

MapReduce Algorithms A Sense of Scale At web scales... Mail: Billions of messages per day Search: Billions of searches per day Social: Billions of relationships 2 A Sense of Scale At web scales... Mail:

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### Machine Learning using MapReduce

Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous

### Data Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

### Lecture Topic: Low-Rank Approximations

Lecture Topic: Low-Rank Approximations Low-Rank Approximations We have seen principal component analysis. The extraction of the first principle eigenvalue could be seen as an approximation of the original

### Complex Networks Analysis: Clustering Methods

Complex Networks Analysis: Clustering Methods Nikolai Nefedov Spring 2013 ISI ETH Zurich nefedov@isi.ee.ethz.ch 1 Outline Purpose to give an overview of modern graph-clustering methods and their applications

### Data visualization and clustering. Genomics is to no small extend a data science

Data visualization and clustering Genomics is to no small extend a data science [www.data2discovery.org] Data visualization and clustering Genomics is to no small extend a data science [Andersson et al.,

### Clustering. Chapter 7. 7.1 Introduction to Clustering Techniques. 7.1.1 Points, Spaces, and Distances

240 Chapter 7 Clustering Clustering is the process of examining a collection of points, and grouping the points into clusters according to some distance measure. The goal is that points in the same cluster

### Fig. 1 A typical Knowledge Discovery process [2]

Volume 4, Issue 7, July 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Review on Clustering

### Closest Pair Problem

Closest Pair Problem Given n points in d-dimensions, find two whose mutual distance is smallest. Fundamental problem in many applications as well as a key step in many algorithms. p q A naive algorithm

### Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining

Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/2004 Hierarchical

### MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

### Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques. My T. Thai

Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques My T. Thai 1 Overview An overview of LP relaxation and rounding method is as follows: 1. Formulate an optimization

### Distance Degree Sequences for Network Analysis

Universität Konstanz Computer & Information Science Algorithmics Group 15 Mar 2005 based on Palmer, Gibbons, and Faloutsos: ANF A Fast and Scalable Tool for Data Mining in Massive Graphs, SIGKDD 02. Motivation

### LABEL PROPAGATION ON GRAPHS. SEMI-SUPERVISED LEARNING. ----Changsheng Liu 10-30-2014

LABEL PROPAGATION ON GRAPHS. SEMI-SUPERVISED LEARNING ----Changsheng Liu 10-30-2014 Agenda Semi Supervised Learning Topics in Semi Supervised Learning Label Propagation Local and global consistency Graph

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### In the following we will only consider undirected networks.

Roles in Networks Roles in Networks Motivation for work: Let topology define network roles. Work by Kleinberg on directed graphs, used topology to define two types of roles: authorities and hubs. (Each

### Random Map Generator v1.0 User s Guide

Random Map Generator v1.0 User s Guide Jonathan Teutenberg 2003 1 Map Generation Overview...4 1.1 Command Line...4 1.2 Operation Flow...4 2 Map Initialisation...5 2.1 Initialisation Parameters...5 -w xxxxxxx...5

### Clustering. Data Mining. Abraham Otero. Data Mining. Agenda

Clustering 1/46 Agenda Introduction Distance K-nearest neighbors Hierarchical clustering Quick reference 2/46 1 Introduction It seems logical that in a new situation we should act in a similar way as in

### Going Big in Data Dimensionality:

LUDWIG- MAXIMILIANS- UNIVERSITY MUNICH DEPARTMENT INSTITUTE FOR INFORMATICS DATABASE Going Big in Data Dimensionality: Challenges and Solutions for Mining High Dimensional Data Peer Kröger Lehrstuhl für