Big Data Graph Algorithms

Transcription

1 Christian Schulz CompSE seminar, RWTH Aachen, Karlsruhe 1 Christian Schulz: Institute for Theoretical Informatics

2 Algorithm Engineering design analyze Algorithms implement experiment 1 Christian Schulz:

3 Research Areas Scalable (parallel) sorting Scalable (parallel/external) graph partitioning Scalable (parallel) graph generation Scalable (parallel/external) matchings Scalable (shared-mem parallel) graph drawing Independent sets on large inputs 2 Christian Schulz:

4 Research Areas Scalable (parallel) sorting Scalable (parallel/external) graph partitioning Scalable (parallel) graph generation Scalable (parallel/external) matchings Scalable (shared-mem parallel) graph drawing Independent sets on large inputs 2 Christian Schulz:

5 Huge Complex Networks Rn n 3 Ax = b Rn 3 Christian Schulz:

6 Scalable Graph Partitioning joint work with: H. Meyerhenke and P. Sanders 3 Christian Schulz:

7 The Common Parallel Approach Mesh partitioned via dual graph 1. Each volume (data, calculation) represented by a vertex (+edges) 2. Interdependencies represented by edges All PE s get same amount of work Communication is expensive Graph Partitioning Problem: Partition a graph into (almost) equally sized blocks, such that the number of edges connecting vertices from different blocks is minimal. 4 Christian Schulz:

8 ɛ-balanced Graph Partitioning Partition graph G = (V, E, c : V R >0, ω : E R >0 ) into k disjoint blocks s.t. total node weight of each block 1 + ɛ total node weight k total weight of cut edges as small as possible Relevant Applications: graph processing frameworks, parallel sparse matrix vector mult Christian Schulz:

9 Multilevel Graph Partitioning input graph output partition... local improvement... match contract initial partitioning uncontract Successful in many systems (using matchings or AMG for coarsening) 6 Christian Schulz:

10 Matching-based Coarsening A B a b a+b A+B 1. compute matching 2. perform contraction 7 Christian Schulz:

11 Matching-based Coarsening A B a b a+b A+B 1. compute matching 2. perform contraction 7 Christian Schulz:

12 Local Search compute gain: g(v) = d ext (v) d int (v) select blocks alternately move nodes greedy edge cut: 7 8 Christian Schulz:

13 Local Search compute gain: g(v) = d ext (v) d int (v) select blocks alternately move nodes greedy edge cut: 7 8 Christian Schulz:

14 Local Search compute gain: g(v) = d ext (v) d int (v) select blocks alternately move nodes greedy edge cut: 7 8 Christian Schulz:

15 Local Search update gain g(v) of neighbors move a node at most once edge cut: 7, 6 8 Christian Schulz:

16 Local Search until stop criteria reached increase in cut possible edge cut: 7, 6, 5 8 Christian Schulz:

17 Local Search until stop criteria reached increase in cut possible edge cut: 7, 6, 5, 5 8 Christian Schulz:

18 Local Search until stop criteria reached increase in cut possible edge cut: 7, 6, 5, 5, 6 8 Christian Schulz:

19 Local Search cut #steps increase in cut possible undo changes until best solution reached 9 Christian Schulz:

20 Local Search final partition with edge cut 5 linear time 10 Christian Schulz:

21 KaHIP Karlsruhe High Quality Partitioning buffoon [ALENEX12] social [SEA14,IPDPS15] separators distr. evol. alg. [ALENEX12] [DIMACS12] highly balanced: [SEA13] A 0 B A 0 B A C C C B V F W [ESA11] cycles a la multigrid input graph Output Partition [IPDPS10]... flows etc. [ESA11]... edge local improvement ratings match + [SEA12/14] contract uncontract initial partitioning parallel [IPDPS10] Multilevel Graph Partitioning 11 Christian Schulz:

22 KaHIP Benchmarks 1. Walschaw Benchmark: runtime neglected 816 instances (ɛ {0%, 1%, 3%, 5%}) focus on solution quality almost all instances improved or reproduced 2. 10th DIMACS Implementation Challenge best scores in categories: solution quality and runtime vs. solution quality 12 Christian Schulz:

23 Partition Complex Networks 12 Christian Schulz:

24 Matching-based Coarsening Problem bad for networks that are highly irregular substantial reduction is hard using matchings may contract wrong edges! 13 Christian Schulz:

25 Matching-based Coarsening Problem bad for networks that are highly irregular substantial reduction is hard using matchings may contract wrong edges! 13 Christian Schulz:

26 Basic Idea aggressive contraction / simple and fast local search main idea: contract clusterings clustering paradigm: internally dense and externally sparse 14 Christian Schulz:

27 Basic Idea Contraction of Clusterings a b c A C B a+b+c A+B+C contraction: respect balance and cut avoid large blocks: size constraint U recurse until graph is small 15 Christian Schulz:

28 Basic Idea Contraction of Clusterings a b c A C B a+b+c A+B+C contraction: respect balance and cut avoid large blocks: size constraint U recurse until graph is small 15 Christian Schulz:

29 Label Propagation Cut-based, Linear Time Clustering Algorithm [Raghavan et. al] cut-based clustering using size-constraint label propagation start with singletons traverse nodes in random order or smallest degree first move node to cluster having strongest eligible connection modification eligible: w.r.t size constraint U Scan 16 Christian Schulz:

30 Label Propagation Iteration Cut [%] Christian Schulz:

31 Label Propagation Iteration Cut [%] Christian Schulz:

32 Label Propagation Iteration Cut [%] Christian Schulz:

33 Label Propagation Iteration Cut [%] Christian Schulz:

34 Label Propagation Iteration Cut [%] Christian Schulz:

35 Label Propagation Iteration Cut [%] Christian Schulz:

36 Label Propagation Iteration Cut [%] Christian Schulz:

37 Label Propagation Iteration Cut [%] Christian Schulz:

38 Label Propagation Iteration Cut [%] Christian Schulz:

39 Label Propagation Iteration Cut [%] Christian Schulz:

40 Label Propagation Simple Local Search Greedy Local Search: start with partition from coarser level traverse nodes in random order move node to cluster having strongest eligible connection eligible: w.r.t size constraint U := (1 + ɛ) V k Scan 18 Christian Schulz:

41 Parallelization 18 Christian Schulz:

42 Graph Distribution over PEs Graph Distribution: a PE receives n/p vertices and their edges Processor I Processor II Communication ghost nodes: adjacent nodes on other processor (communication!) interface nodes: nodes adjacent to ghost nodes 19 Christian Schulz:

43 Label Propagation Distributed Memory each PE has a static part of the graph, only block IDs can change Overlap Computation and Communication (PE centric view): V Phase i 1 Phase i Phase i+1 Scan At the end of phase i: send block ID updates of phase i to neighboring PEs receive block ID updates from neighboring PEs from phase i 1 * while scanning in phase i, messages are routed through the network * algorithm converged nothing will be communicated 20 Christian Schulz:

44 Contraction of Clusterings The Parallel Case High Level a b c A C B a+b+c A+B+C parallel find mapping C : n.. 1 n.. 1 exchange subgraphs, compute contracted graph locally when graph small parallel initial partitioning 21 Christian Schulz:

45 Experiments 21 Christian Schulz:

46 Parallel Solution Quality Performance k = 2 blocks, 32PEs instances: meshes and social networks/web graphs ParMetis has ineffective coarsening (matching-based) due to memory consumption of coarsest graph (dist. among PEs) could not solve arabic-2005, sk-2005 and uk-2007 solved instances (ParMetis): fast and eco yield 19.2% and 27.4% improvement fast and eco slower on average social networks/web graphs: fast: 38% less cut edges and > 2 faster eco: 45% less cut edges and slower best instance: 18 faster and 61.6% less cut edges improvement over Facebook [Ugander and Backstrom]: 45% less cut edges on LiveJournal and much faster (k = 100) 22 Christian Schulz:

47 Strong Scaling Social Networks 1000 Fast sk-2007 Fast arabic-2005 Fast uk-2002 Fast uk-2007 Minimal uk-2007 total time [s] K 2K number of PEs p uk-2007 can be partitioned in 15.2 seconds (seq. 10.5min) 72 seconds for random geometric graph with 22G edges more scaling results in the paper 23 Christian Schulz:

48 Graph Drawing joint work with: H. Meyerhenke and M. Nöllenburg 23 Christian Schulz:

49 Problem x : V R 2 G = (V, E, d) d : E R 24 Christian Schulz:

50 Problem x : V R 2 G = (V, E, d) d : E R d 1 24 Christian Schulz:

51 Maximal Entropy Stress Model [Gansner et al. 13] Entropy H(x): physics: nodes evenly dispersed nodes as far away as possible some nodes have predefined distance! Maximal Entropy Stress Model: max H(x) := ln x u x v {u,v} E subject to x u x v = d uv, {u, v} E 25 Christian Schulz:

52 Maximal Entropy Stress Model [Gansner et al. 13] Entropy H(x): physics: nodes evenly dispersed nodes as far away as possible some nodes have predefined distance! Maximal Entropy Stress Model: max H(x) := ln x u x v {u,v} E subject to x u x v = d uv, {u, v} E not possible to satisfy all constraints! model may be infeasible 25 Christian Schulz:

53 Maximal Entropy Stress Model [Gansner et al. 13] Compromise: min error, max entropy min w uv ( x u x v d uv ) 2 αh(x) u,v E α trade-off parameter Solve optimization problem by repeatedly solving Laplacian systems or iterative scheme Christian Schulz:

54 Maximal Entropy Stress Model [Gansner et al. 13] Compromise: min error, max entropy min w uv ( x u x v d uv ) 2 αh(x) u,v E α trade-off parameter Solve optimization problem by repeatedly solving Laplacian systems or iterative scheme Christian Schulz:

55 Maximal Entropy Stress Model [Gansner et al. 13]... or iterative scheme: x u 1 ( x u x v ρ u w uv x v + d uv x {u,v} E u x v overall update costs O(n 2 ) per iteration ) + α ρ u {u,v}/ E x u x v x u x v 2 Our contributions: make this usable and fast in practice multilevel integration approximate long-range forces employ parallelism 27 Christian Schulz:

56 Multilevel Graph Drawing [Hadany, Harel 99] input graph output drawing... improve drawing... contract initial drawing uncontract as before: contract using size-constrained clusterings 28 Christian Schulz:

57 Multilevel Graph Drawing Initial Drawing coarsen until only two nodes left place them at optimal distance define distances on coarse graphs, stay tuned x u = (0, 0) x v = (0, d uv ) 29 Christian Schulz:

58 Uncoarsening Local Improvement minimize maxent-stress on each level of hierarchy assume disk with radius c(u) to draw c(u) vertices c(u)+ c(v) define distance d uv := 2 on current level Iterative Scheme x u 1 ( x u x v ρ u w uv x v + d uv x {u,v} E u x v ) + α ρ u {u,v}/ E x u x v x u x v 2 30 Christian Schulz:

59 Uncoarsening Local Improvement minimize maxent-stress on each level of hierarchy assume disk with radius c(u) to draw c(u) vertices c(u)+ c(v) define distance d uv := 2 on current level Iterative Scheme x u 1 ( x u x v ρ u w uv x v + d uv x {u,v} E u x v ) + α ρ u {u,v}/ E x u x v x u x v 2 30 Christian Schulz:

60 Local Improvement Iterative Scheme Approximation x u... {u,v}/ E x u x v x u x v } {{ 2 } =:r(u,v) x u r(u, v) + u =v M(u)=M(v) v V v =M(u) ν(v x u x v ) x u x v 2 {u,v} E r(u, v) M(u) cluster of u ν(v ) number of finer vertices of v on current level V vertex set of next coarser level x v coordinate of v 31 Christian Schulz:

61 Local Improvement Approximation x u r(u, v) + u =v M(u)=M(v) v V v =M(u) ν(v x u x v ) x u x v 2 M(u) cluster of u ν(v ) number of finer vertices of v on current level V vertex set of next coarser level x v coordinate of v {u,v} E r(u, v) 32 Christian Schulz:

62 Local Improvement Additional Enhancements after each iteration update barycenter of coarse nodes vertex computations independent add parallelism use approximation multiple h levels beneath in hierarchy input graph output drawing... improve drawing... contract initial drawing uncontract Proposition: Assume equal cluster sizes. The running time of one iteration of MulMent h, h 0, is O(m + n h+2 h+1 ) 33 Christian Schulz:

63 Experiments 33 Christian Schulz:

64 Scalability Running Time [Delaunay n = 2 20 ] 10 6 MulMent 0 MulMent 1 MulMent 2 MulMent 4 MulMent 6 MulMent 8 MulMent 9 MulMent total time [s] number of PEs p 34 Christian Schulz:

65 Running Times graph PMDS MaxEnt MulMent 0 MulMent 1 MulMent 10 btree bus USpowerG elt commanche bcsstk fe pwt del luxembourg nyc auto del Table : Running times in seconds per graph. Smaller is better. PivotMDS and MaxEnt use one thread (sequential codes), the MulMent algorithms use 32 cores (64 threads). Running times of MaxEnt are without the time of PMDS (which yields input coordinates to MaxEnt) 35 Christian Schulz:

66 Experimental Results Summary Influence of h / Comparision increasing h not a large impact on solution quality maxent-stress remains comparable maxent-stress of MaxEnt and MulMent more or less similar Dynamic Networks model: remove x% random edges, insert x% edges (distance D) 4 faster (h = 0), save 50% time (h = 7) 9% worse full-stress, 1% better maxent-stress 36 Christian Schulz:

67 Example Drawings fe pwt PMDS MaxEnt MulMent 37 Christian Schulz:

68 Example Drawings bcsstk31 PMDS 37 Christian Schulz: MaxEnt MulMent

69 Example Drawings commanche PMDS MaxEnt MulMent 37 Christian Schulz:

70 Independent Sets joint work with: S. Lamm, P. Sanders, D. Strash and R. Werneck 37 Christian Schulz:

71 Definitions Independent Set: subset S V such that there are no adjacent nodes in S Maximum Independent Set: maximum cardinality set S related to maximum clique and minimum vertex cover finding a MIS is NP-hard and hard to approximate 38 Christian Schulz:

72 Common Approaches use heuristic algorithms gradually improve a single solution node deletions, insertions and swaps plateau search diversification & restart rules Andrade et al. (2012): iterated local search using (j, k)-swaps 39 Christian Schulz:

73 ARW Local Search Node Swaps: (j, k)-swaps: remove j solution nodes and insert k new ones (1, 2)-swaps: remove single solution node and insert two new ones Local Search: search for (1, 2)-swaps in time O(m) use data structure that supports fast insertion and removal Solution nodes Free nodes Non-free non-solution nodes 40 Christian Schulz:

74 ɛ-balanced Graph Partitioning Node Separators Partition graph G = (V, E, c : V R >0, ω : E R >0 ) into k disjoint blocks + node separator s.t. total node weight of each block 1 + ɛ total node weight k total size of node separator as small as possible 41 Christian Schulz:

75 Evolutionary Algorithms General Structure procedure steady-state-ea create initial population P while stopping criterion not fulfilled select parents P 1, P 2 from P combine P 1 with P 2 to create offspring o mutate offspring o evict individual in population using o return the fittest individual that occurred 42 Christian Schulz:

76 Combine Operations Graph Partitioning: exchange whole blocks of solutions operators for edge separators and node separators small cut-size vital for efficiency + = Local Search: resulting independent sets may not be maximal use maximization step and ARW to reach local optimum 43 Christian Schulz:

77 Node Separator Combine V S V 2 S 1 V1 V 2 S V V 2 1 build node separator V = V 1 V 2 S use node separator as crossover point combination takes linear time O(n) maximize with greedy + ARW local search 44 Christian Schulz:

78 Node Separator Combine V S V 2 S 1 V1 V 2 S V V 2 1 build node separator V = V 1 V 2 S use node separator as crossover point combination takes linear time O(n) maximize with Greedy + ARW local search 44 Christian Schulz:

79 Kernelization [Akiba,Iwata 15] Reductions: rules to decrease graph size, while maintaining optimality solve problem on problem kernel (using EA) obtain solution on input graph 45 Christian Schulz:

80 Kernelization [Akiba,Iwata 15] Reductions: rules to decrease graph size, while maintaining optimality Example: remove degree 0 or 1 vertices v neighbor of v in MIS choose v instead, else add v 45 Christian Schulz:

81 Kernelization [Akiba,Iwata 15] Reductions: rules to decrease graph size, while maintaining optimality Example: remove degree 0 or 1 vertices v neighbor of v in MIS choose v instead, else add v much more reductions used in practice [Lamm et al. 16] 45 Christian Schulz:

82 Guess likely candidates can we guess vertices that are in MIS? idea: select small-degree vertices from fittest independent set apply more reductions and recurse! 46 Christian Schulz:

83 Experiments 46 Christian Schulz:

84 Near-optimal on difficult networks finds exact MIS faster, when exact algorithm is slow: Skitter 48 min 21 min Stanford 13 hours 5 min bcsstk hours 2.4 sec Skitter 2 hours 28 sec,... finds exact MIS, for large networks with known MIS size consistently finds larger solutions on social and road networks Solution Size Time [s] ARW EvoMIS ReduMIS Solution Size Time [s] ARW EvoMIS ReduMIS consistent, even as we scale to graphs on 10M to 100M nodes 47 Christian Schulz:

85 Conclusions 47 Christian Schulz:

86 Conclusion apply algorithm engineering to optimization problems obtain algorithms that scale to large inputs and machines outperform state-of-the-art open source implementations KaHIP: algo2.iti.kit.edu/kahip KaDraw: algo2.iti.kit.edu/kadraw KaMIS: algo2.iti.kit.edu/kamis realistic algorithm models 3 engineering perf. guarantees 48 Christian Schulz: experiments appl. engineering implementation algorithm libraries 9 10 applications analysis deduction real Inputs design 4 falsifiable 5 hypotheses 7 induction 6 8