# Graph Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar

Save this PDF as:

Size: px
Start display at page:

Download "Graph Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar"

## Transcription

1 Graph Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text Introduction to Parallel Computing, Addison Wesley, 3.

2 Topic Overview Definitions and Representation Minimum Spanning Tree: Prim s Algorithm Single-Source Shortest Paths: Dijkstra s Algorithm All-Pairs Shortest Paths Transitive Closure Connected Components Algorithms for Sparse Graphs

3 Definitions and Representation An undirected graph G is a pair (V, E), where V is a finite set of points called vertices and E is a finite set of edges. An edge e E is an unordered pair (u, v), where u, v V. In a directed graph, the edge e is an ordered pair (u, v). An edge (u, v) is incident from vertex u and is incident to vertex v. A path from a vertex v to a vertex u is a sequence v, v, v,..., v k of vertices where v = v, v k = u, and (v i, v i+ ) E for i =,,..., k. The length of a path is defined as the number of edges in the path.

4 Definitions and Representation e f (a) (b) (a) An undirected graph and (b) a directed graph.

5 Definitions and Representation An undirected graph is connected if every pair of vertices is connected by a path. A forest is an acyclic graph, and a tree is a connected acyclic graph. A graph that has weights associated with each edge is called a weighted graph.

6 Definitions and Representation Graphs can be represented by their adjacency matrix or an edge (or vertex) list. Adjacency matrices have a value a i,j = if nodes i and j share an edge; otherwise. In case of a weighted graph, a i,j = w i,j, the weight of the edge. The adjacency list representation of a graph G = (V, E) consists of an array Adj[.. V ] of lists. Each list Adj[v] is a list of all vertices adjacent to v. For a grapn with n nodes, adjacency matrices take T heta(n ) space and adjacency list takes Θ( E ) space.

7 Definitions and Representation A = An undirected graph and its adjacency matrix representation An undirected graph and its adjacency list representation.

8 Minimum Spanning Tree A spanning tree of an undirected graph G is a subgraph of G that is a tree containing all the vertices of G. In a weighted graph, the weight of a subgraph is the sum of the weights of the edges in the subgraph. A minimum spanning tree (MST) for a weighted undirected graph is a spanning tree with minimum weight.

9 Minimum Spanning Tree An undirected graph and its minimum spanning tree.

10 Minimum Spanning Tree: Prim s Algorithm Prim s algorithm for finding an MST is a greedy algorithm. Start by selecting an arbitrary vertex, include it into the current MST. Grow the current MST by inserting into it the vertex closest to one of the vertices already in current MST.

11 Minimum Spanning Tree: Prim s Algorithm (a) Original graph b 5 d a 3 c 4 3 f e 5 d[] a b c d e f a b c d e f (b) After the first edge has been selected b 5 d a 3 c 4 3 f e 5 d[] a b c d e f a b c d e f (c) After the second edge has been selected b 5 d a 3 c 4 3 f e 5 d[] a b c d e f a b c d e f (d) Final minimum spanning tree b 5 d a 3 c 3 f d[] a b c d e 3 a b 5 c 3 5 PSfrag replacements d 4 e e f 5 f

12 Prim s minimum spanning tree algorithm.

13 Minimum Spanning Tree: Prim s Algorithm. procedure PRIM MST(V, E, w, r). begin 3. V T := {r}; 4. d[r] := ; 5. for all v (V V T ) do 6. if edge (r, v) exists set d[v] := w(r, v); 7. else set d[v] := ; 8. while V T V do 9. begin. find a vertex u such that d[u] := min{d[v] v (V V T )};. V T := V T {u};. for all v (V V T ) do 3. d[v] := min{d[v], w(u, v)}; 4. endwhile 5. end PRIM MST Prim s sequential minimum spanning tree algorithm.

14 Prim s Algorithm: Parallel Formulation The algorithm works in n outer iterations it is hard to execute these iterations concurrently. The inner loop is relatively easy to parallelize. Let p be the number of processes, and let n be the number of vertices. The adjacency matrix is partitioned in a -D block fashion, with distance vector d partitioned accordingly. In each step, a processor selects the locally closest node. followed by a global reduction to select globally closest node. This node is inserted into MST, and the choice broadcast to all processors. Each processor updates its part of the d vector locally.

15 Prim s Algorithm: Parallel Formulation d[..n] n p (a) A n (b) PSfrag replacements Processors i p- The partitioning of the distance array d and the adjacency matrix A among p processes.

16 Prim s Algorithm: Parallel Formulation The cost to select the minimum entry is O(n/p + log p). The cost of a broadcast is O(log p). The cost of local updation of the d vector is O(n/p). The parallel time per iteration is O(n/p + log p). The total parallel time is given by O(n /p + n log p). The corresponding isoefficiency is O(p log p).

17 Single-Source Shortest Paths For a weighted graph G = (V, E, w), the single-source shortest paths problem is to find the shortest paths from a vertex v V to all other vertices in V. Dijkstra s algorithm is similar to Prim s algorithm. It maintains a set of nodes for which the shortest paths are known. It grows this set based on the node closest to source using one of the nodes in the current shortest path set.

18 Single-Source Shortest Paths: Dijkstra s Algorithm. procedure DIJKSTRA SINGLE SOURCE SP(V, E, w, s). begin 3. V T := {s}; 4. for all v (V V T ) do 5. if (s, v) exists set l[v] := w(s, v); 6. else set l[v] := ; 7. while V T V do 8. begin 9. find a vertex u such that l[u] := min{l[v] v (V V T )};. V T := V T {u};. for all v (V V T ) do. l[v] := min{l[v], l[u] + w(u, v)}; 3. endwhile 4. end DIJKSTRA SINGLE SOURCE SP Dijkstra s sequential single-source shortest paths algorithm.

19 Dijkstra s Algorithm: Parallel Formulation Very similar to the parallel formulation of Prim s algorithm for minimum spanning trees. The weighted adjacency matrix is partitioned using the -D block mapping. Each process selects, locally, the node closest to the source, followed by a global reduction to select next node. The node is broadcast to all processors and the l-vector updated. The parallel performance of Dijkstra s algorithm is identical to that of Prim s algorithm.

20 All-Pairs Shortest Paths Given a weighted graph G(V, E, w), the all-pairs shortest paths problem is to find the shortest paths between all pairs of vertices v i, v j V. A number of algorithms are known for solving this problem.

21 All-Pairs Shortest Paths: Matrix-Multiplication Based Algorithm Consider the multiplication of the weighted adjacency matrix with itself except, in this case, we replace the multiplication operation in matrix multiplication by addition, and the addition operation by minimization. Notice that the product of weighted adjacency matrix with itself returns a matrix that contains shortest paths of length between any pair of nodes. It follows from this argument that A n contains all shortest paths.

22 Matrix-Multiplication Based Algorithm A 3 C B D E G F 3 H I A = A 4 = C A C A A = A 8 = C A C A

23 Matrix-Multiplication Based Algorithm A n is computed by doubling powers i.e., as A, A, A 4, A 8, and so on. We need log n matrix multiplications, each taking time O(n 3 ). The serial complexity of this procedure is O(n 3 log n). This algorithm is not optimal, since the best known algorithms have complexity O(n 3 ).

24 Matrix-Multiplication Based Algorithm: Parallel Formulation Each of the log n matrix multiplications can be performed in parallel. We can use n 3 / log n processors to compute each matrix-matrix product in time log n. The entire process takes O(log n) time.

25 Dijkstra s Algorithm Execute n instances of the single-source shortest path problem, one for each of the n source vertices. Complexity is O(n 3 ).

26 Dijkstra s Algorithm: Parallel Formulation Two parallelization strategies execute each of the n shortest path problems on a different processor (source partitioned), or use a parallel formulation of the shortest path problem to increase concurrency (source parallel).

27 Dijkstra s Algorithm: Source Partitioned Formulation Use n processors, each processor P i finds the shortest paths from vertex v i to all other vertices by executing Dijkstra s sequential single-source shortest paths algorithm. It requires no interprocess communication (provided that the adjacency matrix is replicated at all processes). The parallel run time of this formulation is: Θ(n ). While the algorithm is cost optimal, it can only use n processors. Therefore, the isoefficiency due to concurrency is p 3.

28 Dijkstra s Algorithm: Source Parallel Formulation In this case, each of the shortest path problems is further executed in parallel. We can therefore use up to n processors. Given p processors (p > n), each single source shortest path problem is executed by p/n processors. Using previous results, this takes time: T P = computation { (}} ){ n 3 Θ p + communication {}}{ Θ(n log p). () For cost optimality, we have p = O(n / log n) and the isoefficiency is Θ((p log p).5 ).

29 Floyd s Algorithm For any pair of vertices v i, v j V, consider all paths from v i to v j whose intermediate vertices belong to the set {v, v,..., v k }. Let p (k) i,j (of weight d (k) i,j be the minimum-weight path among them. If vertex v k is not in the shortest path from v i to v j, then p (k) i,j same as p (k ) i,j. is the If f v k is in p (k) i,j, then we can break p(k) i,j into two paths one from v i to v k and one from v k to v j. Each of these paths uses vertices from {v, v,..., v k }.

30 Floyd s Algorithm From our observations, the following recurrence relation follows: { w(vi i,j = {, v j ) } if k = min d (k ), d (k ) + d (k ) if k d (k) i,j i,k k,j () This equation must be computed for each pair of nodes and for k =, n. The serial complexity is O(n 3 ).

31 Floyd s Algorithm. procedure FLOYD ALL PAIRS SP(A). begin 3. D () = A; 4. for k := to n do 5. for i := to n do 6. for j := to n do 7. d (k) i,j := min i,j 8. end FLOYD ALL PAIRS SP d (k ), d (k ) i,k + d (k ) k,j ; Floyd s all-pairs shortest paths algorithm. This program computes the all-pairs shortest paths of the graph G = (V, E) with adjacency matrix A.

32 Floyd s Algorithm: Parallel Formulation Using -D Block Mapping Matrix D (k) is divided into p blocks of size (n/ p) (n/ p). Each processor updates its part of the matrix during each iteration. To compute d (k) l,r processor P i,j must get d (k ) l,k and d (k ) k,r. In general, during the k th iteration, each of the p processes containing part of the k th row send it to the p processes in the same column. Similarly, each of the p processes containing part of the k th column sends it to the p processes in the same row.

33 Floyd s Algorithm: Parallel Formulation Using -D Block Mapping n p n p (,) (,) (,) (i ) n p +, (j ) n p + (i,j) PSfrag replacements i n p, j n p (a) (b) (a) Matrix D (k) distributed by -D block mapping into p p subblocks, and (b) the subblock of D (k) assigned to process P i,j.

34 Floyd s Algorithm: Parallel Formulation Using -D Block Mapping k column k column d (k ) k,r k row d (k ) l,k d (k) l,r PSfrag replacements (a) (b) (a) Communication patterns used in the -D block mapping. When computing d (k) i,j, information must be sent to the highlighted process from two other processes along the same row and column. (b) The row and column of p processes that contain the k th row and column send them along process columns and rows.

35 Floyd s Algorithm: Parallel Formulation Using -D Block Mapping. procedure FLOYD DBLOCK(D () ). begin 3. for k := to n do 4. begin 5. each process P i,j that has a segment of the k th row of D (k ) ; broadcasts it to the P,j processes; 6. each process P i,j that has a segment of the k th column of D (k ) ; broadcasts it to the P i, processes; 7. each process waits to receive the needed segments; 8. each process P i,j computes its part of the D (k) matrix; 9. end. end FLOYD DBLOCK Floyd s parallel formulation using the -D block mapping. P,j denotes all the processes in the j th column, and P i, denotes all the processes in the i th row. The matrix D () is the adjacency matrix.

36 Floyd s Algorithm: Parallel Formulation Using -D Block Mapping During each iteration of the algorithm, the k th row and k th column of processors perform a one-to-all broadcast along their rows/columns. The size of this broadcast is n/ p elements, Θ((n log p)/ p). taking time The synchronization step takes time Θ(log p). The computation time is Θ(n /p). The parallel run time of the -D block mapping formulation of Floyd s algorithm is T P = computation { (}} ){ n 3 Θ p communication { ( }} ){ n + Θ log p. p

37 Floyd s Algorithm: Parallel Formulation Using -D Block Mapping The above formulation can use O(n / log n) processors costoptimally. The isoefficiency of this formulation is Θ(p.5 log 3 p). This algorithm can be further improved by relaxing the strict synchronization after each iteration.

38 Floyd s Algorithm: Speeding Things Up by Pipelining The synchronization step in parallel Floyd s algorithm can be removed without affecting the correctness of the algorithm. A process starts working on the k th iteration as soon as it has computed the (k ) th iteration and has the relevant parts of the D (k ) matrix.

39 Floyd s Algorithm: Speeding Things Up by Pipelining Time t t+ t+ t+3 t+4 t Processors Communication protocol followed in the pipelined -D block mapping formulation of Floyd s algorithm. Assume that process 4 at time t has just computed a segment of the k th column of the D (k ) matrix. It sends the segment to processes 3 and 5. These processes receive the segment at time t + (where the time unit is the time it takes for a matrix segment to travel over the communication link between adjacent processes). Similarly, processes farther away from process 4 receive the segment later. Process (at the boundary) does not forward the segment after receiving it.

40 Floyd s Algorithm: Speeding Things Up by Pipelining In each step, n/ p elements of the first row are sent from process P i,j to P i+,j. Similarly, elements of the first column are sent from process P i,j to process P i,j+. Each such step takes time Θ(n/ p). After Θ( p) steps, process P p, p gets the relevant elements of the first row and first column in time Θ(n). The values of successive rows and columns follow after time Θ(n /p) in a pipelined mode. Process P p, p finishes its share of the shortest path computation in time Θ(n 3 /p) + Θ(n). When process P p, p has finished the (n ) th iteration, it sends the relevant values of the n th row and column to the other processes.

41 Floyd s Algorithm: Speeding Things Up by Pipelining The overall parallel run time of this formulation is T P = computation { (}} ){ n 3 Θ p + communication {}}{ Θ(n). The pipelined formulation of Floyd s algorithm uses up to O(n ) processes efficiently. The corresponding isoefficiency is Θ(p.5 ).

42 All-pairs Shortest Path: Comparison The performance and scalability of the all-pairs shortest paths algorithms on various architectures with O(p) bisection bandwidth. Similar run times apply to all k d cube architectures, provided that processes are properly mapped to the underlying processors. Maximum Number of Processes Corresponding Isoefficiency for E = Θ() Parallel Run Time Function Dijkstra source-partitioned Θ(n) Θ(n ) Θ(p 3 ) Dijkstra source-parallel Θ(n / log n) Θ(n log n) Θ((p log p).5 ) Floyd -D block Θ(n/ log n) Θ(n log n) Θ((p log p) 3 ) Floyd -D block Θ(n / log n) Θ(n log n) Θ(p.5 log 3 p) Floyd pipelined -D block Θ(n ) Θ(n) Θ(p.5 )

43 Transitive Closure If G = (V, E) is a graph, then the /em transitive closure of G is defined as the graph G = (V, E ), where E = {(v i, v j ) there is a path from v i to v j in G}. The /em connectivity matrix of G is a matrix A = (a i,j ) such that a i,j = if there is a path from v i to v j or i = j, and a i,j = otherwise. To compute A we assign a weight of to each edge of E and use any of the all-pairs shortest paths algorithms on this weighted graph.

44 Connected Components The connected components of an undirected graph are the equivalence classes of vertices under the is reachable from relation A graph with three connected components: {,, 3, 4}, {5, 6, 7}, and {8, 9}.

45 Connected Components: Depth-First Search Based Algorithm Perform DFS on the graph to get a forest eac tree in the forest corresponds to a separate connected component (a) (b) Part (b) is a depth-first forest obtained from depth-first traversal of the graph in part (a). Each of these trees is a connected component of the graph in part (a).

46 Connected Components: Parallel Formulation Partition the graph across processors and run independent connected component algorithms on each processor. At this point, we have p spanning forests. In the second step, spanning forests are merged pairwise until only one spanning forest remains.

47 Connected Components: Parallel Formulation Processor Processor (b) (d) (e) (f) (c) (a) Computing connected components in parallel. The adjacency matrix of the graph G in (a) is partitioned into two parts (b). Each process gets a subgraph of G ((c) and (e)). Each process then computes the spanning forest of the subgraph ((d) and (f)). Finally, the two spanning trees are merged to form the solution.

48 Connected Components: Parallel Formulation To merge pairs of spanning forests efficiently, the algorithm uses disjoint sets of edges. We define the following operations on the disjoint sets: find(x) returns a pointer to the representative element of the set containing x. Each set has its own unique representative. union(x, y) unites the sets containing the elements x and y. The two sets are assumed to be disjoint prior to the operation.

49 Connected Components: Parallel Formulation For merging forest A into forest B, for each edge (u, v) of A, a find operation is performed to determine if the vertices are in the same tree of B. If not, then the two trees (sets) of B containing u and v are united by a union operation. Otherwise, no union operation is necessary. Hence, merging A and B requires at most (n ) find operations and (n ) union operations.

50 Connected Components: Parallel -D Block Mapping The n n adjacency matrix is partitioned into p blocks. Each processor can compute its local spanning forest in time Θ(n /p). Merging is done by embedding a logical tree into the topology. There are log p merging stages, and each takes time Θ(n). Thus, the cost due to merging is Θ(n log p). During each merging stage, spanning forests are sent between nearest neighbors. Recall that Θ(n) edges of the spanning forest are transmitted.

51 Connected Components: Parallel -D Block Mapping The parallel run time of the connected-component algorithm is local computation { (}} ){ forest merging n {}}{ T P = Θ + Θ(n log p). p For a cost-optimal formulation p = O(n/ log n). The corresponding isoefficiency is Θ(p log p).

52 Algorithms for Sparse Graphs A graph G = (V, E) is sparse if E is much smaller than V. (a) (b) (c) Examples of sparse graphs: (a) a linear graph, in which each vertex has two incident edges; (b) a grid graph, in which each vertex has four incident vertices; and (c) a random sparse graph.

53 Algorithms for Sparse Graphs Dense algorithms can be improved significantly if we make use of the sparseness. For example, the run time of Prim s minimum spanning tree algorithm can be reduced from Θ(n ) to Θ( E log n). Sparse algorithms use adjacency list instead of an adjacency matrix. Partitioning adjacency lists is more difficult for sparse graphs do we balance number of vertices or edges? Parallel algorithms typically make use of graph structure or degree information for performance.

54 Algorithms for Sparse Graphs (a) (b) A street map (a) can be represented by a graph (b). In the graph shown in (b), each street intersection is a vertex and each edge is a street segment. The vertices of (b) are the intersections of (a) marked by dots.

55 Finding a Maximal Independent Set A set of vertices I V is called /em independent if no pair of vertices in I is connected via an edge in G. An independent set is called /em maximal if by including any other vertex not in I, the independence property is violated. a b d e e g f i j {a, d, i, h} is an independent set {a, c, j, f, g} is a maximal independent set {a, d, h, f} is a maximal independent set c h Examples of independent and maximal independent sets.

56 Finding a Maximal Independent Set (MIS) Simple algorithms start by MIS I to be empty, and assigning all vertices to a candidate set C. Vertex v from C is moved into I and all vertices adjacent to v are removed from C. This process is repeated until C is empty. This process is inherently serial!

57 Finding a Maximal Independent Set (MIS) Parallel MIS algorithms use randimization to gain concurrency (Luby s algorithm for graph coloring). Initially, each node is in the candidate set C. Each node generates a (unique) random number and communicates it to its neighbors. If a nodes number exceeds that of all its neighbors, it joins set I. All of its neighbors are removed from C. This process continues until C is empty. On average, this algorithm converges after O(log V ) such steps.

58 Finding a Maximal Independent Set (MIS) Vertex in the independent set Vertex adjacent to a vertex in the independent set 5 4 (a) After the st random number assignment 5 (b) After the nd random number assignment (c) Final maximal independent set The different augmentation steps of Luby s randomized maximal independent set algorithm. The numbers inside each vertex correspond to the random number assigned to the vertex.

59 Finding a Maximal Independent Set (MIS): Parallel Formulation We use three arrays, each of length n I, which stores nodes in MIS, C, which stores the candidate set, and R, the random numbers. Partition C across p processors. Each processor generates the corresponding values in the R array, and from this, computes which candidate vertices can enter MIS. The C array is updated by deleting all the neighbors of vertices that entered MIS. The performance of this algorithm is dependent on the structure of the graph.

60 Single-Source Shortest Paths Dijkstra s algorithm, modified to handle sparse graphs is called Johnson s algorithm. The modification accounts for the fact that the minimization step in Dijkstra s algorithm needs to be performed only for those nodes adjacent to the previously selected nodes. Johnson s algorithm uses a priority queue Q to store the value l[v] for each vertex v (V V T ).

61 Single-Source Shortest Paths: Johnson s Algorithm. procedure JOHNSON SINGLE SOURCE SP(V, E, s). begin 3. Q := V ; 4. for all v Q do 5. l[v] := ; 6. l[s] := ; 7. while Q do 8. begin 9. u := extract min(q);. for each v Adj[u] do. if v Q and l[u] + w(u, v) < l[v] then. l[v] := l[u] + w(u, v); 3. endwhile 4. end JOHNSON SINGLE SOURCE SP Johnson s sequential single-source shortest paths algorithm.

62 Single-Source Shortest Paths: Parallel Johnson s Algorithm Maintaining strict order of Johnson s algorithm generally leads to a very restrictive class of parallel algorithms. We need to allow exploration of multiple nodes concurrently. This is done by simultaneously extracting p nodes from the priority queue, updating the neighbors cost, and augmenting the shortest path. If an error is made, it can be discovered (as a shorter path) and the node can be reinserted with this shorter path.

63 ents Single-Source Shortest Paths: Parallel Johnson s Algorithm d e f 7 g h i a b c 3 () () (3) (4) Priority Queue b:, d:7, c:inf, e:inf, f:inf, g:inf, h:inf, i:inf e:3, c:4, g:, f:inf, h:inf, i:inf h:4, f:6, i:inf g:5, i:6 Array l[] a b c d e f g h i An example of the modified Johnson s algorithm for processing unsafe vertices concurrently.

64 Single-Source Shortest Paths: Parallel Johnson s Algorithm Even if we can extract and process multiple nodes from the queue, the queue itself is a major bottleneck. For this reason, we use multiple queues, one for each processor. Each processor builds its priority queue only using its own vertices. When process P i extracts the vertex u V i, it sends a message to processes that store vertices adjacent to u. Process P j, upon receiving this message, sets the value of l[v] stored in its priority queue to min{l[v], l[u] + w(u, v)}.

65 Single-Source Shortest Paths: Parallel Johnson s Algorithm If a shorter path has been discovered to node v, it is reinserted back into the local priority queue. The algorithm terminates only when all the queues become empty. A number of node paritioning schemes can be used to exploit graph structure for performance.

### Graph. Consider a graph, G in Fig Then the vertex V and edge E can be represented as:

Graph A graph G consist of 1. Set of vertices V (called nodes), (V = {v1, v2, v3, v4...}) and 2. Set of edges E (i.e., E {e1, e2, e3...cm} A graph can be represents as G = (V, E), where V is a finite and

### Sorting Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar

Sorting Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text Introduction to Parallel Computing, Addison Wesley, 2003. Topic Overview Issues in Sorting on Parallel

### Social Media Mining. Graph Essentials

Graph Essentials Graph Basics Measures Graph and Essentials Metrics 2 2 Nodes and Edges A network is a graph nodes, actors, or vertices (plural of vertex) Connections, edges or ties Edge Node Measures

### CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.

Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure

### CMSC 451: Graph Properties, DFS, BFS, etc.

CMSC 451: Graph Properties, DFS, BFS, etc. Slides By: Carl Kingsford Department of Computer Science University of Maryland, College Park Based on Chapter 3 of Algorithm Design by Kleinberg & Tardos. Graphs

### 1 Basic Definitions and Concepts in Graph Theory

CME 305: Discrete Mathematics and Algorithms 1 Basic Definitions and Concepts in Graph Theory A graph G(V, E) is a set V of vertices and a set E of edges. In an undirected graph, an edge is an unordered

### GRAPH THEORY and APPLICATIONS. Trees

GRAPH THEORY and APPLICATIONS Trees Properties Tree: a connected graph with no cycle (acyclic) Forest: a graph with no cycle Paths are trees. Star: A tree consisting of one vertex adjacent to all the others.

Chapter 7 Load Balancing and Termination Detection Load balancing used to distribute computations fairly across processors in order to obtain the highest possible execution speed. Termination detection

### Analysis of Algorithms, I

Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications

### 1 Representing Graphs With Arrays

Lecture 0 Shortest Weighted Paths I Parallel and Sequential Data Structures and Algorithms, 5-20 (Spring 202) Lectured by Margaret Reid-Miller 6 February 202 Today: - Representing Graphs with Arrays -

### Load Balancing and Termination Detection

Chapter 7 Load Balancing and Termination Detection 1 Load balancing used to distribute computations fairly across processors in order to obtain the highest possible execution speed. Termination detection

### CS2 Algorithms and Data Structures Note 11. Breadth-First Search and Shortest Paths

CS2 Algorithms and Data Structures Note 11 Breadth-First Search and Shortest Paths In this last lecture of the CS2 Algorithms and Data Structures thread we will consider the problem of computing distances

### Minimum Spanning Trees

Minimum Spanning Trees Algorithms and 18.304 Presentation Outline 1 Graph Terminology Minimum Spanning Trees 2 3 Outline Graph Terminology Minimum Spanning Trees 1 Graph Terminology Minimum Spanning Trees

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### IE 680 Special Topics in Production Systems: Networks, Routing and Logistics*

IE 680 Special Topics in Production Systems: Networks, Routing and Logistics* Rakesh Nagi Department of Industrial Engineering University at Buffalo (SUNY) *Lecture notes from Network Flows by Ahuja, Magnanti

### Graphs. Graph G=(V,E): representation

Graphs G = (V,E) V the vertices of the graph {v 1, v 2,..., v n } E the edges; E a subset of V x V A cost function cij is the cost/ weight of the edge (v i, v j ) Graph G=(V,E): representation 1. Adjacency

### Network (Tree) Topology Inference Based on Prüfer Sequence

Network (Tree) Topology Inference Based on Prüfer Sequence C. Vanniarajan and Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology Madras Chennai 600036 vanniarajanc@hcl.in,

### 5. A full binary tree with n leaves contains [A] n nodes. [B] log n 2 nodes. [C] 2n 1 nodes. [D] n 2 nodes.

1. The advantage of.. is that they solve the problem if sequential storage representation. But disadvantage in that is they are sequential lists. [A] Lists [B] Linked Lists [A] Trees [A] Queues 2. The

### COT5405 Analysis of Algorithms Homework 3 Solutions

COT0 Analysis of Algorithms Homework 3 Solutions. Prove or give a counter example: (a) In the textbook, we have two routines for graph traversal - DFS(G) and BFS(G,s) - where G is a graph and s is any

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

### Home Page. Data Structures. Title Page. Page 1 of 24. Go Back. Full Screen. Close. Quit

Data Structures Page 1 of 24 A.1. Arrays (Vectors) n-element vector start address + ielementsize 0 +1 +2 +3 +4... +n-1 start address continuous memory block static, if size is known at compile time dynamic,

### Heuristics and Local Search. Version 2 c 2009, 2001 José Fernando Oliveira, Maria Antónia Carravilla FEUP

Heuristics and Local Search Version c 009, 001 José Fernando Oliveira, Maria Antónia Carravilla FEUP Approximate methods to solve combinatorial optimization problems Heuristics Aim to efficiently generate

### Load Balancing and Termination Detection

Chapter 7 slides7-1 Load Balancing and Termination Detection slides7-2 Load balancing used to distribute computations fairly across processors in order to obtain the highest possible execution speed. Termination

### Lecture 9. 1 Introduction. 2 Random Walks in Graphs. 1.1 How To Explore a Graph? CS-621 Theory Gems October 17, 2012

CS-62 Theory Gems October 7, 202 Lecture 9 Lecturer: Aleksander Mądry Scribes: Dorina Thanou, Xiaowen Dong Introduction Over the next couple of lectures, our focus will be on graphs. Graphs are one of

### Floyd s Algorithm. Introduction. Used to find shortest paths in a weighted graph. Travel maps containing driving distance from one point to another

Floyd s Algorithm Introduction Used to find shortest paths in a weighted graph Travel maps containing driving distance from one point to another Represented by tables Shortest distance from point A to

### Solutions to Homework 6

Solutions to Homework 6 Debasish Das EECS Department, Northwestern University ddas@northwestern.edu 1 Problem 5.24 We want to find light spanning trees with certain special properties. Given is one example

### Dynamic Programming. Applies when the following Principle of Optimality

Dynamic Programming Applies when the following Principle of Optimality holds: In an optimal sequence of decisions or choices, each subsequence must be optimal. Translation: There s a recursive solution.

### CIS 700: algorithms for Big Data

CIS 700: algorithms for Big Data Lecture 6: Graph Sketching Slides at http://grigory.us/big-data-class.html Grigory Yaroslavtsev http://grigory.us Sketching Graphs? We know how to sketch vectors: v Mv

### Long questions answer Advanced Mathematics for Computer Application If P= , find BT. 19. If B = 1 0, find 2B and -3B.

Unit-1: Matrix Algebra Short questions answer 1. What is Matrix? 2. Define the following terms : a) Elements matrix b) Row matrix c) Column matrix d) Diagonal matrix e) Scalar matrix f) Unit matrix OR

### Warshall s Algorithm: Transitive Closure

CS 0 Theory of Algorithms / CS 68 Algorithms in Bioinformaticsi Dynamic Programming Part II. Warshall s Algorithm: Transitive Closure Computes the transitive closure of a relation (Alternatively: all paths

### 3. Markov property. Markov property for MRFs. Hammersley-Clifford theorem. Markov property for Bayesian networks. I-map, P-map, and chordal graphs

Markov property 3-3. Markov property Markov property for MRFs Hammersley-Clifford theorem Markov property for ayesian networks I-map, P-map, and chordal graphs Markov Chain X Y Z X = Z Y µ(x, Y, Z) = f(x,

### 1 Digraphs. Definition 1

1 Digraphs Definition 1 Adigraphordirected graphgisatriplecomprisedofavertex set V(G), edge set E(G), and a function assigning each edge an ordered pair of vertices (tail, head); these vertices together

### Chapter 4. Trees. 4.1 Basics

Chapter 4 Trees 4.1 Basics A tree is a connected graph with no cycles. A forest is a collection of trees. A vertex of degree one, particularly in a tree, is called a leaf. Trees arise in a variety of applications.

### Trees and Fundamental Circuits

Trees and Fundamental Circuits Tree A connected graph without any circuits. o must have at least one vertex. o definition implies that it must be a simple graph. o only finite trees are being considered

### A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points

A. V. Gerbessiotis CS 610-102 Spring 2014 PS 3 Mar 24, 2014 No points Problem 1. Suppose that we insert n keys into a hash table of size m using open addressing and uniform hashing. Let p(n, m) be the

### Lecture 6: Approximation via LP Rounding

Lecture 6: Approximation via LP Rounding Let G = (V, E) be an (undirected) graph. A subset C V is called a vertex cover for G if for every edge (v i, v j ) E we have v i C or v j C (or both). In other

Load Balancing Load Balancing Load balancing: distributing data and/or computations across multiple processes to maximize efficiency for a parallel program. Static load-balancing: the algorithm decides

### Theorem A graph T is a tree if, and only if, every two distinct vertices of T are joined by a unique path.

Chapter 3 Trees Section 3. Fundamental Properties of Trees Suppose your city is planning to construct a rapid rail system. They want to construct the most economical system possible that will meet the

### Lecture 11: All-Pairs Shortest Paths

Lecture 11: All-Pairs Shortest Paths Introduction Different types of algorithms can be used to solve the all-pairs shortest paths problem: Dynamic programming Matrix multiplication Floyd-Warshall algorithm

### Exam study sheet for CS2711. List of topics

Exam study sheet for CS2711 Here is the list of topics you need to know for the final exam. For each data structure listed below, make sure you can do the following: 1. Give an example of this data structure

### Computational Geometry

Motivation 1 Motivation Polygons and visibility Visibility in polygons Triangulation Proof of the Art gallery theorem Two points in a simple polygon can see each other if their connecting line segment

### Load Balancing and Termination Detection

Chapter 7 Slide 1 Slide 2 Load Balancing and Termination Detection Load balancing used to distribute computations fairly across processors in order to obtain the highest possible execution speed. Termination

### Shortest Paths. Algorithms and Networks 2014/2015 Hans L. Bodlaender Johan M. M. van Rooij

Shortest Paths Algorithms and Networks 2014/2015 Hans L. Bodlaender Johan M. M. van Rooij 1 Contents The shortest path problem: Statement Versions Applications Algorithms Reminders: Dijkstra, Bellman-Ford,

V. Adamchik 1 Graph Theory Victor Adamchik Fall of 2005 Plan 1. Basic Vocabulary 2. Regular graph 3. Connectivity 4. Representing Graphs Introduction A.Aho and J.Ulman acknowledge that Fundamentally, computer

### Chapter 2. Basic Terminology and Preliminaries

Chapter 2 Basic Terminology and Preliminaries 6 Chapter 2. Basic Terminology and Preliminaries 7 2.1 Introduction This chapter is intended to provide all the fundamental terminology and notations which

### Efficient parallel string comparison

Efficient parallel string comparison Peter Krusche and Alexander Tiskin Department of Computer Science ParCo 2007 Outline 1 Introduction Bulk-Synchronous Parallelism String comparison 2 Sequential LCS

### Planar Tree Transformation: Results and Counterexample

Planar Tree Transformation: Results and Counterexample Selim G Akl, Kamrul Islam, and Henk Meijer School of Computing, Queen s University Kingston, Ontario, Canada K7L 3N6 Abstract We consider the problem

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### CSE 373 Analysis of Algorithms Date: Nov. 1, Solution to HW3. Total Points: 100

CSE 373 Analysis of Algorithms Date: Nov. 1, 2016 Steven Skiena Problem 1 5-4 [5] Solution to HW3 Total Points: 100 Let T be the BFS-tree of the graph G. For any e in G and e T, we have to show that e

### Asking Hard Graph Questions. Paul Burkhardt. February 3, 2014

Beyond Watson: Predictive Analytics and Big Data U.S. National Security Agency Research Directorate - R6 Technical Report February 3, 2014 300 years before Watson there was Euler! The first (Jeopardy!)

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### Parallel and Distributed Computing Chapter 5: Basic Communications Operations

Parallel and Distributed Computing Chapter 5: Basic Communications Operations Jun Zhang Laboratory for High Performance Computing & Computer Simulation Department of Computer Science University of Kentucky

### Quiz 2 Solutions. Solution: False. Need stable sort.

Introduction to Algorithms April 13, 2011 Massachusetts Institute of Technology 6.006 Spring 2011 Professors Erik Demaine, Piotr Indyk, and Manolis Kellis Quiz 2 Solutions Problem 1. True or false [24

### Graphs and Network Flows IE411 Lecture 1

Graphs and Network Flows IE411 Lecture 1 Dr. Ted Ralphs IE411 Lecture 1 1 References for Today s Lecture Required reading Sections 17.1, 19.1 References AMO Chapter 1 and Section 2.1 and 2.2 IE411 Lecture

### Data Structures in Java. Session 16 Instructor: Bert Huang

Data Structures in Java Session 16 Instructor: Bert Huang http://www.cs.columbia.edu/~bert/courses/3134 Announcements Homework 4 due next class Remaining grades: hw4, hw5, hw6 25% Final exam 30% Midterm

### Chapter 7 Load Balancing and Termination Detection

Chapter 7 Load Balancing and Termination Detection Load balancing used to distribute computations fairly across processors in order to obtain the highest possible execution speed. Termination detection

### Example 5.5. The heuristics are demonstrated with this example of n=5 vertices and distances in the following matrix:

39 5.3. HEURISTICS FOR THE TSP Notation: f = length of the tour given by the algortihm a f = length of the optimal tour min Most of the following heuristics are designed to solve symmetric instances of

### COLORED GRAPHS AND THEIR PROPERTIES

COLORED GRAPHS AND THEIR PROPERTIES BEN STEVENS 1. Introduction This paper is concerned with the upper bound on the chromatic number for graphs of maximum vertex degree under three different sets of coloring

### All-Pairs Shortest Paths

All-Pairs Shortest Paths Input: A directed graph G = (V, E) where each edge (v i, v j ) has a weight w(i, j). Output: A shortest path from u to v, for all u, v V. Weight of path: Given a path p =< v 1,...,

### Chapter 2 Paths and Searching

Chapter 2 Paths and Searching Section 2.1 Distance Almost every day you face a problem: You must leave your home and go to school. If you are like me, you are usually a little late, so you want to take

### Graph Theory for Articulated Bodies

Graph Theory for Articulated Bodies Alba Perez-Gracia Department of Mechanical Engineering, Idaho State University Articulated Bodies A set of rigid bodies (links) joined by joints that allow relative

### Incremental Routing Algorithms For Dynamic Transportation Networks

UCGE Reports Number 20238 Department of Geomatics Engineering Incremental Routing Algorithms For Dynamic Transportation Networks (URL: http://www.geomatics.ucalgary.ca/research/publications/gradtheses.html)

### Asynchronous Computations

Asynchronous Computations Asynchronous Computations Computations in which individual processes operate without needing to synchronize with other processes. Synchronizing processes is an expensive operation

### CROSS LAYER BASED MULTIPATH ROUTING FOR LOAD BALANCING

CHAPTER 6 CROSS LAYER BASED MULTIPATH ROUTING FOR LOAD BALANCING 6.1 INTRODUCTION The technical challenges in WMNs are load balancing, optimal routing, fairness, network auto-configuration and mobility

### Unit 4: Layout Compaction

Unit 4: Layout Compaction Course contents Design rules Symbolic layout Constraint-graph compaction Readings: Chapter 6 Unit 4 1 Design rules: restrictions on the mask patterns to increase the probability

### CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992

CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992 Original Lecture #6: 28 January 1991 Topics: Triangulating Simple Polygons

### Divide-and-Conquer. Three main steps : 1. divide; 2. conquer; 3. merge.

Divide-and-Conquer Three main steps : 1. divide; 2. conquer; 3. merge. 1 Let I denote the (sub)problem instance and S be its solution. The divide-and-conquer strategy can be described as follows. Procedure

### 10. Graph Matrices Incidence Matrix

10 Graph Matrices Since a graph is completely determined by specifying either its adjacency structure or its incidence structure, these specifications provide far more efficient ways of representing a

### 5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

### Scheduling Shop Scheduling. Tim Nieberg

Scheduling Shop Scheduling Tim Nieberg Shop models: General Introduction Remark: Consider non preemptive problems with regular objectives Notation Shop Problems: m machines, n jobs 1,..., n operations

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### The PRAM Model and Algorithms. Advanced Topics Spring 2008 Prof. Robert van Engelen

The PRAM Model and Algorithms Advanced Topics Spring 2008 Prof. Robert van Engelen Overview The PRAM model of parallel computation Simulations between PRAM models Work-time presentation framework of parallel

### Graph Algorithms using Map-Reduce

Graph Algorithms using Map-Reduce Graphs are ubiquitous in modern society. Some examples: The hyperlink structure of the web 1/7 Graph Algorithms using Map-Reduce Graphs are ubiquitous in modern society.

### Algorithms and Data Structures (INF1) Lecture 14/15 Hua Lu

Algorithms and Data Structures (INF1) Lecture 14/15 Hua Lu Department of Computer Science Aalborg University Fall 2007 This Lecture Shortest paths Problem preliminary Shortest paths in DAG Bellman-Moore

### Lecture Notes on Spanning Trees

Lecture Notes on Spanning Trees 15-122: Principles of Imperative Computation Frank Pfenning Lecture 26 April 26, 2011 1 Introduction In this lecture we introduce graphs. Graphs provide a uniform model

Load Balancing Backtracking, branch & bound and alpha-beta pruning: how to assign work to idle processes without much communication? Additionally for alpha-beta pruning: implementing the young-brothers-wait

### SCAN: A Structural Clustering Algorithm for Networks

SCAN: A Structural Clustering Algorithm for Networks Xiaowei Xu, Nurcan Yuruk, Zhidan Feng (University of Arkansas at Little Rock) Thomas A. J. Schweiger (Acxiom Corporation) Networks scaling: #edges connected

### Notes on Matrix Multiplication and the Transitive Closure

ICS 6D Due: Wednesday, February 25, 2015 Instructor: Sandy Irani Notes on Matrix Multiplication and the Transitive Closure An n m matrix over a set S is an array of elements from S with n rows and m columns.

### OPTIMAL DESIGN OF DISTRIBUTED SENSOR NETWORKS FOR FIELD RECONSTRUCTION

OPTIMAL DESIGN OF DISTRIBUTED SENSOR NETWORKS FOR FIELD RECONSTRUCTION Sérgio Pequito, Stephen Kruzick, Soummya Kar, José M. F. Moura, A. Pedro Aguiar Department of Electrical and Computer Engineering

### Simplified External memory Algorithms for Planar DAGs. July 2004

Simplified External Memory Algorithms for Planar DAGs Lars Arge Duke University Laura Toma Bowdoin College July 2004 Graph Problems Graph G = (V, E) with V vertices and E edges DAG: directed acyclic graph

### Distributed Computing over Communication Networks: Maximal Independent Set

Distributed Computing over Communication Networks: Maximal Independent Set What is a MIS? MIS An independent set (IS) of an undirected graph is a subset U of nodes such that no two nodes in U are adjacent.

### Matrix Product. Matrix-Multiply (A, B) Cost of multiplying these matrices is p q r.

Dynamic Programming A powerful paradigm for algorithm design. Often leads to elegant and efficient algorithms when greedy or diide-and-conquer don t work. DP also breaks a problem into subproblems, but

CSE / Notes : Task Scheduling & Load Balancing Task Scheduling A task is a (sequential) activity that uses a set of inputs to produce a set of outputs. A task (precedence) graph is an acyclic, directed

### Any two nodes which are connected by an edge in a graph are called adjacent node.

. iscuss following. Graph graph G consist of a non empty set V called the set of nodes (points, vertices) of the graph, a set which is the set of edges and a mapping from the set of edges to a set of pairs

### DO NOT RE-DISTRIBUTE THIS SOLUTION FILE

Professor Kindred Math 04 Graph Theory Homework 7 Solutions April 3, 03 Introduction to Graph Theory, West Section 5. 0, variation of 5, 39 Section 5. 9 Section 5.3 3, 8, 3 Section 7. Problems you should

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### A linear combination is a sum of scalars times quantities. Such expressions arise quite frequently and have the form

Section 1.3 Matrix Products A linear combination is a sum of scalars times quantities. Such expressions arise quite frequently and have the form (scalar #1)(quantity #1) + (scalar #2)(quantity #2) +...

### COURSE: B.TECH-ECE. IV Sem. Data structure Using C. b) Deletion of element in an array

COURSE: B.TECH-ECE. IV Sem Data structure Using C 1. Determine the formula to find the address location of an element in three dimensions array, suppose each element takes four bytes of space & elements

### Part 2: Community Detection

Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection - Social networks -

### System Interconnect Architectures. Goals and Analysis. Network Properties and Routing. Terminology - 2. Terminology - 1

System Interconnect Architectures CSCI 8150 Advanced Computer Architecture Hwang, Chapter 2 Program and Network Properties 2.4 System Interconnect Architectures Direct networks for static connections Indirect

### 1. What s wrong with the following proofs by induction?

ArsDigita University Month : Discrete Mathematics - Professor Shai Simonson Problem Set 4 Induction and Recurrence Equations Thanks to Jeffrey Radcliffe and Joe Rizzo for many of the solutions. Pasted

Load Balancing Techniques 1 Lecture Outline Following Topics will be discussed Static Load Balancing Dynamic Load Balancing Mapping for load balancing Minimizing Interaction 2 1 Load Balancing Techniques

### DYNAMIC GRAPH ANALYSIS FOR LOAD BALANCING APPLICATIONS

DYNAMIC GRAPH ANALYSIS FOR LOAD BALANCING APPLICATIONS DYNAMIC GRAPH ANALYSIS FOR LOAD BALANCING APPLICATIONS by Belal Ahmad Ibraheem Nwiran Dr. Ali Shatnawi Thesis submitted in partial fulfillment of

### Graph definition Degree, in, out degree, oriented graph. Complete, regular, bipartite graph. Graph representation, connectivity, adjacency.

Mária Markošová Graph definition Degree, in, out degree, oriented graph. Complete, regular, bipartite graph. Graph representation, connectivity, adjacency. Isomorphism of graphs. Paths, cycles, trials.

### Systems and Algorithms for Big Data Analytics

Systems and Algorithms for Big Data Analytics YAN, Da Email: yanda@cse.cuhk.edu.hk My Research Graph Data Distributed Graph Processing Spatial Data Spatial Query Processing Uncertain Data Querying & Mining

### Lesson 3. Algebraic graph theory. Sergio Barbarossa. Rome - February 2010

Lesson 3 Algebraic graph theory Sergio Barbarossa Basic notions Definition: A directed graph (or digraph) composed by a set of vertices and a set of edges We adopt the convention that the information flows

### Zachary Monaco Georgia College Olympic Coloring: Go For The Gold

Zachary Monaco Georgia College Olympic Coloring: Go For The Gold Coloring the vertices or edges of a graph leads to a variety of interesting applications in graph theory These applications include various