Algorithms and Data Structures (INF1) Lecture 14/15 Hua Lu Department of Computer Science Aalborg University Fall 2007
This Lecture Shortest paths Problem preliminary Shortest paths in DAG Bellman-Moore algorithm Dijkstra s algorithm 2
Motivation Example What is the shortest path from Århus to Skagen? Århus -> Aalborg -> Frederikshavn -> Skagen Total cost: 4 hours This example is not so hard to figure out Only 4 possibilities Consider cases with Large number of nodes Multiple edge or path choices between the same pair of cities 1.5 Hjørring 1 1.5 Viborg Skagen Aalborg 2 1 Frederikshavn 1 1 Århus 3
Shortest Path Problem Given a digraph whose edges are attached with costs, two of its vertices are specified as a and b Find a shortest path from a to b (if there exists one) s.t. the total cost of that path is the minimal This cost is called distance, denoted as d(a, b) Assumptions Edges may have negative costs Cycles with negative cost are not allowed If such a cycle is found, report it and stop the shortest path finding 4
Triangle Inequality If v is reachable from u, and w is reachable from v, we have d(u, w) d(u, v) + d(v, w) Proof d(u, v) indicates the shortest path from u to v d(v, w) indicates the shortest path from v to w d(u, v) + d(v, w) indicates one path from u to w A shortest path from u to w must exist, with cost d(u, w) d(u, w) d(u, v) + d(v, w) holds as d(u, w) indicates the shortest path! If = holds, v is on the shortest path from u to w In particular, if there is an edge between v and w, then d(u, w) d(u, v) + c(v, w) 5
Shortest Path Spanning Tree Given a digraph G with edge costs, each vertex is reachable from some start vertex a. Then, either G contains a negative cycle, or else it contains a spanning tree T, rooted at a, such that for each vertex v in G, the total cost of the path through T from a to v is equal to d(a, v) T reserves shortest path information Skagen 1 Such a tree is called a shortest path spanning tree E.g., start from Århus Hjørring 1 Aalborg Frederikshavn 1 Vibor g 1 2 Århus 6
Single Source Shortest Paths A single start vertex has been specified We call it a single source shortest path problem Three main algorithms exist for such problems Nodes # n, edge # m Topological sorting Dijkstra algorithm Bellman-Moore algorithm 7
A Simple Recurrence The input digraph can have negative costs, but no negative cycles Still positive cycles possibly! Recurrence like the one we used in critical path d(a, a) = 0 d(a, v) = min{d(a, u) + c(u,v)}, u Pred(v) Pred(v) denotes the set of direct predecessors of v i.e. <u, v> is an edge The recurrence cannot be used due to possible cycles! For d(a, v), v can be involved in a cycle, recursion on v can be forever If G is a DAG, then this recurrence works as the basis for the topological sorting based algorithm of O(n+m) 8
Bellman-Moore Algorithm d k (a, v) = the cost of a shortest path from a to v among all paths of length at most k Number of edges involved k If no such a path from a to v of length at most k, d k (a, v)= New recurrence d 0 (a, a) = 0 d 0 (a, v) = d k (a, v) = min{d k-1 (a, v), min u Pred(v) {d k-1 (a, u) + c(u, v)}} Bellman-Moore algorithm Repeatedly compute d k (a, v) for k = 0, 1, 2, For a digraph with n nodes, the largest k to consider is n-1 9
Complexity of Bellman-Moore Alg k=0 k=1 k=n-1 a=v 1 0 v 2 v n We need to compute this table column by column Dynamic programming! Cost to compute one d k (a, v) is O( Pred(v) ) Cost to compute one column is O(n+m) We have O(n) columns Total cost is O(n (n+m)) in the worst case 10
Negative Cycle Detection How to decide whether a digraph has a negative cycle or not? Use Bellman-Moore algorithm and calculate also the column k=n The graph has a negative cycle if and only if d n (a, v) < d n-1 (a, v) for some vertex v 11
Dijkstra s Algorithm Preliminaries Assumption No edge has negative cost Priority queue Like a symbol table, a set of entries Each entry has form of <key, value> Special operations find_min:entry_type Returns the entry in the priority queue whose key is minimum delete_min:entry_type Deletes and returns the entry that would be returned by find_min We use a priority queue when we are more concerned about items with extreme keys than the total order among all keys In shortest path finding, we care about the shortest path so far, not the order of all possible paths w.r.t. their lengths 12
Dijkstra s Algorithm Each vertex has fields v.visited, v.parent, v.distance A priority queue q When one vertex v is inserted into q, we regard v.distance as the key and v itself as the value, i.e., <v.distance, v> dijkstra(g:digraph, a:vertex_type) 1. for each vertex v, set v.visited:=false, v.distance:= 2. a.distance:=0; a.visited:=true; a.parent:=nil; q.insert(a) 3. while NOT q.empty do v:=q.delete_min for each edge (v, w) do if NOT w.visited then w.visited:=true; w.parent:=v; w.distance:=v.distance+c(v, w); q.insert(w) else if v.distance+c(v, w) < w.distance then w.distance:=v.distance+c(v, w); w.parent:=v 13
Correctness of Dijkstra s Algorithm v a x L: vertices Q: vertices R: removed from Q Loop Invariant z L: z.distance=d(a, z) still in Q z L: each successor of z is either in L or in Q z Q: z.distance is the length of one shortest path from node a to node z via vertices in L Prove it by induction on k, loop iteration vertices not visited yet 14
Complexity of Dijkstra s Algorithm Cost of step 1 is O(n) Cost of step 2 is O(1) Cost of step 3 is O(n+m) + time cost on the priority queue delete_min: n times insert: n times distance update: m times Total cost O(n + m + n (T(delete_min)) + n (T(insert)) + m (T(dist_update))) Priority queue implementation Linked list Fibonacci heap delete_min O(n) O(log n) insert O(1) O(1) distance update O(1) O(1) Total time cost O(n 2 ) O(n log n + m) 15
Shortest Paths Summary Arbitrary digraphs Use Bellman-Moore algorithm O(n (n+m)) O(n 3 ) if m=θ(n 2 ) Digraphs with non-negative costs Use Dijkstra s algorithm O(n log n+m) O(n 2 ) if m=θ(n 2 ) DAGs Use topological sorting based algorithm O(n+m) Better than Dijkstra s algorithm, but still O(n 2 ) if m=θ(n 2 ) 16
Next Lecture Minimum spanning trees Definitions Kruskal s algorithm Prim s algorithm Course conclusion 17