CSE 101. Algorithm Design and Analysis Miles Jones Office 4208 CSE Building Lecture 11: MST

Transcription

1 CSE 101 Algorithm Design and Analysis Miles Jones Office 4208 CSE Building Lecture 11: MST

2 GREEDY-STAYS-AHEAD In this proof technique, let s revisit the original event scheduling problem. We want to show that at every step the greedy solution stays ahead of any other solution. Let O = [E j1,, E jm ] be any optimal solution and Let A = [E i1,, E ik ] be a solution returned using the greedy algorithm. We wish to show that k = m.

3 GREEDY-STAYS-AHEAD Let O = [E j1,, E jm ] be any optimal solution and Let A = [E i1,, E ik ] be a solution returned using the greedy algorithm. The greedy method guarantees that:

4 GREEDY-STAYS-AHEAD Let O = [E j1,, E jm ] be any optimal solution (ordered by finish time.) Let A = [E i1,, E ik ] be a solution returned using the greedy algorithm. The greedy method guarantees that: f(i 1 ) f(j 1 ) We want to show that for all indices r k that: f(i r ) f(j r ) (page ) Algorithm Design by Kleinberg and Tardos

5 GREEDY-STAYS-AHEAD Let O = [E j1,, E jm ] be any optimal solution (ordered by finish time.) Let A = [E i1,, E ik ] be a solution returned using the greedy algorithm. To demonstrate using proof by contradiction, assume that m > k Then we can show that since f i k f j k, for event E jk+1 we havef i k f(j k ) s j k+1, so the greedy strategy would pick E jk+1 after E ik, contradicting the fact that the greedy strategy only output k events. Therefore m = k and the greedy strategy is optimal.

6 SPANNING TREES A spanning tree of an undirected graph G=(V,E) is a subgraph G =(V,E ) such that G is a tree and all vertices in V are connected. An output tree of DFS or BFS is a spanning tree.

7 EXAMPLE Suppose you have a network of computers that were linked pairwise Suppose each link has a positive maintenance cost. Your job is to cut some links so that the cost of the network is minimized and the network stays connected.

8 EXAMPLE 1. The minimized network must be a tree 2. removing a cycle edge does not disconnect a graph

9 THE MINIMIZED GRAPH IS A TREE Proof: Suppose by contradiction that the subgraph with minimum weight that is still connected was not a tree. Then there would exist a cycle somewhere in the subgraph. Removing a cycle edge does not disconnect a graph. Each edge has a positive weight value so removing any cycle edge will not disconnect the graph and decrease the total weight contradicting the original claim.

10 EXAMPLE This subgraph has a cost of 12. Is this the minimum cost?

11 EXAMPLE Let s try a greedy approach Is this the minimum cost?

12 KRUSKAL S ALGORITHM FOR FINDING THE MINIMUM SPANNING TREE Start with a graph with only the vertices. Repeatedly add the next lightest edge that does not form a cycle.

13 CORRECTNESS OF KRUSKAL S cut property. Suppose G = V, T is a MST of G = V, E and suppose that X T. Pick any subset of vertices S V such that there is no path from S to V S using edges from X. Let e E be the lightest edge that connect S to V S. Then: X {e} is part of some MST.

14 PROOF OF CUT PROPERTY Case 1: e T Then, X {e} T and we assumed that T is the set of edges of an MST so X {e} is part of that MST.

15 PROOF OF CUT PROPERTY Case 1: e T Then, X {e} T and we assumed that T is the set of edges of an MST so X {e} is part of that MST. Case 2: e T Consider T {e}. Since T is the edge set of a connected tree, T {e} is the edgeset of a graph that has a cycle. Furthermore, the cycle contains e.

16 PROOF OF CUT PROPERTY Case 2: e T (continued ) Consider T {e}. Since T is the edge set of a connected tree, T {e} is the edgeset of a graph that has a cycle. Furthermore, the cycle contains e. there must be another edge e T that connects S to V S. Consider the edgeset T e {e }. This is still a tree.

17 PROOF OF CUT PROPERTY cost T e {e } = cost T + w e w e w e w e 0 w e w e cost T e {e } cost T But T is the edgeset of an MST so it is minimal. Therefore cost T e {e } must be minimal also and we have that cost T e {e } = cost T so T e {e } is the edgeset of an MST. X e T e {e }

18 FACTS ABOUT TREES Definition: A tree is an undirected connected graph with no cycles. An undirected connected graph is a tree if and only if removing any edge results in two disconnected graphs. An undirected connected graph with n vertices is a tree if and only if it has n -1 edges. An undirected connected graph is a tree if and only if there is a unique path between nodes

19 FACTS ABOUT TREES (PROOFS) Theorem: An undirected connected graph is a tree if and only if removing any edge results in two disconnected graphs.

20 FACTS ABOUT TREES (PROOFS) Theorem: An undirected connected graph is a tree if and only if removing any edge results in two disconnected graphs. proof: Suppose by contradiction, I can remove an edge, say (u,v), and have the graph stay connected, Then after removing the edge, there must still be a path P from v to u because the resulting graph is still connected. But in the original graph, P+(u,v) is a cycle. Contradicting the fact that the graph is a tree.

21 FACTS ABOUT TREES An undirected connected graph with n vertices is a tree if and only if it has n -1 edges.

22 FACTS ABOUT TREES An undirected connected graph with n vertices is a tree if and only if it has n -1 edges. proof(=>): Suppose a graph with n vertices is a tree. Then if we remove an edge, the resulting graph will be broken into two disconnected trees. repeatedly remove edges one by one from the graph and at each removal, the resulting graph will add one more connected component. Do this until the graph has n isolated vertices and no edges. This took n-1 many steps. So the graph must have had n-1 edges to start with.

23 FACTS ABOUT TREES An undirected connected graph with n vertices is a tree if and only if it has n -1 edges. proof(<=): Suppose an undirected connected graph G has exactly n-1 edges. By contradiction, suppose G is not a tree and thus has a cycle. While G contains a cycle, repeatedly remove a cycle edge until G is acyclic. Then the resulting graph is a tree and thus has n-1 edges. But the graph had n-1 edges to start with so no edges were removed and therefore the graph was a tree all along!!!!!

24 FACTS ABOUT TREES An undirected connected graph is a tree if and only if there is a unique path between nodes.

25 FACTS ABOUT TREES An undirected connected graph is a tree if and only if there is a unique path between nodes. proof:(=>) Suppose G is a tree. Then since it is connected, there is a path between each node. Suppose by contradiction that there exists a pair of nodes, u,v; such that there are two paths from u to v, namely P1 and P2. Then P1+reverse(P2) is a cycle, contradicting the fact that G is a tree.

26 FACTS ABOUT TREES An undirected connected graph is a tree if and only if there is a unique path between nodes. proof:(<=) Suppose there is a unique path between any two nodes in an undirected connected graph G. Suppose by contradiction that G is not a tree Then G has a cycle: v0,v1,,vk,v0 Then there is a path from v0 to vk by going through v1,v2, There is also an edge from v0 to vk. Therefore there are two distinct paths between v0 and vk.

27 KRUSKAL S ALGORITHM FOR FINDING THE MINIMUM SPANNING TREE Start with a graph with only the vertices. Repeatedly add the next lightest edge that does not form a cycle.

28 HOW TO IMPLEMENT KRUSKAL S How do we efficiently tell if adding an edge will create a cycle?

29 HOW TO IMPLEMENT KRUSKAL S How do we efficiently tell if adding an edge will create a cycle? What if we kept track of the connected components? Then if we connect two vertices in the same connected component, it will create a cycle. And if we connect two vertices in different connected components, it will not create a cycle.

30 KRUSKAL S ALGORITHM procedure kruskal(g,w) input: undirected graph G with edge weights w output: A set of edges X that defines a minimum spanning tree for all u V makeset(u) X = {} sort the edges E in increasing order by weight for all edges (u, v) E if find(u) find(v): add edge (u, v) to X union(u, v)

31 CORRECTNESS OF KRUSKAL S Loop invariant: After k iterations of the last loop, X is a subset of some MST. Proof of Loop invariant:

32 CORRECTNESS OF KRUSKAL S Loop invariant: After k iterations of the last loop, X is a subset of some MST. Proof of Loop invariant: Base case: X={} is trivially a subset of some MST Suppose after k iterations, X is a subset of some MST. Then doing one more iteration, we find the lightest edge e that does not create a cycle. That means that adding this edge to X will connect the disconnected sets S and V-S. Furthermore, this is the lightest edge connecting those sets so X {e} is a subset of some MST!!!

33 CORRECTNESS OF KRUSKAL S Loop invariant: After k iterations of the last loop, X is a subset of some MST. The algorithm terminates when all edges have been considered. No new edge will be added after X is a tree. We know that X is always a subset of some MST, so if X is a tree, it must be an MST!!!! Therefore, Kruskal s algorithm will always output a MST.

34 SUBROUTINES OF KRUSKAL S makeset (u): This creates a set with one element, u find(u): finds the set to which u belongs union(u,v): merges the sets containing u and v Runtime of Kruskal s: procedure kruskal(g,w) input: undirected graph G with edge weights w output: A set of edges X that defines a minimum spanning tree for all u V makeset(u) X = {} sort the edges E in increasing order by weight for all edges (u, v) E if find(u) find(v): add edge (u, v) to X union(u, v)

35 SUBROUTINES OF KRUSKAL S makeset (u): This creates a set with one element, u find(u): finds the set to which u belongs union(u,v): merges the sets containing u and v Runtime of Kruskal s: V makeset + 2 E find + V 1 union + sort( E )

36 EXAMPLE

37 A DATA STRUCTURE FOR KRUSKAL S (DIRECTED TREES WITH RANKS) vertices of the trees are elements of a set and each vertex points to its parent that eventually points to the root. The root points to itself. The root is a convenient representation or name of the set containing it and all of its children. in addition to the parent pointer of x, π(x), each vertex also has a rank that tells you the height of the subtree hanging from that vertex.

38 A DATA STRUCTURE FOR KRUSKAL S (DIRECTED TREES WITH RANKS)

39 A DATA STRUCTURE FOR KRUSKAL S (DIRECTED TREES WITH RANKS) rank(a)=1 rank(b)=0 rank(c)=0 rank(d)=2 rank(e)=0 rank(f)=0 rank(g)=1 π A = D π B = D π C = G π D = D π E = A π F = A π G = G

40 SUBROUTINES. procedure makeset(x) π x : = x rank(x):=0

41 SUBROUTINES. procedure find(x) while x π x x: = π x return x

42 SUBROUTINES. procedure union(x,y) rx:=find(x) ry:=find(y) if rx=ry then return if rank(rx)>rank(ry) then else π ry : = rx π ry : = rx if rank(rx)=rank(ry) then rank(ry):=rank(rx)+1 To save on runtime, we must keep the heights of the trees short. So union of two ranks points the smaller rank to the bigger rank, that way, the tree will stay the same height. If the ranks are equal, then it increments one rank and points the smaller to the bigger. (this is the only way a rank can increase.)

43 SUBROUTINES. procedure makeset(x) π x : = x rank(x):=0 procedure find(x) while x π x x: = π x return x makeset=o(1) find=o(height of tree containing x) union=o(find) procedure union(x,y) rx:=find(x) ry:=find(y) if rx=ry then return if rank(rx)>rank(ry) then else π ry : = rx π ry : = rx if rank(rx)=rank(ry) then rank(ry):=rank(rx)+1

44 EXAMPLE makeset({a,b,c,d,e,f,g}) union(a,d), union(b,e), union(b,f), union(a,g), union(d,g),union(b,d), union(c,e)

45 EXAMPLE makeset({a,b,c,d,e,f,g}) union(b,c), union(e,g), union(d,f), union(a,b), union(d,g),union(a,f)

46 EXAMPLE

47 HEIGHT OF TREE any root node of rank k has at least 2 k vertices in its tree. proof: Base Case: a root of rank 0 has 1 vertex suppose a root of rank k has at least 2 k vertices in its tree Then a root of rank k+1 can only be made by unioning 2 roots each of rank k so a root of rank k+1 must have at least 2 k + 2 k = 2 k+1 vertices in its tree.

48 ANCESTORS OF RANK K any vertex has at most one ancestor of rank k proof: each vertex has one pointer and ranks strictly increase along paths so each element has at most one ancestor of each rank.

49 NUMBER OF VERTICES OF A GIVEN RANK if there are n vertices overall, there can be at most n 2k vertices of rank k proof: each rank k vertex has at least 2 k vertices in its tree so if there are m rank k vertices then there are m2 k vertices overall. if there are n vertices overall then m2 k n so m n 2 k

50 HEIGHT OF TALLEST TREE (MAXIMUM RANK) the maximum rank is log(n) proof: (sort of) How many vertices of rank log(n) can there be?

51 HEIGHT OF TALLEST TREE (MAXIMUM RANK) the maximum rank is log(n) proof: (sort of) How many vertices of rank log(n) can there be? n = 1 2log (n) anything more is not possible.

52 RUNTIME makeset=o(1) find=o(height of tree containing x) union=o(find) makeset (u): This creates a set with one element, u find(u): finds the set to which u belongs union(u,v): merges the sets containing u and v Runtime of Kruskal s: V makeset + 2 E find + V 1 union + sort( E )

53 RUNTIME makeset=o(1) find=o(log(n)) union=o(log(n)) makeset (u): This creates a set with one element, u find(u): finds the set to which u belongs union(u,v): merges the sets containing u and v Runtime of Kruskal s: V makeset + 2 E find + V 1 union + sort( E ) V O E O log V + V 1 O log V + E log E O( V + E log V + V log V + E log E ) O E log( V )