A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points

Transcription

1 A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points Problem 1. Suppose that we insert n keys into a hash table of size m using open addressing and uniform hashing. Let p(n, m) be the probability that no collisions occur. Show that p(n, m) exp( n(n 1)/(2m)). Argue that when n exceed m, the probability of avoiding collisions goes rapidly to zero. Hint: Use induction... Also use exp(x) 1 + x for any real x. Note that exp(x) = e x. Problem 2. Suppose that we are given a key k to search for in a hash table with positions 0,..., m 1, and suppose that we have a hash function h mapping the key space into the set {0,..., m 1}. The search scheme is as follows. 1. Compute the value i = h(k) and set j = Probe in position i for the desired k. If you find it, or if this position is empty, terminate the search. 3. Set j = (j + 1) mod m and i = (i + j) mod m and return to step 2. Assume that m is a power of 2. a. Show that this scheme is an instance of the general quadratic probing scheme by exhibiting the appropriate constant c 1, c 2 for h(k, i) = (h(k) + c 1 i + c 2 i 2 ) mod m. b. Prove that this algorithm examines every table position in the worst case. Problem 3. Given two strings a = a 0 a 1... a k and b = b 0 b 1... b m, where each a i and b j belongs to some ordered set of characters (e.g. english alphabet, binary set) we say that string a is lexicographically less than string b if either 1. there exists an integer j, where 0 j min(k, m), such that a i = b i for all i = 0,..., j 1 and a j < b j, or 2. k < m and a i = b i for all i = 0,..., k Black / \ 0 / \ 1 / \ / \ --/ Red Black Figure: A binary radix tree. \ 1 0 / Black Red /--\- \ 1 0/ \ / Red Red Black \ 1 ---\-- Red For example, 010 is lexicographically less than 0100 which is less than A binary radix tree is a tree built out of a collection of binary strings and one is depicted in the attached Figure. In a binary radix tree an edge leading to a left child can be labeled 0 and to a right child can be labeled 1. A node is either black or red; a black node indicates that the string (using the implied edge labels) corresponding to the path from the root to the black node is NOT in the tree, whereas a red node indicates that the string is in the tree. When searching for a in such a tree we traverse a left child at depth i if a i = 0, and a right child if a i = 1. If a red node is reached, a is in the tree, otherwise it is not. Searching for 101 leads to a black node and for 1011 to a red node: the former string does not exist, the latter does. Let A be a set of distinct binary strings whose total length sum is n. Show how to use a radix tree to sort A in lexicographic order in Θ(n) time. Sorting the tree above with n = 13 would give the strings 0, 011, 10, 100, 1011 in lexicographic order. 1

2 Problem 4. Consider inserting keys 1, 3, 5, 6, 12, 23, 10, 17, 34 into a hash table of length N = 11 using open addressing with the primary hash function h (k) = k mod N. Illustrate the result of inserting these keys using (i) linear probing with hash function h(k, i) = (h (k) + i) mod N, (ii) quadratic probing with hash function h(k, i) = (h (k) + i 2 ) mod N. Problem 5. The maximum-spanning tree problem is defined similarly to the minimum-spanning tree problem except that we want to maximize the sum of the weights of the edges of the tree. Give an algorithm that finds the maximum-spanning tree of an undirected graph G = (V, E, W ) where edges have weights represented in the weight matrix W. What is the running time of the algorithm? Problem 6. You are given six polynomials f 1,... f 6 of degrees 1, 2, 3, 1, 4, 5 respectively. We are interested in finding the product f = f 1 f 2 f 3 f 4 f 5 f 6. Assume that the cost of multiplying two polynomials of degree a and b is a b. Find a schedule for multiplying the six polynomials that is of the lowest possible total cost. Problem 7. a. What is the largest k such that if you can multiply 3 3 matrices using k multiplications (not assuming commutativity of multiplication), then you can multiply n n matrices in time o(n lg 7 )? What would the running time of this algorithm be? b. V. Pan has discovered a way of multiplying matrices using multiplications, matrices using multiplications, matrices using multiplications. Which method yields the best asymtotic running time when used in a divide and conquer matrix multiplication algorithm? How does it compare to Strassen s algorithm? Problem 8. You are given a directed graph with positive weights on its edges. Show how you can solve the minimum cost directed cycle problem in that graph in time faster than o(n 4 ) (this is a little-oh) in the worst-case. A directed cycle is a directed path that starts and ends in the same vertex and has at least one edge. A minimum cost directed cycle is one for which the sum of the weights of the edges of the cycle is minimal. Problem 9. How would you detect the existence of a negative-weight cycle in Floyd-Warshall? Problem 10. You are given an acyclic connected and undirected graph G = (V, E). Can you determine in O(n) time whether it has or not a cyle (note that the bound involves only n but not m)? Problem 11. You are given an undirected graph G = (V, E). Can you determine in O(n) time whether it has or not a cyle (note that the bound involves only n but not m)? Assume an adjacency list representation of the graph. Problem 12. We have an undirected weighed graph whose edges have weights that are only 10 or 20. Can you find the minimum spanning tree of this graph in O(n + m) time? Explain. Problem 13. What s the best way to find the product M 0 M 1 M 2 M 3 where the matrices are of dimension , , , and respectively? (In Problem 6, you had to deal with polynomials. The polynomials have now become matrices.) 2

3 PS 3 Solution Outline Problem 1. Consider p(n + 1, m) and argue similarly to the derivation for Insertion done in class. For the n + 1-sty key not to cause any collision in a hash table with n keys inserted with no collisions, it must hit an empty slot an event that occurs with probability 1 n/m. Therefore p(n + 1, m) = (1 n/m)p(n, m), since the probability that the insertion of say n + 1 cause not collision is the probability that the insertion of the n + 1-st key causes no collisions and the prior insertion of the remaining n keys caused no collisions; the latter is p(n, m). Therefore p(n + 1, m) = (1 n/m)p(n, m) = (1 n/m)(1 (n 1)/m)p(n 1, m) = (1 n/m)(1 (n 1)/m)... (1 1/m)(1 0/m)p(0, m) = (1 n/m)(1 (n 1)/m)... (1 1/m) Since e x 1 + x, setting x = i/m we obtain that 1 i/m e i/m. Therefore p(n + 1, m) = (1 n/m)(1 (n 1)/m)... (1 1/m) exp( n/m) exp( (n 1)/m)... exp( 1/m) exp( ( n)/m) = exp( n(n + 1)/2m) Therefore p(n, m) exp( (n 1)n/(2m)) as required. Consider n m, the n(n 1)/2m 1, and thus p(n, m) is upper bounded by exp( A), where A is large and positive. However exp( A) goes to zero and thus p(n, m) goes to zero as well. This problem is a variant known as the Birthday problem. Let m = 365 (forget about leap years). Suppose you have n people in a room. What is the probability of having no two with the same birthday? For what value of n is the probability of having two people with the same birthday close to 1? One can see that for n = m collisions start to show up with a significant (non negligible) probability, i.e. people exist with the same birthday. Problem 2. a. The hashing scheme searches entries h(k), h(k) + 1, h(k) , h(k) ,..., h(k) j modm, where m is a power of two. The value of i after j iterations (i.e. after j probes) is i = h(k, j) = h(k) j = h(k) + j(j + 1)/2 = h(k) + 1/2j + 1/2j 2 mod m, i.e. we indeed have a quadratic probing scheme with c 1 = c 2 = 1/2 and hash function h(k, j). b. (Proof by contradiction). Suppose that the algorithm does not examine every table position in the m probes h(k, j), j = 0,..., m 1. This means that at least one table entry is visited at least twice (and at least one is not visited at all) i.e. there exist 0 l, r m 1 such that h(k, l) = h(k, r), l r and therefore h(k, l) h(k, r) 0 (mod m). Without loss of generality, we assume l < r. We shall then show that for l, r as above we have that h(k, l) h(k, r), and therefore h(k, l) h(k, r) 0 (mod m), i.e. a contradiction. It is true that h(k, r) h(k, l) = r(r + 1)/2 l(l + 1)/2 = (r l)(r + l + 1)/2 0 (mod m). Consider (r l)(r + l + 1)/2 0 (mod m) or in the other words. (r l)(r + l + 1)/2 = Am for some integer A. For this to be true, m or a power of two must divide r l or r + l + 1 or both. We distinguish two major cases. 1. Let r l be even (i.e. r, l are both odd or both even). Then, r + l + 1 is odd and it is not divisible by m or 2 as both m (a power of 2) and 2 are even and r +l +1 was shown to be odd. On the other hand, r l m 1 as r, l m 1 < m. Then (r l)/2 < m and therefore m cannot divide (r l)/2 either. This means that (r l)(r +l +1)/2 0 (mod m), since m can divide neither r + l + 1 nor r l. 2. Let r l be odd. Then, if one of r, l is odd, the other one is even, and the sum r + l + 1 is even. As r l is odd it is not divisible by m or 2. We shall show that (r + l + 1)/2 is not divisible by m as well. As r, l m 1, r + l + 1 2(m 1) + 1 = 2m 1. r + l + 1 is even, 2m 1 is odd, thus r + l + 1 can not be equal to 2m 1 and therefore r + l + 1 2m 2 = 2(m 1). Then (r + l + 1)/2 m 1 < m, and therefore it is not divisible by m (as this would imply that (r + l + 1)/2 m). This means that (r l)(r + l + 1)/2 0 (mod m) as well. For both cases we derived a contradiction h(k, l) h(k, r) = (r l)(r + l + 1)/2 0 (mod m) to the assumption that h(k, l) h(k, r) 0 (mod m). Since no two values among the 0 through m 1 collide with each other, the cover all the values/indices/slots of the hash table as required. 3

4 Problem 3. First insert the strings and then perform a preorder traversal. In the preorder traversal a string is printed only for the red nodes (not for the black nodes that do not correspond to strings). Note that the preorder is the correct traversal as a node is printed before its descendants. A printed node x is a red node and the string corresponding to that node should be printed before the strings corresponding to its descendants (in the left or right subtree of x). Strings corresponding to nodes of x s left subtree must be printed before the ones corresponding to nodes of x s right subtree as up to node x both strings are the same, but just after x the strings in the left subtree have a leading zero and the ones in the right subtree have a leading 1; the former must be printed before the latter ones. Preorder traversal guarantees that. Running time of algorithm is O(n), as insertion takes time proportional to length of individual string being inserted and sum of all strings is n. Problem 4. coll@a,b,c means a,b,c keys caused collision in the indicated slot. LinearProbing Quadratic Probing Slot 0 : 10 1 : 1 coll@12,23,34 1 coll@12,23,34 2 : 12 coll@23,34 12 coll@23,34 3 : 3 coll@23, : 23 coll@ : 5 coll@34 5 coll@23,34 6 : 6 coll@17,34 6 coll@17,34 7 : 17 coll@ : 34 9 : 10 : coll@10,34 In the Linear probing case you can observe the long cluster that causes problems to 34. Observe that for quadratic probing the collisions are fewer (by one). Problem 5. If the graph is G = (V, E, W ) consider graph G = (V, E, W ) i.e. we negate the weights of all the edges in G. A maximum spanning tree T in G is a minimum spanning tree in G. Note that the conversion of W into W takes time O(m). If G is represented by an adjacency-matrix we find all the edges and negate the corresponding weights in O(n 2 ) time. If G is represented by an adjacency-list we go through all the edges in time O(m) and negate the corresponding weights. Whatever the cost is O(m lg m), the running time of Kruskal s algorithm will contribute more. The maximum-spanning tree problem is defined similarly to the minimum-spanning tree problem Problem 6. Multiply them any way you like. The final result is a polynomial of degree 16 and the total cost is 100. Note that 100 is the sum of all products d i d j for every combination of two polynomials f i and f j among the six and d i and d j are respectively the degrees of f i and f j. In our example we have 6(6 1)/2 such pairs of polynomials. 100 = d 1 (d 2 + d 3 + d 4 + d 5 + d 6 ) + d 2 (d 3 + d 4 + d 5 + d 6 ) + d 3 (d 4 + d 5 + d 6 ) + d 4 (d 5 + d 6 ) + d 5 d 6 = 1( ) + 2( ) + 3( ) + 1(4 + 5) = This sum is independent of the order of the evaluation of the products; an easy proof is by induction (eg. with reference to the 6 polynomials above, the product of five of them f 2,..., f 6 takes time equal to the 4 summands except the first one d 1 (d 2 + d 3 + d 4 + d 5 + d 6 ); in the last step of an inductive proof the polynomial f 1 whose degree is d 1 is multiplied by the result f 2... f 6 whose degree is d d 6 and the cost of that multiplication is the missing/absent (first) term. Permute the polynomial and the result remains the same except that the pairs of the products d i d j get rearranged.). Note. One of the reasons that this is so is that polynomials satisfy commutativity therefore f 1 f 2 = f 2 f 1. What happens if instead of polynomials you have matrices and the operation is matrix multiplication? The cost then depends on the specific schedule that you choose. [Check Subject 8 (to appear), pp. 4-7, on how to determine the optimal schedule using dynamic programming]. 4

5 Problem 7. a. Strassen s algorithm uses the pattern of multiplication for 2 2 matrices and divide-and-conquer by splitting a matrix multiplication of n n matrices into a matrix multiplication of n/2 n/2 matrices to derive a time bound for matrix multiplication given by the recurrence T (n) = 7T (n/2) + O(n 2 ). In such a recurrence the 7 indicates the number of scalar multiplications in multiplying 2 2 matrices or the number of n/2 n/2 matrix multiplications in the recursive (divide-and-conquer) decomposition. Note that Strassen s method on 2 2 matrices can not be improved. We cannot multiply 2 2 matrices with say 6 (scalar) multiplications. Now if we can multiply 3 3 matrices and k is the number of scalar multiplications required and we use a recursive decomposition of an n n matrix into n/3 n/3 matrices we get a recurrence for the running time of this method of T (n) = kt (n/3) + O(n 2 ). The solution of this recurrence (master method) yields T (n) = Θ(n log 3 k ). Therefore if we compare the exponent log 3 k to lg 7, then for k = 21 we have log < = lg 7. For that value of k = 21 or smaller a by 3 decomposition beats Strassen s method. b. This is similar to part (a). If we decompose not by-2 or by-3, but by-t, where t now is 68, 70, 72 we obtain that the running time for such a decomposition is given by the recurrence, T (n) = kt (n/t) + O(n 2 ). with an asymptotically tight solution given by T (n) = Θ(n log t k ). Therefore if we compare the exponent log t k to lg 7, then for t = 68, log = , t = 70, log = , t = 72, log = All three of them beat Strassen s exponent. The best of the three is the t = 70 split. Calculations used the Unix bc calculator and hopefully are correct! Problem 8. Set a ii =. Run Floyd-Warshall. Then a n ii among all a n ii for all i. will give the length of the smallest cycle from i to i. Find the minimum Problem 9. At every iteration inspect the diagonal elements. Initially, they are all zero. If a value along the diagonal changes it can only grow smaller, i.e. negative. A negative a k ii < 0 show the existence of a cycle with end-points i whose all intermediate vertices are k or less. Problem 10. Yes! G is a tree and thus m = n 1. Therefore depth-first-search solves this problem in O(n + m) time which is O(n) as well. Problem 11. Yes! Run depth first search by also using two counters for each connected component discovered during the top-level call. One counts the vertices discovered and the other the edges discovered. If edges discovered is equal or more than vertices discovered, we have a cycle by the simple observation that in a tree edges are one less than vertices, and if in a connected component searched through depth-first-search we have discovered a number of edges equal to or more than the number of vertices, a cycle should exist there. Note that in this algorithm the running time is i O(n i + m i ), where n i is the number of vertices in the i-th component searched and m i the number of edges searched in that component. Naturally i n i = n. Given that the search process stops as soons as m i n i, we have m i n i. Therefore i m i = O(n). Combining the two we obtain that i O(n i + m i ) = O(n) as well. Problem 12. Yes! We only need to maintain a heap whose operations are O(1) rather than the traditional O(lg n). Note that in Prim s algorithm the key in a heap is the edge weight. We can only have two values 10 or 20. Therefore we can implement a heap with an array of size two that stores two (doubly) linked lists. We attach to element 0 a linked list of all edges of weight 10, and to element 1 a linked list of all edges of weight 20. We only insert or delete from the head of the list to mainain O(1) time operations. Insert inserts the weight of edge (u, v) into the Heap as the weight of v. We insert either to the linked list attached to element 0 or to the one attached to element 1. Either time is constant. At the same time we point point[v] to this location (we can also choose to record in (u, v) information that this edge is related to v). ExtractMin takes O(1) time. Check list 0 first. If list 0 is not empty, extract its element pointed by head. If list 0 is empty go to list 1 and extract an element through the head pointer. If both lists 0 and 1 are empty return underflow. 5

6 Decrease takes constant time because we use doubly linked lists. We might move an element (edge) from one list to the other. Note that in accordance to Prim s algorithm for every vertex in an array of size n we store a pointer point[v] that points to the position of the vertex in the linked list/heap. Therefore DecreaseMin(v) (lines 10-12), first locates v in the corresponding linked list using point[v] (and this should be the list attached to element 1) and if the value needs to be decreased (i.e. from 20 to 10) then the element pointed to by point[v] is deleted in constant time, a new element is inserted in the linked list of element 0 and point[v] points to this new location (and also the new location points to v). Problem 13. Easy. at a cost of 2,200,000. ((M 0 (M 1 M 2 )) M 3 ) j=i+b M 1.2M 2.2M i 1 0 1M 3M 2 0 5M < N[i,i+1] M=1,000, < N[i.i]=0 Example Calculation - Diagonal elements are 0 by N[i,i] initialization - Elements over diagonal are computed by N[i,i+1]=di di+1 di+2 formulae - Elements below the diagonal are irrelevant - Consider N[0,2] (the entry that will become 1.2M); This is N[i,j]=N[i,i+b] - Its entry is determined by elements of the same row and column - One choice is M[0,0]*M[1,2] with M[0,0]=0 M[1,2]=1M - Another choice is M[0,1]*M[2,2] with M[0,1]=10M M[2,2]=0 - First choice gives 0+ 1M + 100x200x10 = 1.2M - Second choice gives 10M x500x10 } = 10.5M - The smallest of the two and original value (infinity) is 1.2 M - Therefore so far (M0 x (M1 x M2)) is the way to go. - For an element N[i,j] - the element of the same row used is N[i,k] and di - the element of the same column used is N[k+1,j] and dk+1 and dj+1 6