# Problem Set #1 Solutions

Save this PDF as:

Size: px
Start display at page:

## Transcription

2 Problem Set #1 Solutions 2 Answer: Most of the ranking is fairly straightforward. Several identities are helpful: lg lg n n = n 2 (lg n) lg n = 4 lg n n = 2 lg n 2lg n 2 = n 2/lg n 1 = n 1/lg n lg (lg n) = lg n 1 for n > 1 In addition, asymptotic bounds for Stirling s formula are helpful in ranking the expressions with factorials: n! = Θ(n n+1/2 e n ) lg(n!) = Θ(n lg n) (lg n)! = Θ((lg n) lg n+1/2 e lg n ) Each term gives a different equivalence class, where the > symbol means ω. 2 2n+1 > 2 2n > (n + 1)! > n! > e n > n 2 n > 2 n > ( 3 2 )n > n 3 > n 2 4 lg n > n lg n lg(n!) 2lg n 2 > lg 2 n > ln n > > lg lg n n (lg n) lg n > (lg n)! > n 2 lg n > ( 2) lg n > lg n > ln ln n > 2 lg n > lg (lg n) lg n > lg(lg n) > 1 n 1/lg n (b) [2 points] Do problem 3-3(b) on page 58 of CLRS. Give an example of a single nonnegative function f(n) such that for all functions g i (n) in part (a), f(n) is neither O(g i (n)) nor Ω(g i (n)). Answer: f(n) = (1 + sin n) 2 2n+2. (c) [2 points] Give an example of a single nonnegative function f(n) such that for all functions g i (n) in part (a), f(n) = o(g i (n)). Answer: f(n) = 1/n. (d) [2 points] Give an example of a single nonnegative function f(n) such that for all functions g i (n) in part (a), f(n) = ω(g i (n)). Answer: f(n) = 2 22n.

3 Problem Set #1 Solutions 3 2. [16 points] Recurrences Give asymptotic upper and lower bounds for T(n) in each of the following recurrences. Make your bounds as tight as possible, and justify your answers. You may assume T(n) is constant for sufficiently small n. (a) [2 points] T(n) = T(9n/10) + n. Answer: T(n) = Θ(n). We have a = 1, b = 10/9, f(n) = n so n log b a = n 0 = 1. Since f(n) = n = Ω(n 0+ɛ ), case 3 of the Master Theorem applies if we can show the regularity condition holds. For all n, af(n/b) = 9n 9 9 n = cf(n) for c = Therefore, case 3 tells us that T(n) = Θ(n). (b) [2 points] T(n) = T( n) + 1. Answer: T(n) = Θ(lg lg n). We solve this problem by change of variables. Let m = lg n, and S(m) = T(2 m ). Then T(n) = T(n 1/2 ) + 1 T(2 m ) = T(2 m/2 ) + 1 S(m) = S(m/2) + 1 We have a = 1, b = 2, f(m) = 1 so m log b a = m 0 = 1. Since f(m) = Θ(1), case 2 of the Master Theorem applies and we have S(m) = Θ(lg m). To change back to T(n), we have T(n) = T(2 m ) = S(m) = Θ(lg m) = Θ(lg lg n). Alternatively, we could solve this problem by algebraic substitution. T(n) = T(n 1/2 ) + 1 = (T(n 1/4 ) + 1) = T(n 1/2k ) + k When n 1/2k 2, we have that k lg lg n. Then, T(n) = Θ(1) + lg lg n = Θ(lg lg n). (c) [2 points] T(n) = 4T(n/2) + n 2 lg n. Answer: T(n) = Θ(n 2 lg 2 n). We have a = 4, b = 2, f(n) = n 2 lg n so n log b a = n 2. Since f(n) = Θ(n 2 lg n), case 2 of the Master Theorem applies and T(n) = Θ(n 2 lg 2 n). (d) [2 points] T(n) = 5T(n/5) + n/ lg n. Answer: T(n) = Θ(n lg lg n). We have a = 5, b = 5, f(n) = n/ lg n so n log b a = n. None of the Master Theorem cases may be applied here, since f(n) is neither polynomially bigger or smaller than n, and is not equal to Θ(n lg k n) for any k 0. Therefore, we will solve this problem by algebraic substitution. T(n) = 5T(n/5) + n/ lg n = 5(5T(n/25) + (n/5)/(lg(n/5))) + n/ lg n

4 Problem Set #1 Solutions 4 = 25T(n/25) + n/ lg(n/5) + n/ lg(n)... = 5 i T(n/5 i i 1 ) + n/ lg(n/5 j ). j=1 When i = log 5 n the first term reduces to 5 log 5 n T(1), so we have T(n) = nθ(1) + lg 5 n 1 j=1 (n/(lg(n/5 j 1 ))) lg 5 n 1 = Θ(n) + n (1/(lg n (j 1) lg 2 5)) j=1 log 5 n 1 = Θ(n) + n(1/ log 2 5) (1/(log 5 n (j 1))) j=1 log 5 n = Θ(n) + n log 5 2 (1/i). i=2 This is the harmonic sum, so we have T(n) = Θ(n) + c 2 n ln(log 5 n) + Θ(1) = Θ(n lg lg n). (e) [2 points] T(n) = T(n/2) + T(n/4) + T(n/8) + n. Answer: T(n) = Θ(n). We solve this problem by guess-and-check. The total size on each level of the recurrance tree is less than n, so we guess that f(n) = n will dominate. Assume for all i < n that c 1 n T(i) c 2 n. Then, c 1 n/2 + c 1 n/4 + c 1 n/8 + kn T(n) c 2 n/2 + c 2 n/4 + c 2 n/8 + kn c 1 n(1/2 + 1/4 + 1/8 + k/c 1 ) T(n) c 2 n(1/2 + 1/4 + 1/8 + k/c 2 ) c 1 n(7/8 + k/c 1 ) T(n) c 2 n(7/8 + k/c 2 ) If c 1 8k and c 2 8k, then c 1 n T(n) c 2 n. So, T(n) = Θ(n). In general, if you have multiple recursive calls, the sum of the arguments to those calls is less than n (in this case n/2 + n/4 + n/8 < n), and f(n) is reasonably large, a good guess is T(n) = Θ(f(n)). (f) [2 points] T(n) = T(n 1) + 1/n. Answer: T(n) = Θ(lg n). We solve this problem by algebraic substitution. T(n) = T(n 1) + 1/n = T(n 2) + 1/(n 1) + 1/n... n = Θ(1) + 1/i i=1 = ln n + Θ(1)

5 Problem Set #1 Solutions 5 (g) [2 points] T(n) = T(n 1) + lg n. Answer: T(n) = Θ(n lg n). We solve this problem by algebraic substitution. T(n) = T(n 1) + lg n = T(n 2) + lg(n 1) + lg n... n = Θ(1) + lg i i=1 n = Θ(1) + lg( i) i=1 = Θ(1) + lg(n!) = Θ(n lg n) (h) [2 points] T(n) = nt( n) + n. Answer: T(n) = Θ(n lg lg n). We solve this problem by algebraic substitution. T(n) = nt( n) + n = n 1/2 (n 1/4 T(n 1/4 ) + n 1/2 ) + n = n 3/4 T(n 1/4 ) + 2n = n 3/4 (n 1/8 T(n 1/8 ) + n 1/4 ) + 2n = n 7/8 T(n 1/8 ) + 3n... = n 1 1/2k T(n 1/2k ) + kn When, n 1/2k falls under 2, we have k > lg lg n. We then have T(n) = n 1 1/ lg n T(2)+ n lg lg n = Θ(n lg lg n). 3. [10 points] Integer Multiplication Let u and v be two n-bit numbers, where for simplicity n is a power of 2. The traditional multiplication algorithm requires Θ(n 2 ) operations. A divide-and-conquer based algorithm splits the numbers into two equal parts, computing the product as uv = (a2 n/2 + b)(c2 n/2 + d) = ac2 n + (ad + bc)2 n/2 + bd Here, a and c are the higher order bits, and b and d are the lower order bits of u and v respectively. Multiplications are done recursively, except multiplication by a power of 2, which is a simple bitshift and takes Θ(n) time for an n-bit number. Addition and subtraction also take Θ(n) time. (a) [4 points] Write a recurrence for the running time of this algorithm as stated. Solve the recurrence and determine the running time.

6 Problem Set #1 Solutions 6 Answer: T(n) = 4T(n/2) + Θ(n) = Θ(n 2 ). We divide the problem into 4 subproblems (ac, ad, bc, bd) of size n/2 each. The dividing step takes constant time, and the recombining step involves adding and shifting n-bit numbers and therefore takes time Θ(n). This gives the recurrence relation T(n) = 4T(n/2) + Θ(n). For this case, we have a = 4, b = 2, f(n) = n so n log b a = n 2. f(n) = O(n log b a ɛ ) so case 1 of the Master Theorem applies. Therefore, T(n) = Θ(n 2 ). (b) [6 points] You now notice that ad+bc can be computed as (a+b)(c+d) ac bd. Why is this advantageous? Write and solve a recurrence for the running time of the modified algorithm. Answer: T(n) = 3T(n/2) + Θ(n) = Θ(n log 2 3 ). If we write ad + bc as (a + b)(c + d) ac bd, then we only need to compute three integer multiplications of size n/2, namely ac, bd, and (a + c)(c + d). This is advantageous since we replace one multiplication with additions and subtractions, which are less expensive operations. The divide step will now take time Θ(n), since we need to calculate a + c and c + d, and the recombining step will still take Θ(n). This leads to the recurrence relation T(n) = 3T(n/2) + Θ(n). We have a = 3, b = 2, and f(n) = n so n log b a n f(n) = O(n ɛ ) so case 1 of the Master Theorem applies. Therefore, T(n) = Θ(n lg 3 ). 4. [20 points] Stable Sorting Recall that a sorting algorithm is stable if numbers with the same value appear in the output array in the same order as they do in the input array. For each sorting algorithm below, decide whether it is stable or not stable. If the algorithm is stable, give a formal proof of its stability (using loop invariants if necessary). If the algorithm is unstable, suggest a way to modify the algorithm to make it stable. This modification should not alter the fundamental idea of the algorithm or its running time! Explain (informally) why your modifications make the algorithm stable. (a) [6 points] Insertion Sort: pseudocode given on page 17 of CLRS. INSERTION-SORT(A) 1 for j 2 to length[a] 2 do key A[j] 3 Insert A[j] into the sorted sequence A[1... j 1] 4 i j 1 5 while i > 0 and A[i] > key 6 do A[i + 1] A[i] 7 i i 1 8 A[i + 1] key Answer: Insertion sort is stable. We will prove this using the following loop invariant: At the start of each interation of the for loop of lines 1-8, if A[a] = A[b], a < b j 1 and distinct, then A[a] appeared before A[b] in the initial array. Initialization: Before the first loop iteration, j = 2 so there are no distinct

7 Problem Set #1 Solutions 7 elements a < b j 1 = 1. So, the condition holds trivially. Maintenance: Given that the property holds before any iteration, we need to show that it holds at the end of the iteration. During each iteration, we insert A[j] into the sorted sequence A[1...j 1]. For all a < b j 1, the property holds at the end of the loop since we simply shift elements in the subarray and don t change their respective order. For all a < b = j, the property holds at the end of the loop since we insert j after all elements whose value is equal to A[j]. This is because the condition for shifting elements to the right is A[i] > key (line 5), so all the elements whose value equals key are not shifted and A[j] is inserted after them. Thus, since we have shown the property is true for all a < b j 1 and a < b = j, we have shown it true for all a < b j and so the property is true at the end of the loop. Termination: When the for loop ends, j = n + 1, and the loop invariant states that for all A[a] = A[b], a < b n, implies A[a] appeared before A[b] in the initial array. This is the definition of stability and since it applies to the entire array A, insertion sort is stable. (b) [6 points] Mergesort: pseudocode given on pages 29 and 32 of CLRS. MERGE(A, p, q, r) 1 n 1 q p n 2 r q 3 create arrays L[1...n 1 + 1] and R[1...n 2 + 1] 4 for i 1 to n 1 5 do L[i] A[p + i 1] 6 for j 1 to n 2 7 do R[j] A[q + j] 8 L[n 1 + 1] 9 R[n 2 + 1] 10 i 1 11 j 1 12 for k p to r 13 do if L[i] R[j] 14 then A[k] L[i] 15 i i else A[k] R[j] 17 j j + 1 MERGE-SORT(A, p, r) 1 if p < r 2 then q (p + r)/2 3 MERGE-SORT(A, p, q) 4 MERGE-SORT(A, q + 1, r) 5 MERGE(A, p, q, r) Answer: Mergesort is stable. We prove this by induction on the fact that Mergesort is stable.

8 Problem Set #1 Solutions 8 Base Case: When we call merge-sort with indices p and r such that p r (therefore p == r), we return the same array. So, calling mergesort on an array of size one returns the same array, which is stable. Induction: We assume that calling merge-sort on an array of size less than n returns a stably sorted array. We then show that if we call merge-sort on an array of size n, we also return a stably sorted array. Each call to mergesort contains two calls to mergesort on smaller arrays. In addition, it merges these two subarrays and returns. Since we assume that the calls to mergesort on smaller arrays return stably sorted arrays, we need to show that the merge step on two stably sorted arrays returns a stable array. If A[i] = A[j], i < j in the initial array, we need f(i) < f(j) in the new array, where f is the function which gives the new positions in the sorted array. If i and j are in the same half of the recursive merge-sort call, then by assumption, they are in order when the call returns and they will be in order in the merged array (since we take elements in order from each sorted subarray). If i and j are in different subarrays, then we know that i is in the left subarray and j is in the right subarray since i < j. In the merge step, we take the elements from the left subarray while the left subarry element is less than or equal to the right subarray element (line 13). Therefore, we will take element i before taking element j and f(i) < f(j), the claim we are trying to prove. Therefore, Mergesort is stable. (c) [8 points] Quicksort: pseudocode given on page 146 of CLRS. QUICKSORT(A, p, r) 1 if p < r 2 then q PARTITION(A, p, r) 3 QUICKSORT(A, p, q 1) 4 QUICKSORT(A, q + 1, r) PARTITION(A, p, r) 1 x A[r] 2 i p 1 3 for j p to r 1 4 do if A[j] x 5 then i i exchange A[i] A[j] 7 exchange A[i + 1] A[r] 8 return i + 1 Answer: Quicksort is not stable. To make it stable, we can add a new field to each array element, which indicates the index of that element in the original array. Then, when we sort, we can sort on the condition (line 4) A[j] < x OR (A[j] == x AND index(a[j]) < index(x)). At the end of the sort, we are guaranteed to have all the elements in sorted order, and for any i < j, such that A[i] equals A[j], we will have index(a[i]) < index(a[j]) so the sort will be stable.

9 Problem Set #1 Solutions 9 5. [22 points] Computing Fibonacci Numbers Recall the Fibonacci numbers as defined in lecture and on page 56 of CLRS. Throughout this problem assume that the cost of integer addition, subtraction, multiplication, as well as reading from memory or writing to memory is Θ(1), independent of the actual size of the numbers. We will compare the efficiency of different algorithms for computing F n, the nth Fibonacci number. (a) [5 points] First, consider a naive recursive algorithm: int Fib (int n) { // recursion if (n >= 2) return Fib(n-1) + Fib(n-2); // base cases if (n == 1) return 1; if (n == 0) return 0; } // error if none of the above returns Sketch a recursion tree for this algorithm. During the computation of Fib(n), how many times is Fib(n-1) called? Fib(n-2)? Fib(n-3)? Fib(2)? Use formula (3.23) on page 56 of CLRS to conclude that this algorithm takes time exponential in n. Answer: F i = φi ˆφ i 5, where φ = and ˆφ = (1) F(n) F(n 1) F(n 2) F(n 2) F(n 3) F(n 3) F(n 4) F(n 3) F(n 4) F(n 4) F(n 5) F(n 4) F(n 5) F(n 5) F(n 6) We call Fib(n 1) one time during the recursion, Fib(n 2) 2 times, Fib(n 3) 3 times, Fib(n 4) 5 times,..., Fib(2) F n 1 times. Each node in the recurrence

10 Problem Set #1 Solutions 10 tree takes constant time to evaluate, so we need to calculate the number of nodes in the tree. T(n) = Θ( n i=1 F i ). Using formula 3.23 in the book, we get T(n) = Θ( n i=1 φ i ˆφ i 5 ) As n grows large, the ˆφ i term goes to zero, so we are left with T(n) = Θ( n i=1 φ i 5 ) = Ω(φ n ) So, the algorithm takes time exponential in n. (b) [5 points] Notice that our recursion tree grows exponentially because the same computation is done many times over. We now modify the naive algorithm to only compute each Fib(i) once, and store the result in an array. If we ever need Fib(i) again, we simply reuse the value rather than recomputing it. This technique is called memoization and we will revisit it when we cover dynamic programming later in the course. Prune your recursion tree from part (a) to reflect this modification. Considering the amount of computation done at each node is Θ(1), what is the total amount of computation involved in finding Fib(n)? Answer: Fib(n) Fib(n 1) Fib(n 2) Fib(n 2) Fib(n 3) Fib(n 3) Fib(n 4)... Fib(n 5) As in part (a), the time to calculate the nth Fibonacci number F n is equal to the time to calculate F n 1, plus the time to calculate F n 2, plus the time to add F n 1 to F n 2. However, because of memoization, the time to calculate F n 2 is constant once F n 1 has already been calculated. Therefore, the recurrence for this modified algorithm is T(n) = T(n 1) + Θ(1). It is easy to use the algebraic substitution method to verify that the solution to this recurrence is T(n) = Θ(n).

11 Problem Set #1 Solutions 11 (c) [4 points] Prove that ( ) n = ( Fn+1 F n F n F n 1 ) for n = 1, 2, 3,.... Hint: The last line of the question should immediately suggest what proof technique would be useful here. Answer: We prove the statement by mathematical induction. For the base case of n = 1, we get ( ) 1 ( ) 1 1 F2 F = 1 (2) 1 0 F 1 F 0 which certainly holds. Now, we assume the statement holds for n 1. Then, ( ) n ( ) n 1 ( ) = ( ) ( ) Fn F = n F n 1 F n ( ) Fn + F = n 1 F n F n 1 + F n 2 F n 1 ( ) Fn+1 F = n F n F n 1 Thus, we have shown that the statement holds for all n. (d) [8 points] Use part (c) or formula (3.23) to design a simple divide-and-conquer algorithm that computes the nth Fibonacci number F n in time Θ(lg n). ( ) 1 1 Answer: Let A be the matrix of part (c). To compute the nth Fibonacci number F n, we can simply exponentiate the matrix A to the nth power 1 0 and take the element A n 2,1. An efficient divide-and-conquer algorithm for this problem uses the notion of repeated squaring. In other words, to calculate A n, observe that { (A A n = n/2 ) 2 if n is even (A n/2 ) 2 A if n is odd This algorithm satisfies the recurrence T(n) = T(n/2)+Θ(1) because we consider multiplying two 2 2 matrices as a constant time operation and because we have reduced a problem of size n to one of size n/2. We can solve this recurrence using the master theorem case 2, and arrive at a solution of T(n) = Θ(lg n). 6. [26 points] Algorithm Design

12 Problem Set #1 Solutions 12 (a) [10 points] Design a Θ(n lg n)-time algorithm that, given an array A of n integers and another integer x, determines whether or not there exist two (not necessarily distinct) elements in A whose sum is exactly x. This is problem on page 37 of CLRS, slightly reworded for the sake of clarity. Answer: We first sort the elements in the array using a sorting algorithm such as merge-sort which runs in time Θ(n lg n). Then, we can find if two elements exist in A whose sum is x as follows. For each element A[i] in A, set y = A[i] x. Using binary search, find if the element y exists in A. If so, return A[i] and y. If we can t find y for any A[i], then return that no such pair of elements exists. Each binary search takes time Θ(lg n), and there are n of them. So, the total time for this procedure is T(n) = Θ(n lg n)+θ(n lg n) where the first term comes from sorting and the second term comes from performing binary search for each of the n elements. Therefore, the total running time is T(n) = Θ(n lg n). An alternate procedure to find the two elements in the sorted array is given below: SUM-TO-X(A) 1 Merge-Sort(A) 2 i 1 3 j length(a) 4 while i j 5 if A[i] + A[j] equals x 6 return A[i], A[j] 7 if A[i] + A[j] < x 8 then i i if A[i] + A[j] > x 10 then j j 1 We set counters at the two ends of the array. If their sum is x, we return those values. If the sum is less than x, we need a bigger sum so we increment the bottom counter. If the sum is greater than x, we decrement the top counter. The loop does Θ(n) iterations, since at each iteration we either increment i or decrement j so j i is always decreasing and we terminate when j i < 0. However, this still runs in time Θ(n lg n) since the running time is dominated by sorting. (b) [16 points] The majority element of an array A of length n is the element that appears strictly more than n/2 times in the array. Note that if such an element exists, it must be unique. Design a Θ(n)-time algorithm that, given an array A of n objects, finds the majority element or reports that no such element exists. Assume that two objects can be compared for equality, read, written and copied in Θ(1) time, but no other operations are allowed (for instance, there is no < operator). Explain why your algorithm is correct (no formal proof necessary). Hint: It is possible to find a candidate solution using only one pass through the array, with the help of an auxiliary data structure such as a stack or a queue. Then with one more pass you can determine whether this candidate is indeed the majority element. Answer: We will give an algorithm to solve this problem which uses an auxiliary

13 Problem Set #1 Solutions 13 stack and a proof of why it is correct. The following is the pseudocode for the algorithm. FIND-MAJORITY-ELEMENT(A) 1 for i 1 to length[a] 2 if stack is empty 3 then push A[i] on the stack 4 else if A[i] equals top element on the stack 5 then push A[i] on the stack 6 else pop the stack 7 if stack is empty 8 then return NoMajority 9 candidate top element on the stack 10 counter 0 11 for i 1 to length[a] 12 if A[i] equals candidate 13 then counter counter if counter > length[a]/2 15 then return candidate 16 return NoMajority The procedure first generates a candidate object from the array. It finds this element using the stack by looping over the array. If the stack is empty or the current element is the same as the top object on the stack, the element is pushed onto the stack. If the top object of the stack is different from the current element, we pop that object off the stack. Claim: If a majority element exists, it will be on the stack at the end of this part of the procedure. Proof: (by contradiction) Call the majority element x and say it appears i > n/2 times. Each time x is encountered, it is either pushed on the stack or an element (different from x) is popped off the stack. There are n i < i elements different from x. Assume x is not on the stack at the end of the procedure. Then, each of the i elements of x must have either popped another element off the stack, or been popped off by another element. However, there are only n i < i other elements, so this is a contradiction. Therefore, x must be on the stack at the end of the procedure. Notice that this proof does not show that if an element is on the stack at the end of the procedure that it is the majority element. We can construct simple counterexamples (A = [1, 2, 3]) where this is not the case. Therefore, after obtaining our candidate majority element solution, we check whether it is indeed the majority element (lines 9-16). We do this by scanning through the array and counting the number of times the element candidate appears. We return candidate if it appears more than n/2 times, else we report that no majority element exists. This procedure scans through the array twice and does a constant amount of computation (pushing and popping elements on the stack take constant time) at each step. Therefore the running time is T(n) = cn = Θ(n).

### Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

### CS 102: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4)

CS 10: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4) Problem 1. a. Consider the modified binary search algorithm so that it splits the input not into two sets of almost-equal sizes, but into three

### Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort).

Chapter 7: Quicksort Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort). The three steps of Quicksort are as follows:

### Divide And Conquer Algorithms

CSE341T/CSE549T 09/10/2014 Lecture 5 Divide And Conquer Algorithms Recall in last lecture, we looked at one way of parallelizing matrix multiplication. At the end of the lecture, we saw the reduce SUM

### CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

### Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4

Divide d Conquer Textook Reading Chapters 4, 7 & 33.4 Overview Design principle Divide d conquer Proof technique Induction, induction, induction Analysis technique Recurrence relations Prolems Sorting

### Binary Search. Search for x in a sorted array A.

Divide and Conquer A general paradigm for algorithm design; inspired by emperors and colonizers. Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3.

### Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

### Mergesort and Quicksort

Mergesort Mergesort and Quicksort Basic plan: Divide array into two halves. Recursively sort each half. Merge two halves to make sorted whole. mergesort mergesort analysis quicksort quicksort analysis

### Divide-and-Conquer Algorithms Part Four

Divide-and-Conquer Algorithms Part Four Announcements Problem Set 2 due right now. Can submit by Monday at 2:15PM using one late period. Problem Set 3 out, due July 22. Play around with divide-and-conquer

### Format of Divide-and-Conquer algorithms:

CS 360: Data Structures and Algorithms Divide-and-Conquer (part 1) Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively

### The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >

### Section IV.1: Recursive Algorithms and Recursion Trees

Section IV.1: Recursive Algorithms and Recursion Trees Definition IV.1.1: A recursive algorithm is an algorithm that solves a problem by (1) reducing it to an instance of the same problem with smaller

### CS473 - Algorithms I

CS473 - Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm View in slide-show mode 1 Reminder: Merge Sort Input array A sort this half sort this half Divide Conquer merge two sorted halves Combine

### CS 103X: Discrete Structures Homework Assignment 3 Solutions

CS 103X: Discrete Structures Homework Assignment 3 s Exercise 1 (20 points). On well-ordering and induction: (a) Prove the induction principle from the well-ordering principle. (b) Prove the well-ordering

### Polynomials and the Fast Fourier Transform (FFT) Battle Plan

Polynomials and the Fast Fourier Transform (FFT) Algorithm Design and Analysis (Wee 7) 1 Polynomials Battle Plan Algorithms to add, multiply and evaluate polynomials Coefficient and point-value representation

### Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Quiz 1 Quiz 1 Do not open this quiz booklet until you are directed

### Today s Outline. Exercise. Binary Search Analysis. Linear Search Analysis. Asymptotic Analysis. Analyzing Code. Announcements. Asymptotic Analysis

Today s Outline Announcements Assignment #1 due Thurs, Oct 7 at 11:45pm Asymptotic Analysis Asymptotic Analysis CSE 7 Data Structures & Algorithms Ruth Anderson Autumn 2010 Exercise Analyzing Code bool

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### 3. Mathematical Induction

3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)

### Dynamic Programming. Lecture 11. 11.1 Overview. 11.2 Introduction

Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach

### Data Structures and Algorithms Written Examination

Data Structures and Algorithms Written Examination 22 February 2013 FIRST NAME STUDENT NUMBER LAST NAME SIGNATURE Instructions for students: Write First Name, Last Name, Student Number and Signature where

### Analysis of Algorithms: The Non-recursive Case

Analysis of Algorithms: The Non-recursive Case Key topics: * Introduction * Generalizing Running Time * Doing a Timing Analysis * Big-Oh Notation * Big-Oh Operations * Analyzing Some Simple Programs -

### Class Overview. CSE 326: Data Structures. Goals. Goals. Data Structures. Goals. Introduction

Class Overview CSE 326: Data Structures Introduction Introduction to many of the basic data structures used in computer software Understand the data structures Analyze the algorithms that use them Know

### CSC148 Lecture 8. Algorithm Analysis Binary Search Sorting

CSC148 Lecture 8 Algorithm Analysis Binary Search Sorting Algorithm Analysis Recall definition of Big Oh: We say a function f(n) is O(g(n)) if there exists positive constants c,b such that f(n)

### Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

### Sequential Data Structures

Sequential Data Structures In this lecture we introduce the basic data structures for storing sequences of objects. These data structures are based on arrays and linked lists, which you met in first year

### Week 7: Divide and Conquer

Agenda: Divide and Conquer technique Multiplication of large integers Exponentiation Matrix multiplication 1 2- Divide and Conquer : To solve a problem we can break it into smaller subproblems, solve each

### CS/COE 1501 http://cs.pitt.edu/~bill/1501/

CS/COE 1501 http://cs.pitt.edu/~bill/1501/ Lecture 01 Course Introduction Meta-notes These notes are intended for use by students in CS1501 at the University of Pittsburgh. They are provided free of charge

### MODULAR ARITHMETIC. a smallest member. It is equivalent to the Principle of Mathematical Induction.

MODULAR ARITHMETIC 1 Working With Integers The usual arithmetic operations of addition, subtraction and multiplication can be performed on integers, and the result is always another integer Division, on

### CS473 - Algorithms I

CS473 - Algorithms I Lecture 9 Sorting in Linear Time View in slide-show mode 1 How Fast Can We Sort? The algorithms we have seen so far: Based on comparison of elements We only care about the relative

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### Mathematical Induction. Lecture 10-11

Mathematical Induction Lecture 10-11 Menu Mathematical Induction Strong Induction Recursive Definitions Structural Induction Climbing an Infinite Ladder Suppose we have an infinite ladder: 1. We can reach

### Data Structures. Algorithm Performance and Big O Analysis

Data Structures Algorithm Performance and Big O Analysis What s an Algorithm? a clearly specified set of instructions to be followed to solve a problem. In essence: A computer program. In detail: Defined

### Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1

Recursion Slides by Christopher M Bourke Instructor: Berthe Y Choueiry Fall 007 Computer Science & Engineering 35 Introduction to Discrete Mathematics Sections 71-7 of Rosen cse35@cseunledu Recursive Algorithms

### Biostatistics 615/815

Merge Sort Biostatistics 615/815 Lecture 8 Notes on Problem Set 2 Union Find algorithms Dynamic Programming Results were very ypositive! You should be gradually becoming g y g comfortable compiling, debugging

### Lecture 13 - Basic Number Theory.

Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted

### 8.1 Makespan Scheduling

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programing: Min-Makespan and Bin Packing Date: 2/19/15 Scribe: Gabriel Kaptchuk 8.1 Makespan Scheduling Consider an instance

### Section 6-2 Mathematical Induction

6- Mathematical Induction 457 In calculus, it can be shown that e x k0 x k k! x x x3!! 3!... xn n! where the larger n is, the better the approximation. Problems 6 and 6 refer to this series. Note that

### Elementary Number Theory and Methods of Proof. CSE 215, Foundations of Computer Science Stony Brook University http://www.cs.stonybrook.

Elementary Number Theory and Methods of Proof CSE 215, Foundations of Computer Science Stony Brook University http://www.cs.stonybrook.edu/~cse215 1 Number theory Properties: 2 Properties of integers (whole

### Data Structures and Algorithms

Data Structures and Algorithms Luciano Bononi Computer Science Engineering University of Bologna bononi@cs.unibo.it http://www.cs.unibo.it/~bononi/ Slide credits: these slides have been translated from

### Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras

Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture No. # 31 Recursive Sets, Recursively Innumerable Sets, Encoding

### Sample Induction Proofs

Math 3 Worksheet: Induction Proofs III, Sample Proofs A.J. Hildebrand Sample Induction Proofs Below are model solutions to some of the practice problems on the induction worksheets. The solutions given

### We now explore a third method of proof: proof by contradiction.

CHAPTER 6 Proof by Contradiction We now explore a third method of proof: proof by contradiction. This method is not limited to proving just conditional statements it can be used to prove any kind of statement

### Lecture Notes on Linear Search

Lecture Notes on Linear Search 15-122: Principles of Imperative Computation Frank Pfenning Lecture 5 January 29, 2013 1 Introduction One of the fundamental and recurring problems in computer science is

### Elementary Number Theory We begin with a bit of elementary number theory, which is concerned

CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,

### Induction. Margaret M. Fleck. 10 October These notes cover mathematical induction and recursive definition

Induction Margaret M. Fleck 10 October 011 These notes cover mathematical induction and recursive definition 1 Introduction to induction At the start of the term, we saw the following formula for computing

### Algorithms, Integers

CHAPTER 3 Algorithms, Integers 3.1. Algorithms Consider the following list of instructions to find the maximum of three numbers a, b, c: 1. Assign variable x the value of a. 2. If b > x then assign x the

### The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

### 1 2-3 Trees: The Basics

CS10: Data Structures and Object-Oriented Design (Fall 2013) November 1, 2013: 2-3 Trees: Inserting and Deleting Scribes: CS 10 Teaching Team Lecture Summary In this class, we investigated 2-3 Trees in

### Dynamic Programming Problem Set Partial Solution CMPSC 465

Dynamic Programming Problem Set Partial Solution CMPSC 465 I ve annotated this document with partial solutions to problems written more like a test solution. (I remind you again, though, that a formal

### Algorithms. Margaret M. Fleck. 18 October 2010

Algorithms Margaret M. Fleck 18 October 2010 These notes cover how to analyze the running time of algorithms (sections 3.1, 3.3, 4.4, and 7.1 of Rosen). 1 Introduction The main reason for studying big-o

### L1-2. Special Matrix Operations: Permutations, Transpose, Inverse, Augmentation 12 Aug 2014

L1-2. Special Matrix Operations: Permutations, Transpose, Inverse, Augmentation 12 Aug 2014 Unfortunately, no one can be told what the Matrix is. You have to see it for yourself. -- Morpheus Primary concepts:

### Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

### SECTION 10-2 Mathematical Induction

73 0 Sequences and Series 6. Approximate e 0. using the first five terms of the series. Compare this approximation with your calculator evaluation of e 0.. 6. Approximate e 0.5 using the first five terms

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### 8 Primes and Modular Arithmetic

8 Primes and Modular Arithmetic 8.1 Primes and Factors Over two millennia ago already, people all over the world were considering the properties of numbers. One of the simplest concepts is prime numbers.

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2 Proofs Intuitively, the concept of proof should already be familiar We all like to assert things, and few of us

### Induction Problems. Tom Davis November 7, 2005

Induction Problems Tom Davis tomrdavis@earthlin.net http://www.geometer.org/mathcircles November 7, 2005 All of the following problems should be proved by mathematical induction. The problems are not necessarily

### Quiz 1 Solutions. (a) T F The height of any binary search tree with n nodes is O(log n). Explain:

Introduction to Algorithms March 9, 2011 Massachusetts Institute of Technology 6.006 Spring 2011 Professors Erik Demaine, Piotr Indyk, and Manolis Kellis Quiz 1 Solutions Problem 1. Quiz 1 Solutions True

### Discrete Mathematics: Solutions to Homework (12%) For each of the following sets, determine whether {2} is an element of that set.

Discrete Mathematics: Solutions to Homework 2 1. (12%) For each of the following sets, determine whether {2} is an element of that set. (a) {x R x is an integer greater than 1} (b) {x R x is the square

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Lecture 6. Inverse of Matrix

Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

### What Is Recursion? Recursion. Binary search example postponed to end of lecture

Recursion Binary search example postponed to end of lecture What Is Recursion? Recursive call A method call in which the method being called is the same as the one making the call Direct recursion Recursion

### C H A P T E R Regular Expressions regular expression

7 CHAPTER Regular Expressions Most programmers and other power-users of computer systems have used tools that match text patterns. You may have used a Web search engine with a pattern like travel cancun

### Homework until Test #2

MATH31: Number Theory Homework until Test # Philipp BRAUN Section 3.1 page 43, 1. It has been conjectured that there are infinitely many primes of the form n. Exhibit five such primes. Solution. Five such

### 1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++

Answer the following 1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++ 2) Which data structure is needed to convert infix notations to postfix notations? Stack 3) The

### 1/1 7/4 2/2 12/7 10/30 12/25

Binary Heaps A binary heap is dened to be a binary tree with a key in each node such that: 1. All leaves are on, at most, two adjacent levels. 2. All leaves on the lowest level occur to the left, and all

### PUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.

PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include

### Automata and Computability. Solutions to Exercises

Automata and Computability Solutions to Exercises Fall 25 Alexis Maciel Department of Computer Science Clarkson University Copyright c 25 Alexis Maciel ii Contents Preface vii Introduction 2 Finite Automata

### 1 Review of complex numbers

1 Review of complex numbers 1.1 Complex numbers: algebra The set C of complex numbers is formed by adding a square root i of 1 to the set of real numbers: i = 1. Every complex number can be written uniquely

### U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory

### The last three chapters introduced three major proof techniques: direct,

CHAPTER 7 Proving Non-Conditional Statements The last three chapters introduced three major proof techniques: direct, contrapositive and contradiction. These three techniques are used to prove statements

### 2.1. Inductive Reasoning EXAMPLE A

CONDENSED LESSON 2.1 Inductive Reasoning In this lesson you will Learn how inductive reasoning is used in science and mathematics Use inductive reasoning to make conjectures about sequences of numbers

### CS2 Algorithms and Data Structures Note 11. Breadth-First Search and Shortest Paths

CS2 Algorithms and Data Structures Note 11 Breadth-First Search and Shortest Paths In this last lecture of the CS2 Algorithms and Data Structures thread we will consider the problem of computing distances

### This unit will lay the groundwork for later units where the students will extend this knowledge to quadratic and exponential functions.

Algebra I Overview View unit yearlong overview here Many of the concepts presented in Algebra I are progressions of concepts that were introduced in grades 6 through 8. The content presented in this course

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Data Structures Fibonacci Heaps, Amortized Analysis

Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure:

### Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

### PES Institute of Technology-BSC QUESTION BANK

PES Institute of Technology-BSC Faculty: Mrs. R.Bharathi CS35: Data Structures Using C QUESTION BANK UNIT I -BASIC CONCEPTS 1. What is an ADT? Briefly explain the categories that classify the functions

### Full and Complete Binary Trees

Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full

### Basic Proof Techniques

Basic Proof Techniques David Ferry dsf43@truman.edu September 13, 010 1 Four Fundamental Proof Techniques When one wishes to prove the statement P Q there are four fundamental approaches. This document

### Sample Questions Csci 1112 A. Bellaachia

Sample Questions Csci 1112 A. Bellaachia Important Series : o S( N) 1 2 N N i N(1 N) / 2 i 1 o Sum of squares: N 2 N( N 1)(2N 1) N i for large N i 1 6 o Sum of exponents: N k 1 k N i for large N and k

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### Divide and Conquer: Counting Inversions

Divide and Conquer: Counting Inversions Rank Analysis Collaborative filtering matches your preference (books, music, movies, restaurants) with that of others finds people with similar tastes recommends

### Near Optimal Solutions

Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

### a = bq + r where 0 r < b.

Lecture 5: Euclid s algorithm Introduction The fundamental arithmetic operations are addition, subtraction, multiplication and division. But there is a fifth operation which I would argue is just as fundamental

### Math 55: Discrete Mathematics

Math 55: Discrete Mathematics UC Berkeley, Fall 2011 Homework # 5, due Wednesday, February 22 5.1.4 Let P (n) be the statement that 1 3 + 2 3 + + n 3 = (n(n + 1)/2) 2 for the positive integer n. a) What

### Pythagorean Triples. Chapter 2. a 2 + b 2 = c 2

Chapter Pythagorean Triples The Pythagorean Theorem, that beloved formula of all high school geometry students, says that the sum of the squares of the sides of a right triangle equals the square of the

### CHAPTER 5. Number Theory. 1. Integers and Division. Discussion

CHAPTER 5 Number Theory 1. Integers and Division 1.1. Divisibility. Definition 1.1.1. Given two integers a and b we say a divides b if there is an integer c such that b = ac. If a divides b, we write a

### 3 0 + 4 + 3 1 + 1 + 3 9 + 6 + 3 0 + 1 + 3 0 + 1 + 3 2 mod 10 = 4 + 3 + 1 + 27 + 6 + 1 + 1 + 6 mod 10 = 49 mod 10 = 9.

SOLUTIONS TO HOMEWORK 2 - MATH 170, SUMMER SESSION I (2012) (1) (Exercise 11, Page 107) Which of the following is the correct UPC for Progresso minestrone soup? Show why the other numbers are not valid

### Closest Pair Problem

Closest Pair Problem Given n points in d-dimensions, find two whose mutual distance is smallest. Fundamental problem in many applications as well as a key step in many algorithms. p q A naive algorithm

### 1.1 Logical Form and Logical Equivalence 1

Contents Chapter I The Logic of Compound Statements 1.1 Logical Form and Logical Equivalence 1 Identifying logical form; Statements; Logical connectives: not, and, and or; Translation to and from symbolic

### Integer Factorization using the Quadratic Sieve

Integer Factorization using the Quadratic Sieve Chad Seibert* Division of Science and Mathematics University of Minnesota, Morris Morris, MN 56567 seib0060@morris.umn.edu March 16, 2011 Abstract We give

### WRITING PROOFS. Christopher Heil Georgia Institute of Technology

WRITING PROOFS Christopher Heil Georgia Institute of Technology A theorem is just a statement of fact A proof of the theorem is a logical explanation of why the theorem is true Many theorems have this

### Solution to Homework 2

Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if