Homework1: Algorithm analysis (due on Wednesday, 1/5/17) Answers 1. Give the growth rate of the following functions using big-theta notation. f() = + log + 5 3 = Θ( 3 ) f() = 5 + 14log + 5 = Θ( ) f() = 3 + log + 3 4 = Θ( ) f() = 55 = Θ(1) f() = + 100log + 5 = Θ() f() = 5log 3 + + 3 = Θ() f() =! + = Θ( ) f() = 0.0005 5000 = Θ() f() = 7log + 5 = Θ(log ) f() = 3 4 + 3 +3 + 1 +3 = Θ( 4 ) f() =! + 5 5 = Θ(!) f() = 100 + = Θ( ). The following functions represent running times of different algorithms:,!, log, 3,, 1, log 4,,, log, a) Sort these functions from the slowest growing function (representing the fastest algorithm) to the fastest growing function (representing the slowest algorithm). 1, log, log 4,,, log,, 3,,!, b) Are there any functions that are growing at the same rate (yes/no)? If yes, list them. Yes: log and log 4 grow at the same rate since log 4 =4log = Θ(log) 3. Here is the pseudocode for bubble sort algorithm to sort a given A list of numbers (indexing is natural, i.e. starts with 1; =A.lenght). ote: maximum number of passes (while-loop iterations) through the list is. BubbleSort(A) frequencies(worst case) frequencies(best case) frequencies( avg case) 1 done = false 1 1 1 while done == false +1 1+1 /+1 3 done = true 1 / 4 for i = 1 to A.length-1 (-1+1) 1 (-1+1) / (-1+1) 5 if A[i] > A[i+1] (-1) 1 (-1) / (-1) swap A[i] and A[i+1] (-1) 0 / / 7 done = false (-1) 0 / / Analyze this algorithm: ote: This algorithm has two loops where one of them, the while-loop, executes different number of iterations depending on the data values in the given list. This means that for the algorithm s execution we have best, worst, and average scenarios. 1) Find out the time function of this algorithm and its growth rate, showing detailed calculations, as done in class: (i) figure out frequencies of all lines of the pseudocode, (ii) compute the total number of steps executed by the algorithm, and obtain a formula for the running time, i.e. T() function, (iii) get the growth rate of T() function. To get the time function of the algorithm, we need to analyze the worst case, i.e. when the algorithm executes maximum number of steps. For this algorithm the worst case is when the while-loop executes max number of iterations, which is, doing max number of steps per iteration. The worst case happens when the list is sorted backward the while loop makes -1 iterations (and each iteration every check causes element-swap), and then one more iteration where no wrong pairs are detected and no swaps occur. In the worst case the frequencies of all lines will be as stated above, in the first column next to the pseudocode. T() will be obtained by adding all values in that first column: T() = 1 + (+1) + + + 3 (-1) = 4 + T() = O( ) 1
) Analyze the bes t case scenario of this algorithm the same way you ve done analysis in bullet 1). Obtain the formula for T() function and then get its growth rate. For this algorithm the best case is when the while-loop executes min number of iterations, which is 1 (during this iteration the if-condition will never be true because otherwise it would lead to another iteration of the while-loop). The best case happens when the list is already sorted the while loop executes once where it does not detect any wrong pairs. In the best case the frequencies of all lines will be as stated above, in the second column next to pseudocode. T() will be obtained by adding all values in that second column: T() = 1 + + 1 + + (-1) = + 3 T() = O() 3) Analyze the average case scenario of this algorithm the same way you ve done analysis in bullet 1). Obtain the formula for T() function and then get its growth rate. For this algorithm the average case can be computed by taking the average number of iterations that the while-loop executes, doing the average number of steps per iteration. The average number of iterations of the while loop is computed as the average of all possible numbers of iterations (1 iteration, iterations,, iterations): 1++ + = i=1 i (+1) = = +1 //for simplicity, it s fine to approximate Although it is not going to make a difference in the end, but to be proper, let s also consider the average number of steps executed per iteration of the nested for-loop, i.e. compute the average frequencies of lines,7 (they can execute min 0, max -1 times): 0+1+ +( 1) = 1 i=0 i = ( 1) = 1 //for simplicity, it s fine to approximate In average case, the frequencies of all lines will be as stated above, in the third column next to pseudocode. T() will be obtained by adding all values in that third column: T() = 1 + (/+1) + / + / + / (-1) + / / = = + 1 / + 3 / = 1.5 +0.5 + T() = O( ) 4) Indicate whether or not you can use big-theta to represent the growth rate of the running time if you are showing the time for: a) the best case, b) the worst case, c) the average case, d) the algorithm in general. a) yes b) yes c) yes d) no ote: for the algorithm in general (the d-option) we can t use big-theta since the growth rate of the time function in the best case has lower order than in the worst case. 4. Here is the pseudocode for insertion sort algorithm to sort a given A list of numbers (indexing is natural, i.e. starts with 1; =A.lenght). InsertionSort(A) frequencies(worst case) frequencies(best case) frequencies( avg case) 1 for i= to A.length -1+1-1+1-1+1 temp = A[i] -1-1 -1 3 j = i -1-1 -1 4 while j>1 and A[j-1]>temp i= ( i 1 + 1) (-1) 1 i= ( i/ + 1) 5 A[j] = A[j-1] i= ( i 1) 0 //(-1) 0 i= ( i/) j = j 1 i= ( i 1) 0 //(-1) 0 i= ( i/) 7 A[j] = temp -1-1 -1 ote: instead of i-1 inside the sum, we can write i j= 1 (this sum equals to i-1)
Analyze the insertion sort algorithm: ote: This algorithm has two loops where one of them, the while-loop, executes different number of iterations depending on the data values in the given list. This means that for the algorithm s execution we have best, worst, and average scenarios. 1) Find out the time function of this algorithm and its growth rate, showing detailed calculations, as done in class: (i) figure out frequencies of all lines of the pseudocode, (ii) compute the total number of steps executed by the algorithm, and obtain a formula for the running time, i.e. T() function, (iii) get the growth rate of T() function. To get the time function of the algorithm, we need to analyze the worst case, i.e.when the algorithm executes maximum number of steps. For this algorithm the worst case is when the while-loop executes max number of iterations, i.e does i-1 iterations in each for-loop iteration (for each i-value). The worst case happens when the list is sorted backward in each for-loop iteration the A[i] element ends up being sent to the front of the list after all i-1 of its left neighbors are checked and shifted one cell to the right (the second condition of the while loop is always true). In the worst case the frequencies of all lines will be as stated above, in the first column next to the pseudocode. T() will be obtained by adding values in first column: T() = + (-1) + i= ( i 1 + 1) + i= ( i 1) + (-1) = = 4 3 + 1 i= 1 + 3 i= ( i 1) = 4 3 + -1 + 3 i=1 i = //ote that the sum i= ( i 1) is the same as the sum 1 i=1 i = 5 4 + 3 ( 1) = 1.5 + 3.5 4 T() = O( ) ) Analyze the best case scenario of this algorithm the same way you ve done analysis in bullet 1). Obtain the formula for T() function and then get its growth rate. For this algorithm the best case is when the while-loop executes min number of iterations, which is 0 iterations in each for-loop iteration (for each i-value). The best case happens when the list is already sorted in each for-loop iteration the A[i] element stays put (the second condition of the while-loop is false right away so the loop never iterates). In the best case the frequencies of all lines will be as stated above, in the second column next to the pseudocode. T() will be obtained by adding all values of that second column: T() = + 4 (-1) = 5-4 T() = O() 3) Analyze the average case scenario of this algorithm the same way you ve done analysis in bullet 1). Obtain the formula for T() function and then get its growth rate. For this algorithm the average case can be computed by taking the average number of iterations of the while-loop in each for-loop iteration (for each i-value). This number is computed as the average of all possible numbers of while-loop iterations for a given i (0 iterations, 1 iteration, iterations, i-1 iterations): 0+1++ +(i 1) = i 1 k=0 k i (i 1)i = i //for simplicity, it s fine to approximate i i In the average case, the frequencies of all lines will be as stated above, in the third column next to pseudocode. T() will be obtained by adding values in third column: = i 1 T() = + (-1) + i= (i/ + 1) + i= ( i/) + -1 = 4 3 + i= 1 + 3 ( i/) 3 = 4 3 + -1 + i= i = 5 4 + 3 ((+1) 1) = 0.75 + 5.75 5.5 T() = O( ) //ote that i i= = i= 1 i - 1 i= = 3
4) Indicate whether or not you can use big-theta to represent the growth rate of the running time if you are showing the time for: a) the best case, b) the worst case, c) the average case, d) the algorithm in general. a) yes b) yes c) yes d) no ote: for the algorithm in general (d-option) we can t use big-theta since the growth rate of time function in the best case has lower order than in the worst case. 5. Here is an efficient (logarithmic) recursive algorithm for computing a n. Power(a, n) 1 if n ==0 return 1 3 else 4 if n==1 5 return a else 7 if n% ==0 8 return Power (a*a, n/) //if n is an even number, then a n = (a ) n/ 9 else 10 return Power (a*a, n/ )*a //if n is an odd number, then a n = a (a ) n/ Give the recurrence equation representing the time function of this algorithm (the time function of this algorithm depends only on n). T(n) = T(n/) + Θ(1) T(1) = Θ(1); T(0) = Θ(1) //two base cases. Here is a recursive algorithm for computing n-th Fibonacci number (VERY slow exponential algorithm; it is used as an example for educational purposes). Fibonacci(n) 1 if n ==1 or n == return 1 3 else 4 return Fibonacci(n-1) + Fibonacci(n-) Give the recurrence equation representing the time function of this algorithm. T(n) = T(n-1) + T(n-) + Θ(1) T() = Θ(1); T(1) = Θ(1) //two base cases 7. Solve the following recurrence equations using the recursion tree method: draw the recursion tree, do the math as done in class (show level-sums, as well as the detailed math when obtaining their sum) and give an asymptotic upper bound on the recurrence (via big-theta notation). a) T(n) = T(n-1) + n At each level of the recursion tree argument s value level-sums n n n (n-1) n-1 (n-1) (n-) n- (n-) : : : 1 1 1 4
To get the total number of steps, add all level-sums (numbers in the last column): T(n) = 1 + + + (n-) + (n-1) + n = n i=1 i = n(n+1)(n+1). Thus T(n) = O(n 3 ) ote: we used k i=0 i formula here. = k(k+1)(k+1) b) T(n) = T(n-1) + 1 At each level of the recursion tree argument s value #nodes level-sums 1 n 1= 0 0 1 1 n-1 = 1 1 1 1 1 1 n- 4= 1 1 1 1 1 1 n 1 1 n-3 8= 3 3 : : : : : : : : : : : : 1 1 1 1 1 1 1 1 1 = n-(n-1) n-1 n-1 ote: number of levels in the tree is n. To get the total number of steps, add all level-sums (numbers in the last column): T(n) = 0 + 1 + + n- + n-1 = n 1 i=0 i = n 1. Thus T(n) = O( n ) ote: we used k i=0 i = k+1 1 formula here. c) T(n) = 4T(n/4) + 1 At each level of the recursion tree argument s value #nodes level-sums 1 n=n/4 0 1=4 0 4 0 1 1 1 1 n/4=n/4 1 4=4 1 4 1 1 1 1 1 n/1=n/4 1=4 4 1 1 1 1 n/4=n/4 3 4=4 3 4 3 : : : : : : : : : : : : 1 1 1 1 1= n 4 log 4 n 4log 4 n 4 log 4 n ote: the height of the tree is log 4 n and the number of levels is log 4 n +1. To get the total number of steps, add all level-sums (numbers in the last column): T(n) = 4 0 + 4 1 + 4 n- + + 4 log 4 n log = 4 n i=0 4 i = (4 log 4 n + 1 1)/3 = (4 4 log 4 n 1)/3 = 4 n 1 3 3 Thus T(n) = O(n). ote: we used k i=0 x i = xk+1 1, for x > 1 formula here. x 1 5
d) T(n) = 3T(n/) + n At each level of the recursion tree arugment s value #nodes level-sums n n = n 0 1=30 n //i.e. 3 0 n 0 n/ n/ n/ n/= n 1 3=31 3 n n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4= n 9=3 3 n //i.e. 9 n 4 ::: ::: ::: ::: ::: ::: ::: n/8 n/8 n/8 n/8 n n/8 n/8 n/8= n 3 7=33 3 3 n //i.e. 7 n 3 8 : : : : : : : : : : : : : : : 1 1 1 1 1 1 1= n logn 3logn 3 logn n logn ote: the height of the tree is log n and the number of levels is log n +1. The get the total number of steps we need to add up all level-sums (numbers in the last column). We will get: T(n) = 3 0 n + 0 31 n + 1 3 n + 33 n + + 3 3logn n = n logn logn i=0 (3 )i = n ( = n( 3 3logn 3 3logn logn 1) = n n = 3 3 logn n = 3 n log3 n. n 3 )logn+1 1 3 = 1 Thus T(n) = O(n log3 ) = O(n 1.59 ) //note that log3 1.584 ote: formulas used here are: k i=0 x i = xk+1 1, for x>1 and x 1 xlog a y = y log a x ote: some useful formulas for your reference: k i = k(k+1) i=0 k i=0 i k i=0 = k+1 1 x i i k i=0 = k(k+1)(k+1) = xk+1 1 x 1, for x>1 xlog a y = y log a x