1 Exercises and Solutions

Transcription

1 Exercises ad Solutios Most of the exercises below have solutios but you should try first to solve them. Each subsectio with solutios is after the correspodig subsectio with exercises. T. Time complexity ad Big-Oh otatio: exercises. A sortig method with Big-Oh complexity O( log ) speds exactly millisecod to sort,000 data items. Assumig that time T () of sortig items is directly proportioal to log, that is, T () c log, derive a formula for T (), give the time T (N) for sortig N items, ad estimate how log this method will sort,000,000 items.. A quadratic algorithm with processig time T () c speds T (N) secods for processig N data items. How much time will be spet for processig 5000 data items, assumig that N 00 ad T (N) ms? 3. A algorithm with time complexity O(f()) ad processig time T () cf(), where f() is a kow fuctio of, speds 0 secods to process 000 data items. How much time will be spet to process 00,000 data items if f() ad f() 3? 4. Assume that each of the expressios below gives the processig time T () spet by a algorithm for solvig a problem of size. Select the domiat term(s) havig the steepest icrease i ad specify the lowest Big-Oh complexity of each algorithm. Expressio Domiat term(s) O(...) log log (log ) log 3 log 3 log 8 log log log log (log ) 00 log log 4 log log

2 5. The statemets below show some features of Big-Oh otatio for the fuctios f f() ad g g(). Determie whether each statemet is TRUE or FALSE ad correct the formula i the latter case. Statemet Rule of sums: O(f g) O(f) O(g) Rule of products: O(f g) O(f) O(g) Trasitivity: if g O(f) ad h O(f) the g O(h) O( 4 ) Is it TRUE or FALSE? If it is FALSE the write the correct formula O( log ) 6. Prove that T () a 0 a a a 3 3 is O( 3 ) usig the formal defiitio of the Big-Oh otatio. Hit: Fid a costat c ad threshold 0 such that c 3 T () for Algorithms A ad B sped exactly T A () 0. log 0 ad T B ().5 microsecods, respectively, for a problem of size. Choose the algorithm, which is better i the Big-Oh sese, ad fid out a problem size 0 such that for ay larger size > 0 the chose algorithm outperforms the other. If your problems are of the size 0 9, which algorithm will you recommed to use? 8. Algorithms A ad B sped exactly T A () c A log ad T B () c B microsecods, respectively, for a problem of size. Fid the best algorithm for processig 0 data items if the algoritm A speds 0 microsecods to process 04 items ad the algorithm B speds oly microsecod to process 04 items. 9. Algorithms A ad B sped exactly T A () 5 log 0 ad T B () 5 microsecods, respectively, for a problem of size. Which algorithm is better i the Big-Oh sese? For which problem sizes does it outperform the other? 0. Oe of the two software packages, A or B, should be chose to process very big databases, cotaiig each up to 0 records. Average processig time of the package A is T A () 0. log microsecods, ad the average processig time of the package B is T B () 5 microsecods.

3 Which algorithm has better performace i a Big-Oh sese? Work out exact coditios whe these packages outperform each other.. Oe of the two software packages, A or B, should be chose to process data collectios, cotaiig each up to 0 9 records. Average processig time of the package A is T A () 0.00 millisecods ad the average processig time of the package B is T B () 500 millisecods. Which algorithm has better performace i a Big-Oh sese? Work out exact coditios whe these packages outperform each other.. Software packages A ad B of complexity O( log ) ad O(), respectively, sped exactly T A () c A log 0 ad T B () c B millisecods to process data items. Durig a test, the average time of processig 0 4 data items with the package A ad B is 00 millisecods ad 500 millisecods, respectively. Work out exact coditios whe oe package actually outperforms the other ad recommed the best choice if up to 0 9 items should be processed. 3. Let processig time of a algorithm of Big-Oh complexity O(f()) be directly proportioal to f(). Let three such algorithms A, B, ad C have time complexity O( ), O(.5 ), ad O( log ), respectively. Durig a test, each algorithm speds 0 secods to process 00 data items. Derive the time each algorithm should sped to process 0,000 items. 4. Software packages A ad B have processig time exactly T EP 3.5 ad T WP , respectively. If you are iterested i faster processig of up to 0 8 data items, the which package should be choose?. Time complexity ad Big-Oh otatio: solutios. Because processig time is T () c log, the costat factor c T (N) N log N, ad T () T (N) log N log N. Ratio of logarithms of the same base is idepedet of the base (see Appedix i the textbook), hece, ay appropriate base ca be used i the above formula (say, base of 0). Therefore, for the time is T (, 000, 000) T (, 000) log log , 000 ms. The costat factor c T (N) N, therefore T () T (N) N 0000 T (5000), 500 ms. ms ad 3. The costat factor c T (000) f(000) 0 f(000) millisecods per item. Therefore, T () 0 f() f(00,000) f(000) ms ad T (00, 000) 0 f(000) ms. If f() the T (00, 000) 000 ms. If f() 3, the T (00, 000) 0 7 ms. 3

4 Expressio Domiat term(s) O(...) O( 3 ) log O(.5 ) O(.75 ) log (log ) log O( log ) log 3 log log 3, log O( log ) 3 log 8 log log log 3 log 8 O(log ) O( ) O( ) O(.5 ) 0.0 log (log ) (log ) O((log ) ) 00 log O( 3 ) log 4 log log log 4 O(log ) Statemet Is it TRUE or FALSE? If it is FALSE the write the correct formula Rule of sums: O(f g) O(f) O(g) FALSE O(f g) max {O(f), O(g) Rule of products: O(f g) O(f) O(g) TRUE 5. Trasitivity: if g O(f) ad h O(f) the g O(h) FALSE if g O(f) ad f O(h) the g O(h) O( 4 ) TRUE O( log ) FALSE O( 3 ) 6. It is obvious that T () a 0 a a a 3 3. Thus if, the T () c 3 where c a 0 a a a 3 so that T () is O( 3 ). 7. I the Big-Oh sese, the algorithm B is better. It outperforms the algo- 4

5 rithm A whe T B () T A (), that is, whe.5 0. log 0. This iequality reduces to log 0 5, or If 0 9, the algorithm of choice is A. 8. The costat factors for A ad B are: c A 0 04 log ; c B 04 Thus, to process 0 04 items the algorithms A ad B will sped T A ( 0 ) 04 0 log ( 0 ) 080µs ad T B ( 0 ) µs, respectively. Because T B ( 0 ) T A ( 0 ), the method of choice is A. 9. I the Big-Oh sese, the algorithm B is better. It outperforms the algorithm A if T B () T A (), that is, if 5 5 log 0, or log 0 5, or 00, I the Big-Oh sese, the algorithm B of complexity O() is better tha A of complexity O( log ). he package B has better performace i a The package B begis to outperform A whe (T A () T B (), that is, whe 0. log 5. This iequality reduces to 0. log 5, or Thus for processig up to 0 data items, the package of choice is A.. I the Big-Oh sese, the package B of complexity O( 0.5 ) is better tha A of complexity O(). The package B begis to outperform A whe (T A () T B (), that is, whe This iequality reduces to 5 0 5, or Thus for processig up to 0 9 data items, the package of choice is A.. I the Big-Oh sese, the package B of liear complexity O() is better tha the package A of O( log ) complexity. The processig times of the packages are T A () c A log 0 ad T B () c B, respectively. The tests allows us to derive the costat factors: 00 c A 0 4 log c B The package B begis to outperform A whe we must estimate the data size 0 that esures T A () T B (), that is, whe log This iequality reduces to log , or 00. Thus for processig up to 0 9 data items, the package of choice is A. 3. Complexity Time to process 0,000 items A O( ) T (0, 000) T (00) , 000 sec. A O(.5 ) T (0, 000) T (00) , 000 sec log 0000 A3 O( log ) T (0, 000) T (00) 00 log , 000 sec. 5

6 4. I the Big-Oh sese, the package A is better. But it outperforms the package B whe T A () T B (), that is, whe This iequality reduces to 0.5 3/0.03( 00), or 0 8. Thus for processig up to 0 8 data items, the package of choice is B..3 Recurreces ad divide-ad-coquer paradigm: exercises. Ruig time T () of processig data items with a give algorithm is described by the recurrece: ( ) T () k T c ; T () 0. k Derive a closed form formula for T () i terms of c,, ad k. What is the computatioal complexity of this algorithm i a Big-Oh sese? Hit: To have the well-defied recurrece, assume that k m with the iteger m log k ad k.. Ruig time T () of processig data items with aother, slightly differet algorithm is described by the recurrece: ( ) T () k T c k ; T () 0. k Derive a closed form formula for T () i terms of c,, ad k ad detemie computatioal complexity of this algorithm i a Big-Oh sese. Hit: To have the well-defied recurrece, assume that k m with the iteger m log k ad k. 3. What value of k, 3, or 4 results i the fastest processig with the above algorithm? Hit: You may eed a relatio log k l l k where l deotes the atural logarithm with the base e ). 4. Derive the recurrece that describes processig time T () of the recursive method: public static it recurretmethod( it[] a, it low, it high, it goal ) { it target arragetarget( a, low, high ); if ( goal < target ) retur recurretmethod( a, low, target-, goal ); else if ( goal > target ) retur recurretmethod( a, target, high, goal ); else retur a[ target ]; 6

7 The rage of iput variables is 0 low goal high a.legth. A o-recursive method arragetarget() has liear time complexity T () c where high low ad returs iteger target i the rage low target high. Output values of arragetarget() are equiprobable i this rage, e.g. if low 0 ad high, the every target 0,,..., occurs with the same probability. Time for performig elemetary operatios like if else or retur should ot be take ito accout i the recurrece. Hit: cosider a call of recurretmethod() for a.legth data items, e.g. recurretmethod( a, 0, a.legth -, goal ) ad aalyse all recurreces for T () for differet iput arrays. Each recurrece ivolves the umber of data items the method recursively calls to. 5. Derive a explicit (closed form) formula for time T () of processig a array of size if T () is specified implicitly by the recurrece: T () (T (0) T () T ( )) c ; T (0) 0 Hit: You might eed the -th harmoic umber H 3 l for derivig the explicit formula for T (). 6. Determie a explicit formula for the time T () of processig a array of size if T () relates to the average of T ( ),..., T (0) as follows: where T(0) 0. T () (T (0) T ( )) c Hit: You might eed the equatio 3 () for derivig the explicit formula for T (). 7. The obvious liear algorithm for expoetiatio x uses multiplicatios. Propose a faster algorithm ad fid its Big-Oh complexity i the case whe m by writig ad solvig a recurrece formula..4 Recurreces ad divide-ad-coquer paradigm: solutios. The closed-form formula ca be derived either by telescopig or by guessig from a few values ad usig the math iductio. If you kow differece equatios i math, you will easily otice that the recurreces are the differece equatios ad telescopig is their solutio by substitutio of the same equatio for gradually decreasig argumets: or /k. 7

8 (a) Derivatio with telescopig : the recurrece T (k m ) k T (k m ) c k m ca be represeted ad telescoped as follows: T (k m ) k m T (km ) k m c T (k m ) k m T (km ) k m c T (k) T () c k Therefore, T (km ) k m c m, or T (k m ) c k m m, or T () c log k. The Big-Oh complexity is O( log ). (b) Aother versio of telescopig: the like substitutio ca be applied directly to the iitial recurrece: T (k m ) k T (k m ) c k m k T (k m ) k T (k m ) c k m k m T (k) k m T () c k m so that T (k m ) c m k m (c) Guessig ad math iductio: let us costruct a few iitial values of T (), startig from the give T () ad successively applyig the recurrece: T () 0 T (k) k T () c k c k T (k ) k T (k) c k c k T (k 3 ) k T (k ) c k 3 3 c k 3 This sequece leads to a assumptio that T (k m ) c m k m. Let us explore it with math iductio. The iductive steps are as follows: (i) The base case T () T (k 0 ) c 0 k 0 0 holds uder our assumptio. (ii) Let the assumptio holds for l 0,..., m. The T (k m ) k T (k m ) c k m k c (m ) k m c k m c (m ) k m c m k m Therefore, the assumed relatioship holds for all values m 0,,...,, so that T () c log k. 8

9 The Big-Oh complexity of this algorithm is O( log ).. The recurrece T (k m ) k T (k m ) c k m telescopes as follows: T (k m ) k m T (k m ) k m T (k) k T (k m ) k m T (k m ) k m Therefore, T (km ) k m c m, or T (k m ) c k m m, or T () c k log k. The complexity is O( log ) because k is costat. 3. Processig time T () c k log k ca be easily rewritte as T () c k l k l to give a explicit depedece of k. Because l.8854, 3 4 l , ad l , the fastest processig is obtaied for k Because all the variats: T () k c c c T () T (0) c or T () T ( ) c if target 0 T () T () c or T () T ( ) c if target T () T () c or T () T ( 3) c if target T () T ( ) c or T () T (0) c if target are equiprobable, the the recurrece is T () (T (0)... T ( )) c 5. The recurrece suggests that T () T (0)T () T ( )T ( )c. It follows that ( )T ( ) T (0)...T ( )c ( ), ad by subtractig the latter equatio from the former oe, we obtai the followig basic recurrece: T () ( )T ( ) T ( ) c c. It reduces to T () T ( ) c c, or T () T ( ) c c. Telescopig results i the followig system of equalities: T () T ( ) c c T ( ) T ( ) c c T () T () c c T () T (0) c c Because T (0) 0, the explicit expressio for T () is: ( T () c c ) c H c l

10 6. The recurrece suggests that T () (T (0)T ()...T ( )T ( )) c. Because ( )T ( ) (T (0)... T ( )) c ( ), the subtractio of the latter equality from the former oe results i the followig basic recurrece T () ( )T ( ) T ( ) c. It reduces to T () ( )T ( ) c, or T () T ( ) c (). Telescopig results i the followig system of equalities: T () T ( ) T ( ) c () T ( ) c ( ) T () 3 T () T () c 3 T (0) c Because T (0) 0, the explicit expressio for T () is: T () c 3 ( ) c ( ) c so that T () c. A alterative approach (guessig ad math iductio): T (0) 0 T () 0 c c T () (0 c) c c T (3) 3 (0 c c) c 3c T (4) 4 (0 c c 3c) c 4c It suggests a assumptio T () c to be explored with math iductio: (i) The assumptio is valid for T (0) 0. (ii) Let it be valid for k,...,, that is, T (k) kc for k,...,. The T () (0 c c... ( )c) c ( ) c c ( )c c c The assumptio is prove, ad T () c. 7. A straightforward liear expoetiatio eeds m multiplicatios to produce x after havig already x. But because x x x, it ca be computed with oly oe multiplicatio more tha to compute x. Therefore, more efficiet expoetiatio is performed as follows: x x x, x 4 x x, x 8 x 4 x 4, etc. Processig time for such more efficiet 0

11 algorithm correspods to a recurrece: T ( m ) T ( m ), ad by telescopig oe obtais: T ( m ) T ( m ) T ( m ) T ( m ) T () T () T () 0 that is, T ( m ) m, or T () log. The Big-Oh complexity of such algorithm is O(log )..5 Time complexity of code: exercises. Work out the computatioal complexity of the followig piece of code: for( it i ; i > 0; i / ) { for( it j ; j < ; j * ) { for( it k 0; k < ; k ) {... // costat umber of operatios. Work out the computatioal complexity of the followig piece of code. for ( i; i < ; i * ) { for ( j ; j > 0; j / ) { for ( k j; k < ; k ) { sum (i j * k ); 3. Work out the computatioal complexity of the followig piece of code assumig that m : for( it i ; i > 0; i-- ) { for( it j ; j < ; j * ) { for( it k 0; k < j; k ) {... // costat umber C of operatios

12 4. Work out the computatioal complexity (i the Big-Oh sese) of the followig piece of code ad explai how you derived it usig the basic features of the Big-Oh otatio: for( it boud ; boud < ; boud * ) { for( it i 0; i < boud; i ) { for( it j 0; j < ; j ) {... // costat umber of operatios for( it j ; j < ; j * ) {... // costat umber of operatios 5. Determie the average processig time T () of the recursive algorithm: it mytest( it ) { if ( < 0 ) retur 0; 3 else { 4 it i radom( - ); 5 retur mytest( i ) mytest( - - i ); 6 7 providig the algorithm radom( it ) speds oe time uit to retur a radom iteger value uiformly distributed i the rage [0, ] whereas all other istructios sped a egligibly small time (e.g., T (0) 0). Hits: derive ad solve the basic recurrece relatig T () i average to T ( ),..., T (0). You might eed the equatio 3 () for derivig the explicit formula for T (). 6. Assume that the array a cotais values, that the method radomvalue takes costat umber c of computatioal steps to produce each output value, ad that the method goodsort takes log computatioal steps to sort the array. Determie the Big-Oh complexity for the followig fragmets of code takig ito accout oly the above computatioal steps: for( i 0; i < ; i ) { for( j 0; j < ; j ) a[ j ] radomvalue( i ); goodsort( a );

13 .6 Time complexity of code: solutios. I the outer for-loop, the variable i keeps halvig so it goes roud log times. For each i, ext loop goes roud also log times, because of doublig the variable j. The iermost loop by k goes roud times. Loops are ested, so the bouds may be multiplied to give that the algorithm is O ( (log ) ).. Ruig time of the ier, middle, ad outer loop is proportioal to, log, ad log, respectively. Thus the overall Big-Oh complexity is O((log ) ). More detailed optioal aalysis gives the same value. Let k. The the outer loop is executed k times, the middle loop is executed k times, ad for each value j k, k,...,,, the ier loop has differet executio times: j Ier iteratios k k ( k k ) k ( k k ) ( k ) 0 ( k 0 ) I total, the umber of ier/middle steps is k k (... k ) k k ( k ) Thus, the total complexity is O((log ) )..5 (k ) k (log ) O( log ) 3. The outer for-loop goes roud times. For each i, the ext loop goes roud m log times, because of doublig the variable j. For each j, the iermost loop by k goes roud j times, so that the two ier loops together go roud 4... m m times. Loops are ested, so the bouds may be multiplied to give that the algorithm is O ( ). 4. The first ad secod successive iermost loops have O() ad O(log ) complexity, respectively. Thus, the overall complexity of the iermost part is O(). The outermost ad middle loops have complexity O(log ) ad O(), so a straightforward (ad valid) solutio is that the overall complexity is O( log ). 3

14 More detailed aalysis shows that the outermost ad middle loops are iterrelated, ad the umber of repeatig the iermost part is as follows:... m m where m log is the smallest iteger such that m >. Thus actually this code has quadratic complexity O( ). But selectio of either aswer (O( log ) ad O( ) is valid. 5. The algorithm suggests that T () T (i) T ( i). By summig this relatioship for all the possible radom values i 0,,...,, we obtai that i average T () (T (0)T ()...T ( )T ( )). Because ( )T ( ) (T (0)... T ( )) ( ), the basic recurrece is as follows: T () ( )T ( ) T ( ), or T () ( )T ( ), or T () T ( ) (). The telescopig results i the followig system of expressios: T () T ( ) T ( ) () T ( ) ( ) T () 3 T () T () 3 T (0) Because T (0) 0, the explicit expressio for T () is: T () 3 ( ) ( ) so that T (). 6. The ier loop has liear complexity c, but the ext called method is of higher complexity log. Because the outer loop is liear i, the overall complexity of this piece of code is log. 4