Learning outcomes. Algorithms and Data Structures. Time Complexity Analysis. Time Complexity Analysis How fast is the algorithm? Prof. Dr.

Transcription

1 Algorithms ad Data Structures Algorithm efficiecy Learig outcomes Able to carry out simple asymptotic aalysisof algorithms Prof. Dr. Qi Xi 2 Time Complexity Aalysis How fast is the algorithm? Code the algorithm ad ru the program, the measure the ruig time 1. Dep o the speed of the computer 2. Waste time codig ad testig if the algorithm is slow Time Complexity Aalysis How to measure efficiecy? Number of operatios usually expressed i terms of iput size If we doubled/trebled the iput size, how much loger would the algorithm take? Idetify some importat operatios/steps ad cout how may times these operatios/steps eeded to be executed 3 4

2 Why efficiecy matters? speed of computatio by hardware has bee improved efficiecy still matters ambitio for computer applicatios grow with computer power demad a great icrease i speed of computatio Amout of data hadled matches speed icrease? Whe computatio speed vastly icreased, ca we hadle much more data? Suppose a algorithm takes 2 comparisos to sort umbers we eed 1 sec to sort 5 umbers (25 comparisos) computig speed icreases by factor of 100 Usig 1 sec, we ca ow perform 100x25 comparisos, i.e., to sort 50 umbers With 100 times speedup, oly sort 10 times more umbers! 5 6 Time/Space Complexity Aalysis Importat operatio of summatio: additio How may additios this algorithm requires? iput sum = 0 for i = 1 to do begi sum = sum + i output sum Look for improvemet Mathematical formula gives us a alterative way to fid the sum of first umbers: = (+1)/2 iput sum = *(+1)/2 output sum We eed additios (dep o the iput size ) We eed 3 variables, sum, & i => eeds 3 memory space We oly eed 3 operatios: 1 additio, 1 multiplicatio, ad 1 divisio (o matter what the iput size is) I other cases, space complexity may dep o the iput size 7 8

3 Improve Searchig We've leart sequetial search ad it takes comparisos i the worst case. If the umbers are pre-sorted, the we ca improve the time complexity of searchig by biary search. Biary Search more efficiet way of searchig whe the sequece of umbers is pre-sorted Iput:a sequece of sortedumbers a 1, a 2,, a i ascig order ad a umber X Idea of algorithm: compare X with umber i the middle the focus o oly the first half or the secod half (dep o whether X is smaller or greater tha the middle umber) reduce the amout of umbers to be searched by half 9 10 Biary Search (2) To fid X foud! 10 os Biary Search (3) To fid X ot foud! 10 os 11 12

4 Biary Search Pseudo Code first = 1 last = while (first <= last) do begi // check with o. i middle report "Not Foud!" is the floor fuctio, trucates the decimal part mid = (first+last)/2 if (X == a[mid]) report "Foud!" & stop if (X < a[mid]) last = mid-1 first = mid+1 Biary Search Pseudo Code while first <= last do begi mid = (first+last)/2 if (X == a[mid]) report "Foud!" & stop if (X < a[mid]) last = mid-1 first = mid+1 Modify it to iclude stoppig coditios i the while loop Best case: Number of Comparisos X is the umber i the middle 1 compariso Worst case: at most log 2 +1 comparisos Why? Every compariso reduces the amout of umbers by at least half E.g., first=1, last= while (first <= last) do begi mid = (first+last)/2 if (X == a[mid]) report "Foud!" & stop if (X < a[mid]) last = mid-1 first = mid+1 report "Not Foud!" Time complexity -Big O otatio 15

5 Note o Logarithm Logarithm is the iverse of the power fuctio log 2 2 x = x log 2 x*y = log 2 x + log 2 y For example, log 21 = log 22 0 = 0 log 2 2 = log = 1 log 2 x/y = log 2 x - log 2 y log 2 4 = log = 2 log 2 16 = log = 4 log = log = 8 log = log = 10 log 2 4*8 = log log 2 8 = 2+3 = 5 log 2 16*16 = log log 2 16 = 8 log 2 32/8 = log log 2 8 = 5-3 = 2 log 2 1/4 = log log 2 4 = 0-2 = -2 Which algorithm is the fastest? Cosider a problem that ca be solved by 5 algorithms A 1, A 2, A 3, A 4, A 5 usig differet umber of operatios (time complexity). f 1 () = f 2 () = 10 log f 3 () = f 4 () = 2 2 f 5 () = 2 /8 - / f 1 () = f 2 () = 10 log f 3 () = E+05 1E+06 4E+06 f 4 () = E+05 2E+06 8E+06 f 5 () = 2 / 8 - / E+08 2E+18 f 5 () f 3 () f 1 () 17 Deps o the size of the iput! What do we observe? Time f1() = ff2() 1 = log ff3() 2 = log f4() = 22 f f5() 3 () = 2/ /4 + 2 f 4 () = 2 2 f 5 () = 2 /8 - / There is huge differece betwee fuctios ivolvig powers of (e.g.,, 2, called polyomial fuctios) ad fuctios ivolvig powerig by (e.g., 2, 3, called expoetial fuctios) Amog polyomial fuctios, those with same order of power are more comparable e.g., f 3 () = ad f 4 () =

6 Growth of fuctios Relative growth rate 2 2 log c Hierarchy of fuctios We ca defie a hierarchy of fuctios each havig a greater order of growth tha its predecessor: Hierarchy of fuctios (2) 1 log k... 2 costat logarithmic polyomial expoetial 1 log k... 2 costat logarithmic polyomial expoetial We ca further refie the hierarchy by isertig log betwee ad 2, 2 log betwee 2 ad 3, ad so o. Note:as we move from left to right, successive fuctios have greater order of growththa the previous oes. As icreases, the values of the later fuctios icrease more rapidly tha the earlier oes. Relative growth rates icrease 23 24

7 Hierarchy of fuctios (3) What about log 3 &? Which is higher i hierarchy? (log ) 3 Remember: = 2 log So we are comparig (log ) 3 & 2 log log 3 is lower tha i the hierarchy Similarly, log k is lower tha i the hierarchy, for ay costat k Hierarchy of fuctios (4) 1 log k... 2 costat logarithmic polyomial expoetial Now, whe we have a fuctio, we ca classify the fuctio to some fuctio i the hierarchy: For example, f() = The term with the highest power is 2 3. The growth rate of f() is domiated by 3. This cocept is captured by Big-O otatio Big-O otatio f() = O(g()) [read as f() is of order g()] Roughly speakig, this meas f() is at most a costat times g() for all large Examples 2 3 = O( 3 ) 3 2 = O( 2 ) 2 log = O( log ) = O( 3 ) Fuctios o L.H.S ad R.H.S are said to have the same order of growth 27 Determie the order of growthof the followig fuctios log log Exercise O( 3 ) O( 3 ) O( 2 log ) O(2 ) Look for the term highest i the hierarchy 28

8 More Exercise Are the followigs correct? O( 1. 2 log O( 2 log )? ) O()? YES O( 20 )? O(2 ) log + O( 2 log )? O( 3 ) Big-O otatio - formal defiitio f() = O(g()) There exists a costat cad o such that f() cg() for all > o Graphical Illustratio c o > o the f() cg() Time c g() f() iput size () Time = O() c=2, 0 =60 0 Examples iput size () Time costat c & o such that > o, f() c g() c=3, 0 =600 3 log f() Which oe is the fastest? Usually we are oly iterested i the asymptotic time complexity i.e., whe is large O(log ) < O(log 2 ) < O( ) < O() < O( log ) < O( 2 ) < O(2 ) f() = 2log = O( log ) log iput size () 31 32

9 Proof of order of growth Prove that is O( 2 ) Sice 2 1, we have = Therefore, by defiitio, is O( 2 ). Alteratively, Sice 4 2 4, we have = N.B.: plottig a graph is NOT a proof Therefore, by defiitio, is O( 2 ). 33 Proof of order of growth (2) Prove that 2 log + 3 is O( 2 log ) Sice 2 1 ad 1 log 2, we have log 2, ad 2 log log log 2. = 4 2 log Therefore, by defiitio, 2 log + 3 is O( 2 log ). 34 Proof of order of growth (3) Alteratively, Sice 3 2 log 3, we have Prove that 2 log + 3 is O( 2 log ) 2 log log + 2 log = 2 2 log 3. Therefore, by defiitio, 2 log + 3 is O( 2 log ). 35 Exercise Prove the order of growth is O( 3 ) a) 3 = 3 b) c) log is O( 3 ) a) 3 = 3 b) c) a) 2 2 log 3 4 a) 2 2 log b) 3 = 3 b) 3 = 3 c) c) d) 3 1 d) log log

10 Exercise Prove the order of growth is O( 3 ) a) 3 = 3 b) c) log is O( 3 ) a) 2 2 log 3 4 b) 3 = 3 c) d) log log > 3 for 2<<4 2 log Exercise cot d Prove the order of growth log is O( 2 log ) a) log 4 b) 2 log = 2 log log 2 2 log is O(2 ) a) b) 2 = *2 7 a) log 2 b) 2 log = 2 log log 3 2 log 2 a) 2 2 2*2 4 b) 2 = * Some algorithms we leart Sum of 1 st o. iput sum = *(+1)/2 output sum O(1) O(?) iput sum = 0 for i = 1 to do begi sum = sum + i output sum O() O(?) Time complexity of this? for i = 1 to 2 do for j = 1 to do x = x + 1 O( 2 ) O(?) Mi value amog o. loc = 1 for i = 2 to do if (a[i] < a[loc]) the loc = i output a[loc] O() O(?) The outer loop iterates for 2 times. The ier loop iterates for times for each i. Total: 2 * =

11 What about this? i = 1 O(log ) cout = 0 while i < begi O(?) i = 2 * i cout = cout + 1 output cout suppose =8 (@ of ) iteratio i cout suppose =32 (@ of ) iteratio i cout