Ruig Time ( 3.) Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects. The ruig time of a algorithm typically grows with the iput size. Average case time is ofte difficult to determie. We focus o the worst case ruig time. Easier to aalyze Crucial to applicatios such as games, fiace ad robotics Ruig Time 2 8 6 4 2 best case average case worst case 2 3 4 Iput Size Aalysis of Algorithms 2 Experimetal Studies ( 3..) Limitatios of Experimets Write a program implemetig the algorithm Ru the program with iputs of varyig size ad compositio Use a fuctio, like the built-i clock() fuctio, to get a accurate measure of the actual ruig time Plot the results Time (ms) 9 8 7 6 5 4 3 2 5 Iput Size It is ecessary to implemet the algorithm, which may be difficult Results may ot be idicative of the ruig time o other iputs ot icluded i the experimet. I order to compare two algorithms, the same hardware ad software eviromets must be used Aalysis of Algorithms 3 Aalysis of Algorithms 4 Theoretical Aalysis Uses a high-level descriptio of the algorithm istead of a implemetatio Characterizes ruig time as a fuctio of the iput size,. Takes ito accout all possible iputs Allows us to evaluate the speed of a algorithm idepedet of the hardware/software eviromet Pseudocode ( 3..2) High-level descriptio of a algorithm More structured tha Eglish prose Less detailed tha a program Preferred otatio for describig algorithms Hides program desig issues Example: fid max elemet of a array Algorithm arraymax(a, ) Iput array A of itegers Output maximum elemet of A curretmax A[] for i to do if A[i] > curretmax the curretmax A[i] retur curretmax Aalysis of Algorithms 5 Aalysis of Algorithms 6
Pseudocode Details The Radom Access Machie (RAM) Model Cotrol flow if the [else ] while do repeat util for do Idetatio replaces braces Method declaratio Algorithm method (arg [, arg ]) Iput Output Method/Fuctio call var.method (arg [, arg ]) Retur value retur expressio Expressios Assigmet (like = i C++) = Equality testig (like == i C++) 2 Superscripts ad other mathematical formattig allowed A CPU A potetially ubouded bak of memory cells, each of which ca hold a arbitrary umber or character 2 Memory cells are umbered ad accessig ay cell i memory takes uit time. Aalysis of Algorithms 7 Aalysis of Algorithms 8 Primitive Operatios Basic computatios performed by a algorithm Idetifiable i pseudocode Largely idepedet from the programmig laguage Exact defiitio ot importat (we will see why later) Assumed to take a costat amout of time i the RAM model Examples: Evaluatig a expressio Assigig a value to a variable Idexig ito a array Callig a method Returig from a method Coutig Primitive Operatios ( 3.4.) By ispectig the pseudocode, we ca determie the maximum umber of primitive operatios executed by a algorithm, as a fuctio of the iput size Algorithm arraymax(a, ) # operatios curretmax A[] 2 for i to do 2 + if A[i] > curretmax the 2( ) curretmax A[i] 2( ) { icremet couter i } 2( ) retur curretmax Total 7 Aalysis of Algorithms 9 Aalysis of Algorithms Estimatig Ruig Time Algorithm arraymax executes 7 primitive operatios i the worst case. Defie: a = Time take by the fastest primitive operatio b = Time take by the slowest primitive operatio Let T() be worst-case time of arraymax. The a (7 ) T() b(7 ) Hece, the ruig time T() is bouded by two liear fuctios Growth Rate of Ruig Time Chagig the hardware/ software eviromet Affects T() by a costat factor, but Does ot alter the growth rate of T() The liear growth rate of the ruig time T() is a itrisic property of algorithm arraymax Aalysis of Algorithms Aalysis of Algorithms 2
Growth Rates Growth rates of fuctios: Liear Quadratic 2 Cubic 3 T ( ) I a log-log chart, the slope of the lie correspods to the growth rate of the fuctio E+3 E+28 Cubic E+26 E+24 Quadratic E+22 E+2 Liear E+8 E+6 E+4 E+2 E+ E+8 E+6 E+4 E+2 E+ E+ E+2 E+4 E+6 E+8 E+ Costat Factors The growth rate is ot affected by costat factors or lower-order terms Examples T ( ) 2 + 5 is a liear fuctio 5 2 + 8 is a quadratic fuctio E+26 E+24 E+22 E+2 E+8 E+6 E+4 E+2 E+ E+8 E+6 E+4 E+2 E+ Quadratic Quadratic Liear Liear E+ E+2 E+4 E+6 E+8 E+ Aalysis of Algorithms 3 Aalysis of Algorithms 4 Big-Oh Notatio ( 3.5) Give fuctios f() ad g(), we say that f() is O(g()) if there are positive costats c ad such that f() cg() for Example: 2 + is O() 2 + c (c 2) /(c 2) Pick c = 3 ad =,, 3 2+, Big-Oh Example Example: the fuctio 2 is ot O() 2 c c The above iequality caot be satisfied sice c must be a costat,,,,, ^2, Aalysis of Algorithms 5 Aalysis of Algorithms 6 More Big-Oh Examples 7-2 7-2 is O() eed c > ad such that 7-2 c for this is true for c = 7 ad = 3 3 + 2 2 + 5 3 3 + 2 2 + 5 is O( 3 ) eed c > ad such that 3 3 + 2 2 + 5 c 3 for this is true for c = 4 ad = 2 3 log + log log 3 log + log log is O(log ) eed c > ad such that 3 log + log log c log for this is true for c = 4 ad = 2 Aalysis of Algorithms 7 Big-Oh ad Growth Rate The big-oh otatio gives a upper boud o the growth rate of a fuctio The statemet f() is O(g()) meas that the growth rate of f() is o more tha the growth rate of g() We ca use the big-oh otatio to rak fuctios accordig to their growth rate g() grows more f() grows more Same growth f() is O(g()) No g() is O(f()) No Aalysis of Algorithms 8
Big-Oh Rules If is f() a polyomial of degree d, the f() is O( d ), i.e.,. Drop lower-order terms 2. Drop costat factors Use the smallest possible class of fuctios Say 2 is O() istead of 2 is O( 2 ) Use the simplest expressio of the class Say 3 + 5 is O() istead of 3 + 5 is O(3) Asymptotic Algorithm Aalysis The asymptotic aalysis of a algorithm determies the ruig time i big-oh otatio To perform the asymptotic aalysis We fid the worst-case umber of primitive operatios executed as a fuctio of the iput size We express this fuctio with big-oh otatio Example: We determie that algorithm arraymax executes at most 7 primitive operatios We say that algorithm arraymax rus i O() time Sice costat factors ad lower-order terms are evetually dropped ayhow, we ca disregard them whe coutig primitive operatios Aalysis of Algorithms 9 Aalysis of Algorithms 2 Computig Prefix Averages We further illustrate asymptotic aalysis with two algorithms for prefix averages The i-th prefix average of a array X is average of the first (i + ) elemets of X: A[i] = (X[] + X[] + + X[i])/(i+) Computig the array A of prefix averages of aother array X has applicatios to fiacial aalysis 35 3 25 2 5 5 X A 2 3 4 5 6 7 Aalysis of Algorithms 2 Prefix Averages (Quadratic) The followig algorithm computes prefix averages i quadratic time by applyig the defiitio Algorithm prefixaverages(x, ) Iput array X of itegers Output array A of prefix averages of X #operatios A ew array of itegers for i to do s X[] for j to i do + 2 + + ( ) s s + X[j] + 2 + + ( ) A[i] s / (i + ) retur A Aalysis of Algorithms 22 Arithmetic Progressio The ruig time of prefixaverages is O( + 2 + + ) The sum of the first itegers is ( + ) / 2 There is a simple visual proof of this fact Thus, algorithm prefixaverages rus i O( 2 ) time 7 6 5 4 3 2 2 3 4 5 6 Aalysis of Algorithms 23 Prefix Averages (Liear) The followig algorithm computes prefix averages i liear time by keepig a ruig sum Algorithm prefixaverages2(x, ) Iput array X of itegers Output array A of prefix averages of X #operatios A ew array of itegers s for i to do s s + X[i] A[i] s / (i + ) retur A Algorithm prefixaverages2 rus i O() time Aalysis of Algorithms 24
Math you eed to Review Summatios (Sec..3.) Logarithms ad Expoets (Sec..3.2) Proof techiques (Sec..3.3) Basic probability (Sec..3.4) properties of logarithms: log b (xy) = log b x+ log b y log b (x/y) = log b x-log b y log b xa = alog b x log b a= log x a/log x b properties of expoetials: a (b+c) = a b a c a bc = (a b ) c a b /a c = a (b-c) b = a log a b b c = a c*log a b Relatives of Big-Oh big-omega f() is Ω(g()) if there is a costat c > ad a iteger costat such that f() c g() for big-theta f() is Θ(g()) if there are costats c > ad c > ad a iteger costat such that c g() f() c g() for little-oh f() is o(g()) if, for ay costat c >, there is a iteger costat such that f() c g() for little-omega f() is ω(g()) if, for ay costat c >, there is a iteger costat such that f() c g() for Aalysis of Algorithms 25 Aalysis of Algorithms 26 Ituitio for Asymptotic Notatio Big-Oh f() is O(g()) if f() is asymptotically less tha or equal to g() big-omega f() is Ω(g()) if f() is asymptotically greater tha or equal to g() big-theta f() is Θ(g()) if f() is asymptotically equal to g() little-oh f() is o(g()) if f() is asymptotically strictly less tha g() little-omega f() is ω(g()) if is asymptotically strictly greater tha g() Example Uses of the Relatives of Big-Oh 5 2 is Ω( 2 ) f() is Ω(g()) if there is a costat c > ad a iteger costat such that f() c g() for let c = 5 ad = 5 2 is Ω() f() is Ω(g()) if there is a costat c > ad a iteger costat such that f() c g() for let c = ad = 5 2 is ω() f() is ω(g()) if, for ay costat c >, there is a iteger costat such that f() c g() for eed 5 2 c give c, the that satisfies this is c/5 Aalysis of Algorithms 27 Aalysis of Algorithms 28