Running Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis

Transcription

1 Ruig Time ( 3.) Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects. The ruig time of a algorithm typically grows with the iput size. Average case time is ofte difficult to determie. We focus o the worst case ruig time. Easier to aalyze Crucial to applicatios such as games, fiace ad robotics Ruig Time best case average case worst case Iput Size Aalysis of Algorithms 2 Experimetal Studies ( 3..) Limitatios of Experimets Write a program implemetig the algorithm Ru the program with iputs of varyig size ad compositio Use a fuctio, like the built-i clock() fuctio, to get a accurate measure of the actual ruig time Plot the results Time (ms) Iput Size It is ecessary to implemet the algorithm, which may be difficult Results may ot be idicative of the ruig time o other iputs ot icluded i the experimet. I order to compare two algorithms, the same hardware ad software eviromets must be used Aalysis of Algorithms 3 Aalysis of Algorithms 4 Theoretical Aalysis Uses a high-level descriptio of the algorithm istead of a implemetatio Characterizes ruig time as a fuctio of the iput size,. Takes ito accout all possible iputs Allows us to evaluate the speed of a algorithm idepedet of the hardware/software eviromet Pseudocode ( 3..2) High-level descriptio of a algorithm More structured tha Eglish prose Less detailed tha a program Preferred otatio for describig algorithms Hides program desig issues Example: fid max elemet of a array Algorithm arraymax(a, ) Iput array A of itegers Output maximum elemet of A curretmax A[] for i to do if A[i] > curretmax the curretmax A[i] retur curretmax Aalysis of Algorithms 5 Aalysis of Algorithms 6

2 Pseudocode Details The Radom Access Machie (RAM) Model Cotrol flow if the [else ] while do repeat util for do Idetatio replaces braces Method declaratio Algorithm method (arg [, arg ]) Iput Output Method/Fuctio call var.method (arg [, arg ]) Retur value retur expressio Expressios Assigmet (like = i C++) = Equality testig (like == i C++) 2 Superscripts ad other mathematical formattig allowed A CPU A potetially ubouded bak of memory cells, each of which ca hold a arbitrary umber or character 2 Memory cells are umbered ad accessig ay cell i memory takes uit time. Aalysis of Algorithms 7 Aalysis of Algorithms 8 Primitive Operatios Basic computatios performed by a algorithm Idetifiable i pseudocode Largely idepedet from the programmig laguage Exact defiitio ot importat (we will see why later) Assumed to take a costat amout of time i the RAM model Examples: Evaluatig a expressio Assigig a value to a variable Idexig ito a array Callig a method Returig from a method Coutig Primitive Operatios ( 3.4.) By ispectig the pseudocode, we ca determie the maximum umber of primitive operatios executed by a algorithm, as a fuctio of the iput size Algorithm arraymax(a, ) # operatios curretmax A[] 2 for i to do 2 + if A[i] > curretmax the 2( ) curretmax A[i] 2( ) { icremet couter i } 2( ) retur curretmax Total 7 Aalysis of Algorithms 9 Aalysis of Algorithms Estimatig Ruig Time Algorithm arraymax executes 7 primitive operatios i the worst case. Defie: a = Time take by the fastest primitive operatio b = Time take by the slowest primitive operatio Let T() be worst-case time of arraymax. The a (7 ) T() b(7 ) Hece, the ruig time T() is bouded by two liear fuctios Growth Rate of Ruig Time Chagig the hardware/ software eviromet Affects T() by a costat factor, but Does ot alter the growth rate of T() The liear growth rate of the ruig time T() is a itrisic property of algorithm arraymax Aalysis of Algorithms Aalysis of Algorithms 2

3 Growth Rates Growth rates of fuctios: Liear Quadratic 2 Cubic 3 T ( ) I a log-log chart, the slope of the lie correspods to the growth rate of the fuctio E+3 E+28 Cubic E+26 E+24 Quadratic E+22 E+2 Liear E+8 E+6 E+4 E+2 E+ E+8 E+6 E+4 E+2 E+ E+ E+2 E+4 E+6 E+8 E+ Costat Factors The growth rate is ot affected by costat factors or lower-order terms Examples T ( ) is a liear fuctio is a quadratic fuctio E+26 E+24 E+22 E+2 E+8 E+6 E+4 E+2 E+ E+8 E+6 E+4 E+2 E+ Quadratic Quadratic Liear Liear E+ E+2 E+4 E+6 E+8 E+ Aalysis of Algorithms 3 Aalysis of Algorithms 4 Big-Oh Notatio ( 3.5) Give fuctios f() ad g(), we say that f() is O(g()) if there are positive costats c ad such that f() cg() for Example: 2 + is O() 2 + c (c 2) /(c 2) Pick c = 3 ad =,, 3 2+, Big-Oh Example Example: the fuctio 2 is ot O() 2 c c The above iequality caot be satisfied sice c must be a costat,,,,, ^2, Aalysis of Algorithms 5 Aalysis of Algorithms 6 More Big-Oh Examples is O() eed c > ad such that 7-2 c for this is true for c = 7 ad = is O( 3 ) eed c > ad such that c 3 for this is true for c = 4 ad = 2 3 log + log log 3 log + log log is O(log ) eed c > ad such that 3 log + log log c log for this is true for c = 4 ad = 2 Aalysis of Algorithms 7 Big-Oh ad Growth Rate The big-oh otatio gives a upper boud o the growth rate of a fuctio The statemet f() is O(g()) meas that the growth rate of f() is o more tha the growth rate of g() We ca use the big-oh otatio to rak fuctios accordig to their growth rate g() grows more f() grows more Same growth f() is O(g()) No g() is O(f()) No Aalysis of Algorithms 8

4 Big-Oh Rules If is f() a polyomial of degree d, the f() is O( d ), i.e.,. Drop lower-order terms 2. Drop costat factors Use the smallest possible class of fuctios Say 2 is O() istead of 2 is O( 2 ) Use the simplest expressio of the class Say is O() istead of is O(3) Asymptotic Algorithm Aalysis The asymptotic aalysis of a algorithm determies the ruig time i big-oh otatio To perform the asymptotic aalysis We fid the worst-case umber of primitive operatios executed as a fuctio of the iput size We express this fuctio with big-oh otatio Example: We determie that algorithm arraymax executes at most 7 primitive operatios We say that algorithm arraymax rus i O() time Sice costat factors ad lower-order terms are evetually dropped ayhow, we ca disregard them whe coutig primitive operatios Aalysis of Algorithms 9 Aalysis of Algorithms 2 Computig Prefix Averages We further illustrate asymptotic aalysis with two algorithms for prefix averages The i-th prefix average of a array X is average of the first (i + ) elemets of X: A[i] = (X[] + X[] + + X[i])/(i+) Computig the array A of prefix averages of aother array X has applicatios to fiacial aalysis X A Aalysis of Algorithms 2 Prefix Averages (Quadratic) The followig algorithm computes prefix averages i quadratic time by applyig the defiitio Algorithm prefixaverages(x, ) Iput array X of itegers Output array A of prefix averages of X #operatios A ew array of itegers for i to do s X[] for j to i do ( ) s s + X[j] ( ) A[i] s / (i + ) retur A Aalysis of Algorithms 22 Arithmetic Progressio The ruig time of prefixaverages is O( ) The sum of the first itegers is ( + ) / 2 There is a simple visual proof of this fact Thus, algorithm prefixaverages rus i O( 2 ) time Aalysis of Algorithms 23 Prefix Averages (Liear) The followig algorithm computes prefix averages i liear time by keepig a ruig sum Algorithm prefixaverages2(x, ) Iput array X of itegers Output array A of prefix averages of X #operatios A ew array of itegers s for i to do s s + X[i] A[i] s / (i + ) retur A Algorithm prefixaverages2 rus i O() time Aalysis of Algorithms 24

5 Math you eed to Review Summatios (Sec..3.) Logarithms ad Expoets (Sec..3.2) Proof techiques (Sec..3.3) Basic probability (Sec..3.4) properties of logarithms: log b (xy) = log b x+ log b y log b (x/y) = log b x-log b y log b xa = alog b x log b a= log x a/log x b properties of expoetials: a (b+c) = a b a c a bc = (a b ) c a b /a c = a (b-c) b = a log a b b c = a c*log a b Relatives of Big-Oh big-omega f() is Ω(g()) if there is a costat c > ad a iteger costat such that f() c g() for big-theta f() is Θ(g()) if there are costats c > ad c > ad a iteger costat such that c g() f() c g() for little-oh f() is o(g()) if, for ay costat c >, there is a iteger costat such that f() c g() for little-omega f() is ω(g()) if, for ay costat c >, there is a iteger costat such that f() c g() for Aalysis of Algorithms 25 Aalysis of Algorithms 26 Ituitio for Asymptotic Notatio Big-Oh f() is O(g()) if f() is asymptotically less tha or equal to g() big-omega f() is Ω(g()) if f() is asymptotically greater tha or equal to g() big-theta f() is Θ(g()) if f() is asymptotically equal to g() little-oh f() is o(g()) if f() is asymptotically strictly less tha g() little-omega f() is ω(g()) if is asymptotically strictly greater tha g() Example Uses of the Relatives of Big-Oh 5 2 is Ω( 2 ) f() is Ω(g()) if there is a costat c > ad a iteger costat such that f() c g() for let c = 5 ad = 5 2 is Ω() f() is Ω(g()) if there is a costat c > ad a iteger costat such that f() c g() for let c = ad = 5 2 is ω() f() is ω(g()) if, for ay costat c >, there is a iteger costat such that f() c g() for eed 5 2 c give c, the that satisfies this is c/5 Aalysis of Algorithms 27 Aalysis of Algorithms 28