Math Backgroud: Review & Beyod. Asymptotic otatio Desig & Aalysis of Algorithms COMP 48 / ELEC 40 Joh Greier. Math used i asymptotics 3. Recurreces 4. Probabilistic aalysis To do: [CLRS] 4 # T() = O() = T() = 4T(/) + k > Obtaiig Recurreces Itroductory multiplicatio examples: T () = O() = T () = 3T (/) + k > Obtaied from straightforward readig of algorithms. T() = O() = T() = 4T(/) + k > Solvig for Closed Forms T () = O() = T () = 3T (/) + k > log T Θ 3 T' Θ Key observatio: Determiistic algorithms lead to recurreces.. Determie appropriate metric for the size.. Examie how metric chages durig recursio/iteratio. How? I geeral, hard. Solutios ot always kow. Will discuss techiques i a few miutes 3 4 Recurreces Base Case Recurreces T() = 4T(/) + k > T () = 3T (/) + k > T() = O() T() = <b b For costat-sized problems, ca boud algorithm by some costat. This costat is irrelevat for asymptote. Ofte skip writig base case.? What if is odd Next iteratio, is ot itegral. Nosese. T() = 3T(/) + T(/) + k > Above more accurate. The differece rarely matters, so usually igore this detail. 5 6
Two Commo Forms of Recurreces Techiques for Solvig Recurreces Divide-ad-coquer: T() = at(/b) + f() b Substitutio Recursio Tree Master Method for divide & coquer Liear: T() = a T(-)+a T(-) + f() >b Summatio Characteristic Equatio for simple liear for liear 7 8 Techiques: Substitutio Substitutio Example Guess a solutio & check it. Simplified versio of previous example. More detail:. Guess the form of the solutio, usig ukow costats.. Use iductio to fid the costats & verify the solutio. Completely depedet o makig reasoable guesses. T() = O( 3 ). More specifically: T() c 3, for all large eough. 9 0 Substitutio Example Substitutio Example T() c 3 for > 0 T() c 3 for > 0 Prove by strog iductio o. Which meas what exactly? Assume T(k) ck 3 for k> 0, for k<. Show T() c 3 for > 0. Base case, = 0 +: Assume: T(k) ck 3 for k> 0, for k<. Show: T() c 3 for > 0. Awkward. Fortuately, 0 =0 works i these examples.
Substitutio Example Substitutio Example T() c 3 T() c 3 Assume T(k) ck 3, for k<. Show T() c 3. Assume T(k) ck 3, for k<. Show T() c 3. Base case, =: T() = Defiitio. c Choose large eough c for coclusio. Iductive case, >: T() = 4T(/) + Defiitio. 4c(/) 3 + Iductio. = c/ 3 + Algebra. 3 While this is O( 3 ), we re ot doe. Need to show c/ 3 + c 3. Fortuately, the costat factor is shrikig, ot growig. 4 Substitutio Example Substitutio Example T() c 3 Assume T(k) ck 3, for k<. Show T() c 3. Iductive case, >: T() c/ 3 + From before. = c 3 - (c/ 3 - ) Algebra. c 3 For >0, if c. Proved: T() 3 for >0 Thus, T() = O( 3 ). 5 6 Substitutio Example Substitutio Example T() c for > 0 T() = O( ). Same recurrece, but ow try tighter boud. More specifically: T() c for > 0. Follow same steps, ad we get... Assume T(k) ck, for k<. Show T() c. T() = 4T(/) + 4c(/) + = c + Not c! Problem is that the costat is t shrikig. 7 8 3
Substitutio Example Substitutio Example T() c - d Solutio: Use a tighter guess & iductive hypothesis. Subtract a lower-order term a commo techique. T() c - d for >0 Assume T(k) ck - d, for k<. Show T() c - d. Base case, =: T() = Defiitio. c-d Choosig c, d appropriately. Assume T(k) ck - dk, for k<. Show T() c - d. 9 0 Substitutio Example Substitutio Example T() c - d Assume T(k) ck - d, for k<. Show T() c - d. Iductive case, >: T() = 4T(/) + Defiitio. 4(c(/) - d(/)) + Iductio. = c - d + Algebra. = c - d - (d - ) Algebra. c - d Choosig d. Proved: T() for >0 Thus, T() = O( ). Techiques: Recursio Tree Recursio Tree Example Guessig correct aswer ca be difficult! Need a way to obtai appropriate guess. How may levels?. Uroll recurrece to obtai a summatio.. Solve or estimate summatio. Math sometimes tricky. T() / / / / /4 /4 /4 /4 /4 /4 /4 /4 4 log Now, tur picture ito a summatio 3. Use solutio as a guess i substitutio. 4 #levels 3 I this example, all terms o a level are the same. Commo, but ot always true. 4 4
Recursio Tree Example Recursio Tree Example lg i T() i0 T() = T(/3) + T(/3) + > How may levels? / T() / / / /4 /4 /4 /4 /4 /4 /4 /4 4 4 lg lg lg lg T() /3 /3 /9 /9 /9 4/9 log 3/ But, ot all braches have same depth! Makes cost ear the leaves hard to calculate. Estimate! T() = + + 4 + + lg + 4 lg = Θ( ) = ( + + 4 + + lg ) + lg 4 5 6 Recursio Tree Example Recursio Tree Example T() = T(/3) + T(/3) + > T() = T(/3) + T(/3) + > T() T() /3 /3 /9 /9 /9 4/9 Overestimate. Cosider all braches to be of max depth. T() (log 3/ - ) + T() = O( log ) /3 /3 /9 /9 /9 4/9 Uderestimate. Cout the complete levels, & igore the rest. T() (log 3 ) T() = Ω( log ) #levels = log 3/ #levels = log 3 Thus, T() = Θ( log ) 7 8 Techiques: Master Method Cookbook solutio for may recurreces of the form T() = a T(/b) + f() where a, b>, f() asymptotically positive Master Method Case T() = a T(/b) + f() f() = O( logb a - ) for some >0 T() = Θ( logb a ) T() = 7T(/) + c a=7, b= E.g., Strasse matrix multiplicatio. First describe its cases, the outlie proof. c =? O( logb a - ) = O( log 7 - ) O(.8 - ) Yes, for ay 0.8. T() = Θ( lg 7 ) 9 30 5
Master Method Case T() = a T(/b) + f() f() = Θ( logb a ) T() = Θ( logb a lg ) T() = T(/) + c a=, b= E.g., mergesort. c =? Θ( logb a ) = Θ( log ) = Θ() Yes. Master Method Case 3 T() = a T(/b) + f() f() = Ω( logb a + ) for some >0 ad af(/b) cf() for some c< ad all large eough T() = Θ(f()) T() = 4T(/) + 3 a=4, b= 3 =? Ω( logb a + ) = Ω( log 4 + ) = Ω( + ) Yes, for ay. I.e., is the costat factor shrikig? T() = Θ( lg ) 4(/) 3 = ½ 3? c 3 Yes, for ay c ½. T() = Θ( 3 ) 3 3 Master Method Case 4 Master Method Case 4 T() = a T(/b) + f() T() = a T(/b) + f() Noe of previous apply. Master method does t help. Noe of previous apply. Master method does t help. T() = 4T(/) + /lg a=4, b= T() = 4T(/) + /lg a=4, b= Case? /lg =? O( logb a - ) = O( log 4 - ) = O( - ) = O( / ) Case? /lg =? Θ( logb a ) = Θ( log 4 ) = Θ( ) No, sice lg is asymptotically <. Thus, /lg is asymptotically > /. 33 No. 34 Master Method Case 4 Master Method Proof Outlie T() = a T(/b) + f() Noe of previous apply. Master method does t help. T() = 4T(/) + /lg a=4, b= Case 3? /lg =? Ω( logb a + ) = Ω( log 4 + ) = Ω( + ) T() = a T(/b) + f() T() f() f() /b /b af(/b) /b /b /b /b a f(/b ) a #levels How may levels? log b T log b i a f / b i0 logba i No, sice /lg is asymptotically <. Cases correspod to determiig which term domiates & how to compute sum. 35 36 6
Techique: Summatio For liear recurreces with oe recursive term. Techiques: Characteristic Equatio Applies to liear recurreces T() = T(-) + f()? Homogeous: a = c a - + c a - + + c k a -k T() = T(-) + f()? Nohomogeous: a = c a - + c a - + + c k a -k + F() 37 38 7