Algorithms Chapter 4 Recurrences

Size: px

Start display at page:

Download "Algorithms Chapter 4 Recurrences"

Lily McDonald
7 years ago
Views:

1 Algorithms Chpter 4 Recurrences Outline The substitution method The recursion tree method The mster method Instructor: Ching Chi Lin 林清池助理教授 chingchilin@gmilcom Deprtment of Computer Science nd Engineering Ntionl Tiwn Ocen University The purpose of this chpter When n lgorithm contins recursive cll to itself, its running time often be described by recurrence A recurrence is n eqution or inequlity tht describes function in terms of its vlue on smll inputs For exmple: the worst cse running time of Merge Sort The solution is T(n=(nlgn Three methods for solving recurrences the substitution method the recursion tree method the mster method (1 T ( n T ( n / ( n if n 1, if n 1 Techniclities 1/ The running time T(n is only defined when n is n integer, since the size of the input is lwys n integer for most lgorithms For exmple: the running time of Merge Sort is relly Typiclly, we ignore the boundry conditions Since the running time of n lgorithm on constnt sized input is constnt, we hve T(n = (1 for sufficiently smll n Thus, we rewrite the recurrence s 4 (1 T ( n T ( n / T ( n / ( n T(n =T(n/+ (n if n 1, if n 1

2 Techniclities / When we stte nd solve recurrences, we often omit floors, ceilings, nd boundry conditions We forge hed without these detils nd lter determine whether or not they mtter They usully don't, but it is importnt to know when they do The substitution method 1/ The substitution method entils two steps: Guess the form of the solution Use mthemticl induction to find the constnts nd show tht the solution works This method is powerful, but it obviously be pplied only in cses when it is esy to guess the form of the nswer 5 6 The substitution method / For exmple: determine n upper bound on the recurrence guess tht the solution is T(n=O(nlgn prove tht there exist positive constnts c>0 nd n 0 such tht T(nlgn for ll nn 0 Bsis step: T(n =T( n/ + n, where T(1=1 when n=1, T(1 c 1 1lg 1=0, which is odds with T(1=1 since the recurrence does not depend directly on T(1, we replce T(1 by T(=4 nd T(=5 s the bse cses T ( c 1 lg nd T(c 1 lg for ny choice of c 1 thus, we choose c 1 = nd n 0 = The substitution method / Induction step: ssume T( n/ c n/ lg( n/ for n/ then, T(n (c n/ lg( n/ +n c nlg(n/+ n = c nlgn-c nlg+ n = c nlgn-c n + n c nlgn the lst step holds s long s c 1 There exist positive constnts c = mx{,1} nd n 0 = such tht T(n lgn for ll n n 0 7 8

3 Mking good guess Experience: T(n=T( n/ +17+n when n is lrge, the difference between T ( n/ nd T ( n/ +17 is not tht lrge: both cut n nerly evenly in hlf we mke the guess tht T (n = O(nlgn Loose upper nd lower bounds: T(n=T( n/ +n prove lower bound of T (n = (n nd n upper bound of T(n = O(n grdully lower the upper bound nd rise the lower bound until we converge on the tight solution of T (n = (nlgn Recursion trees: will be introduced lter Subtleties Consider the recurrence: T(n = T( n/ +T ( n/ +1 guess tht the solution is O(n, ie, T(n then, T(n c n/ +(c n/+1 = +1 c does not exist new guess T(n-b then, T(n (c n/ -b+(c n/-b+1 = -b+1 -b for b1 lso, the constnt c must be chosen lrge enough to hndle the boundry conditions 9 10 Avoiding pitflls It is esy to err in the use of symptotic nottion For exmple: T(n =T( n/ + n guess tht the solution is O(n, ie, T(n then, T(n (c n/ +n +n = O(n wrong the error is tht we hven't proved the exct form of the inductive hypothesis, tht is, tht T(n Chnging vribles Sometimes, little lgebric mnipultion mke n unknown recurrence similr to one you hve seen before For exmple, consider the recurrence T(n =T( n 1/ +lgn Renming m=lgn yields T( m =T( m/ + m Renming S(m=T( m produces S(m =S(m/ + m S(m = O(mlgm Chnging bck S(mtoT(m, we obtin T(n = T( m = S(m = O(mlgm = O(lgnlglgn 11 1

4 Outline The substitution method The recursion tree method The mster method The recursion tree method A recursion tree is best used to generte good guess, which is then verified by the substitution method Tolerting smll mount of sloppiness, we could use recursion tree to generte good guess One lso use recursion tree s direct proof of solution to recurrence Ides: in recursion tree, ech node represents the cost of single subproblem sum the costs within ech level to obtin set of per level costs sum ll the per level costs to determine the totl cost 1 14 An exmple For exmple: T(n =T( n/4 + (n The construction of recursion tree 1/ T(n =T(n/4 + Tolerting the sloppiness: ignore the floor in the recurrence ssume n is n exct power of 4 Rewrite the recurrence s T(n =T(n/4 + T(n ( T(n/4 T(n/4 T(n/4 (b c(n/4 c(n/4 c(n/4 T(n/ T(n/ T(n/ T(n/ T(n/ T(n/ T(n/ T(n/ T(n/ (c 15

5 log 4 n The construction of recursion tree / Determine the cost of the tree 1/ 17 c(n/4 c(n/4 c(n/4 c(n/ c(n/ c(n/ c(n/ c(n/ c(n/ c(n/ c(n/ c(n/ (/ (/ T(1T(1 T(1 T(1 T(1T(1 (n log 4 log4 n = n log 4 (d Totl:O(n The subproblem size for node t depth i is n/4 i Thus, the tree hs log 4 n + 1 levels (0, 1,,, log 4 n Ech node t depth i, hs cost of c(n/4 i for 0 i log 4 n-1 So, the totl cost over ll nodes t depth i is i *c(n/4 i =(/ i log4 log4 The lst level, t log 4 n,hs n n nodes The cost of the entire tree: 18 T ( n log 4 n 1 i0 ( ( i log 4 n ( / 1 ( / 1 ( n log 4 ( n ( log 4 log 4 n1 ( n log 4 Determine the cost of the tree / Tke dvntge of smll mounts of sloppiness, we hve T ( n log 4 n 1 i 0 i i log 4 ( ( n i 0 1 log 4 ( n 1 ( / ( n 1 O ( n ( ( n log 4 Thus, we hve derived guess of T(n=O(n log 4 Verify the correctness of our guess Now we use the substitution method to verify tht our guess is correct We wnt to show tht T(n dn for some constnt d > 0 Using the sme constnt c > 0 s before, we hve T ( n T ( n / 4 d n / 4 d ( n / 4 /dn dn, where the lst step holds for d (/1c 19 0

6 Another exmple Another exmple: T(n = T(n/+T(n/+O(n The recursion tree: c(n/ c(n/ log / n c(n/9 c(n/9 c(n/9 c(4n/9 Determine the cost of the tree The height of tree is log / n log/ log/ The recursion tree hs fewer thn n n leves The totl cost of ll leves would then be ( n log /, which is ( n lg n Also, not ll levels contribute cost of exctly Thus, we derived guess of T(n=O(nlgn Totl:O(nlgn Verify the correctness of our guess We verify the guess by the substitution method We hve T( n T( n / T(n / d( n / lg( n / d(n / lg(n / ( d( n / lgn d( n / lg ( d(n / lgn d(n / lg(/ dnlgn d(( n / lg (n / lg(/ dnlgn d(( n / lg (n / lg (n / lg dnlgn dn(lg / dnlgn for d c/(lg-(/ Outline The substitution method The recursion tree method The mster method 4

7 The mster method 1/ The mster method provides "cookbook" method for solving recurrences of the form T(n = T(n/b + f(n 1 nd b>1 re constnts f(n is n symptoticlly positive function It requires memoriztion of three cses, but then the solution of mny recurrences be determined quite esily The mster method / The recurrence T(n = T(n/b + f(ndescribes the running time of n lgorithm tht divides problem of size n into subproblems, ech of size n/b ech of subproblems is solved recursively in time T(n/b the cost of dividing nd combining the results is f(n For exmple, the recurrence rising from the MERGE SORT procedure hs =, b =, nd f (n = (n Normlly, we omit the floor nd ceiling functions when writing divide nd conquer recurrences of this form 5 6 Mster theorem Mster theorem: Let 1 nd b > 1 be constnts, let f(n be function, nd let T(n be defined on the nonnegtive integers by the recurrence where we interpret n/b to men either n/b or n/b Then, T(n be bounded symptoticlly s follows log 1 If f ( n O( n for some constnt > 0, then T ( n ( n b log If f ( n ( n b, then T ( n ( n lgn If f ( n ( n for some constnt > 0, nd if f(n/b cf(n for some constnt c<1 nd ll sufficiently lrge n, then T ( n ( f ( n 7 T(n = T(n/b + f(n Intuition behind the mster method Intuitively, the solution to the recurrence is determined by compring the two functions f(n nd n log b b Cse 1: if n log is symptoticlly lrger thn f(n by fctor of n for some constnt > 0, then the solution is T ( n ( n b Cse : if n log is symptoticlly equl to f(n, then the solution is T ( n ( n lgn Cse : if is symptoticlly smller then f(nby fctor of n, nd the function f(n stisfies the "regulrity condition tht f(n/b cf(n, then the solution is T ( n ( f ( n The three cses do not cover ll the possibilities for f(n 8 n log b

8 Using the mster method 1/ Exmple 1: T(n =9T(n/ + n For this recurrence, we hve = 9, b =, f(n = n Thus, log b log n n 9 ( n Since ( ( log 9 f n O n,where = 1, we pply cse 1 The solution is T(n = (n Exmple : T(n = T(n/ + 1 For this recurrence, we hve = 1, b = /, f(n = 1 log / 1 0 Thus, n n n 1 Since f ( n O( n (1,we pply cse The solution is T(n = (lgn Using the mster method / Exmple : T(n =T(n/4 + nlgn For this recurrence, we hve =, b = 4, f(n = nlgn Thus, log b log 079 n n 4 O( n For sufficiently lrge n, f(n/b = (n/4 lg(n/4 (/4nlgn = cf(n for c = /4 log Since ( ( 4 f n n with nd the regulrity condition holds for f(n cse pplies The solution is T(n = (nlgn 9 0 Using the mster method / Exmple 4: T(n =T(n/ + nlgn For this recurrence, we hve =, b =, f(n = nlgn The function f(n = nlgn is symptoticlly lrger log thn n b log n n But, it is not polynomilly lrger since the rtio f ( n / n ( n lg n / n lg n is symptoticlly less thn n for ny positive constnt Consequently, the recurrence flls into the gp between cse nd cse If g(n is symptoticlly lrger thn f(n by fctor of n for some constnt >0, then we sid g(n is polynomilly lrger thn f(n 1

Example 27.1 Draw a Venn diagram to show the relationship between counting numbers, whole numbers, integers, and rational numbers.

Example 27.1 Draw a Venn diagram to show the relationship between counting numbers, whole numbers, integers, and rational numbers. 2 Rtionl Numbers Integers such s 5 were importnt when solving the eqution x+5 = 0. In similr wy, frctions re importnt for solving equtions like 2x = 1. Wht bout equtions like 2x + 1 = 0? Equtions of this