CMPS Aalyss of Algorthms Summer 5 Recurrece Relatos Whe aalyzg the ru tme of recursve algorthms we are ofte led to cosder fuctos T ( whch are defed by recurrece relatos of a certa form A typcal example would be c T ( T ( / ) + T ( / ) + d > where c, d are fxed costats The specfc values of these costats are mportat for determg the explct soluto to the recurrece Ofte however we are oly cocered wth fdg a asymptotc (upper, lower, or tght) boud o the soluto We call such a boud a asymptotc soluto to the recurrece I the above example the partcular costats c ad d have o effect o the asymptotc soluto We may therefore wrte our recurrece as Θ() T ( T ( / ) + T ( / ) + Θ( > I what follows we'll show that a tght asymptotc soluto to the above recurrece s T ( Θ( log We wll study the followg methods for solvg recurreces Substtuto Method Ths method cossts of guessg a asymptotc (upper or lower) boud o the soluto, ad tryg to prove t by ducto Recurso Tree Method Iterato Method These are two closely related methods for expressg T ( as a summato whch ca the be aalyzed A recurso tree s a graphcal depcto of the etre set of recursve vocatos of the fucto T The goal of drawg a recurso tree s to obta a guess whch ca the be verfed by the more rgorous substtuto method Iterato cossts of repeatedly substtutg the recurrece to tself to obta a terated (e summato expresso Master Method Ths s a cooboo method for determg asymptotc solutos to recurreces of a specfc form The Substtuto Method We beg wth the followg example T ( T ( /) + < We wll see later that T ( Θ( log( ), but for the tme beg, suppose that we are able to guess (somehow) that T ( O( log( ) I order to prove ths we must fd postve umbers c ad such
that T ( clog( for all If we ew approprate values for these costats, we could prove ths equalty by ducto Our goal the s to determe c ad such that the ducto proof wll wor I what follows t wll be useful to tae log to mea log We beg by mmcg the ducto step (IId) to fd c: Let > ad assume for all < that ( ) c log( ) T / c / log / We must show that T ( clog( Observe ( ) ( ) T I partcular, tag / T ( / ) c /log( / ) + c ( /)log( / + (sce x T ( + (by the recurrece for T) (by the ducto hypothess) ) c (log( ) + c log( c + x for all x) To obta T ( clog( we eed to have c +, whch mples c gves us We see that as log as c s chose to satsfy the costrat c, the ducto step wll go through It remas to determe the costat The base step requres c log( ) Ths ca be vewed as a sgle costrat o the two parameters c ad, whch dcates that we have some freedom choosg them I other words, the costrats do ot uquely determe both costats, so we must mae some arbtrary choce Choosg s algebracally coveet Ths yelds T ( ) c, whece c T () / Sce T ( ) 9 (usg the recurrece) we have the costrat c Clearly f we choose c, both costrats ( c ad c ) wll be satsfed It s mportat to ote that we have ot proved aythg yet Everythg we have doe up to ths pot has bee scratch wor wth the goal of fdg approprate values for c ad It remas to preset a complete ducto proof of the asserto: T ( log( for all Proof: I Sce T ( ) T ( /) + T () + + 9 ad log() 9 the base case reduces to 9 9 whch s true II Let > ad assume for all < that T ( ) log( ) The T ( / ) /log( / ) + (by the ducto hypothess lettg / 9 ( /)log( / + (sce x T ( + (by the recurrece for T) ) (log( ) + log( log( x for all x) ) It ow follows that T ( log( for all ///
It s a somewhat more dffcult exercse to prove by the same techque that T ( Ω( log( ), ad therefore T ( Θ( log( ) Exercse Determe postve umbers c ad such that T ( clog( for all Ht: Use the followg facts: () x > x, ad () logx log( x) for x / (Wth some effort, ts possble to show that c / 4 ad 4 wor) The ext example llustrates oe complcato that ca arse the substtuto method Defe T ( by the recurrece T ( T ( / ) + We guess that T ( O( To prove ths guess, we attempt to fd postve costats c ad such that T ( c for all As before we proceed by mmcg the ducto step Let > ad assume for all the rage < that T ( ) c I partcular, for / we have T ( / ) c /, ad thus ( / ) / T ( T + (by the recurrece for T ( ) c + (by the ducto hypothess) c ( / ) + c + Our goal s to determe c so that the ducto step s feasble We eed to reach the cocluso that T ( c, ad to do ths we must determe a umber c such that c + c But ths equalty s obvously false for all postve c Apparetly the ducto step caot be made to wor, whch mght lead us to beleve that our guess s correct Actually T ( O( s correct (as ca be see by other methods), so we tae a dfferet approach Our trc s to subtract a lower order term from the rght sde of the equalty we wsh to prove To be precse, we see postve costats c,, ad b such that T ( c b for all Observe that ths equalty also mples T ( O( by a result proved the hadout o asymptotc growth rates It may seem couter-tutve to attempt to prove a equalty whch s stroger tha the oe that faled, but otce that ths strategy allows us to use a stroger ducto hypothess Aga let > ad assume for all the rage < that T ( ) c b, ad partcular T / c / Thus ( ) b T ( / ) ( / b) ( c ( / ) b) (sce x T ( + (by the recurrece for T ( ) c + (by the ducto hypothess) + c b + x ) If we tae b the T ( c b, as desred For the base case let We must show T ( ) c, whch says c T ( ) + Thus we may set, c, ad b
Exercse Use ducto to prove that T ( for all, whece T ( O( Exercse Referrg to the prevous example, use the substtuto method to show that T ( Ω(, whece T ( Θ( The Recurso Tree Iterato Method The recurso tree method ca be used to geerate a good guess for a asymptotc boud o a recurrece Ths guess ca the be verfed by the substtuto method Sce our guess wll be verfed, we ca tae some lbertes our calculatos, such as droppg floors ad celgs or restrctg the values of Let us beg wth the example Θ() T ( T ( /) + Θ( < We smplfy the recurrece by droppg the floor ad replacg Θ ( by to get T ( T ( /) + Each ode a recurso tree represets oe term the calculato of T ( obtaed by recursvely substtutg the expresso for T to tself We costruct a sequece of such trees of creasg depths th tree st tree d tree T ( T ( /) T ( /) ( /) ( /) T ( / ) T ( / ) T ( / ) T ( / ) rd tree ( /) ( /) ( / ) ( / ) ( / ) ( / ) T ( / ) T ( / ) T ( / ) T ( / ) T ( / ) T ( / ) T ( / ) T ( / ) By summg the odes ay oe of these trees, we obta a expresso for T ( After teratos of ths process we reach a tree whch all bottom level odes are T ( / )
th tree ( /) ( /) ( / ) ( / ) ( / ) ( / ) T ( / ) T ( / ) T ( / ) T ( / ) T ( / ) Note that there are odes at depth, each of whch has value / (for ) The sequece of trees termates whe all bottom level odes are T (), e whe /, whch mples log ( log ( log () The umber of odes at ths bottom level s therefore Summg all odes the fal recurso tree gves us the followg expresso for T ( T ( ( / ) + log () T () log () (/) + T () (/) () log + T () (/) log (/) ( () + Θ ( ) ) log If we see a asymptotc upper boud we may drop the egatve term to obta ( ) ( () T + Θ ) Sce log () < the frst term domates, ad so we guess: T ( O( Exercse Prove that ths guess s correct usg the substtuto method It s sometmes possble to push these computatos a lttle further to obta a asymptotc estmate drectly, wth o smplfyg assumptos We call ths the terato method We llustrate o the very same example T ( T ( / ) + < Here we do ot assume that s a exact power of, ad eep the floor Substtutg ths expresso to tself tmes yelds:
T ( + T ( /) ( / T ( //)) + + / T ( / ) / + ( / T ( / / ) / + / T ( / ) + + + + + + / + T ( / ) The process must termate whe s chose so that / < / < + < log ( < + log (, whch mples Wth ths value of we have ( / T ), whece ( / T + Ths expresso essece coverts the computato of T ( from a recursve algorthm to a teratve algorthm, whch s why we call ths the terato method Ths summato ca ow be used to obta asymptotc upper ad lower bouds for T ( / T ( + log ( ( / ) + ( /) + log () (/) + (/) (/) + log () log ) ( ) ( (sce x x ) log () + (droppg the egatve term) O( (sce log () < ) Thus T ( O( has bee proved Note that our proof s rgorous sce at o pot have we made ay smplfyg assumptos (such as beg a exact power of ) Therefore there s o eed to verfy T ( O( by aother method, such as substtuto I a smlar fasho, oe ca show T ( Ω(
/ T ( + log ( ( / ) ) + x ) (sce > x ( /) + + log () log () x ) log ( log () + + (sce x log () + Ω( (sce log () < ) We've show T ( Ω(, ad sce T ( O(, we ow have T( Θ( The terato method ca eve be used to fd exact solutos to some recurreces, as opposed to the asymptotc solutos we have bee satsfed wth up to ow For example defe the fucto T ( by T ( ) T () 5, ad T ( T( ) + for Iteratg tmes we fd T ( + T ( ) + ( ) + T ( 4) + ( ) + ( 4) + T ( 6) ( ) + + T ( ) + T( ) ( ) + + T ( ) + ( ) + T ( ) Ths process must termate whe s chose so that ether or, whch s where the recurrece 'bottoms out' so to spea Ths mples < < < + /
Thus f / we have T ( ) 5, ad therefore the exact soluto to the above recurrece s gve by / / ( / ) 5 T ( + The asymptotc soluto s ow readly see to be T ( Θ( ) Exercse Defe T ( by T ( ) ad ( T( / ) + ducto hadout) Use the terato method to show ( lg( T for (Recall ths was example 5 the T for all, whece T ( Θ(log( ) The Master Method Ths s a method for fdg (asymptotc) solutos to recurreces of the form T ( at ( / b) + f ( where a, b >, ad the fucto f ( s asymptotcally postve Here T ( / b) deotes ether T ( / b) or T ( / b), ad t s uderstood that T ( Θ() for some fte set of tal terms Such a recurrece descrbes the ru tme of a 'dvde ad coquer' algorthm whch dvdes a problem of sze to a subproblems, each of sze / b I ths cotext f ( represets the cost of dog the dvdg ad re-combg Master Theorem Let a, b >, f ( be asymptotcally postve, ad let T ( be defed by T ( at ( / b) + f ( The we have three cases: ε () If f ( O( ) for some ε >, the T ( Θ( ) () If f ( Θ( ), the T ( Θ( log( ) +ε () If f ( Ω( ) for some > / b) suffcetly large, the ( ) ε, ad f af ( cf ( for some < c < ad for all T ( Θ f ( Remars I each case we compare f ( to the fucto fucto s asymptotcally larger I case () the class ( ) log b, ad the soluto s determed by whch s polyomally larger tha ( f ad the soluto s Θ I case () f ( s polyomally larger (ad a addtoal regularty codto s Θ f ( I case () the fuctos are asymptotcally equvalet ad the soluto met) so the soluto s ( ) log b Θ log(, whch s the same as ( f ( log( ) Θ To say that s polyomally larger tha f ( as (), meas that f ( s bouded above by a fucto whch s s the class ( ) ε smaller tha by a polyomal factor, amely for some ε > Note that the cocluso reached by the master theorem does ot chage f we replace f ( by a fucto asymptotcally equvalet to t For ths reaso the recurrece may smply be gve as T( at ( / b) + Θ( f ( ) Notce also that there s o meto of tal terms It s part of the cotet of the master theorem that the tal values of the recurrece do ot effect t s asymptotc soluto
Examples Let T ( 8T ( / ) + The a 8, b, log case () Therefore T( Θ( log( ) log b b, ad hece f ( Θ( ), so we are Now let T ( 5T ( / 4) + Here a 5, b 4, log b 69 > Let ε log 4 (5) so that ε >, log ε 4 (5) log ( O 4 (5) T( Θ ad f ( ) Therefore we are case () ad ( ) Next cosder T ( 5T ( / 4) + Aga a 5, b 4, so log b 69 < Let ε log4 (5), log 4 (5) +ε so that ε >, ad f ( Ω( ), ad therefore case () seems to apply I ths case we must chec the regularty codto: 5 f ( / 4) cf ( for some < c < ad all suffcetly large Ths equalty say / 4), e 5( c Thus we may choose ay c satsfyg 5 /6 < ( 5/6) c Sce the regularty codto s satsfed we have T( Θ( ) c Note that eve though ths s called the Master Theorem the three cases do ot cover all possbltes There s a gap betwee cases () ad () whe s larger tha f (, but ot polyomally larger For example cosder the recurrece T ( T ( / ) + / lg( Observe that log (), ad chec ε that / lg( ) ω( ε ), whece / lg( O( ) for ay ε >, ad therefore we are ot case () But also / lg( o( log( ), so that / lg( Θ( log( ), ad ether are we case () Smlar commets hold for cases () ad () I addto, t s possble that the regularty codto case () fals to hold