CS 6, Lecture 3 Divide ad Coquer, Solvig Recurreces, Iteger Multiplicatio Scribe: Juliaa Cook (05, V Williams Date: April 6, 06 Itroductio Today we will cotiue to talk about divide ad coquer, ad go ito detail o how to solve recurreces Recall that divide ad coquer algorithms divide up a problem ito a umber of subproblems that are the smaller istaces of the same problem, solve those problems recursively, ad combie the solutios to the subproblems ito a solutio for the origial problem Whe a subproblem size is small eough, the subproblem is simply solved i a straightforward maer I past lectures we have see two examples of divide ad coquer algorithms, merge sort ad the media-of-medias approach to the selectio problem This aturally gives rise to rutime recurreces, which express the algorithm o rutimes of iput size i terms of the rutimes of smaller iputs Recurreces Stated more techically, a divide ad coquer algorithm takes a iput of size ad does some operatios all ruig i O(f( time for some f ad rus itself recursively o k istaces of size,,, k, where i < for all i To talk about what the rutime of such a algorithm is, we ca write a rutime recurrece Recurreces are fuctios defied i terms of themselves with smaller argumets, as well as oe or more base cases We ca defie a recurrece more formally as follows: Let T ( be the rutime o istaces of size If we have k recursive calls o a give step (of sizes i ad each step takes time O(f(, the we ca write the rutime as T ( c f( k T ( i for some costat c, where our base case is T (c O( Now let s try fidig recurreces for some of the divide ad coquer algorithms we have see MergeSort Cosider the basic steps for algorithm MergeSort(A, where A Split A ito A, A of size Ru MergeSort(A ad MergeSort(A 3 Merge(A, A Steps ad 3 each take time O( Thus we kow that f( give the recurrece formula above As we are splittig the work up ito two evely sized pieces we kow that k ad each i Therefore, our fial equatio is ( T ( c T We ote that we ca replace c by by, i a sese, chagig uits We ca aalyze T ( T (/, rememberig that each cost operatio actually costs c I the ed we ca just set T ( ct ( O(T ( Thus from ow o we will remove costats as much as possible We will cosider two more recurreces The first oe correspods to a divide ad coquer algorithm for fidig the miimum umber i a (usorted array A Assume that A has size which is a power of We ca make this assumptio without loss of geerality: Suppose that k < k r for some r < k ; i
the we ca icrease the size of A by r by addig r ew etries at the ed that are bigger tha all itegers i the array (say The, the size of A ca at most double, ad ow it is a power of, k The algorithm FidMi(A first checks if the array size is If so, it just returs the cotets of A[0] Otherwise it recursively fids a F idmi(a[0 : / ] ad b F idmi(a[/ : ] ad returs mi{a, b} The rutime recurrece is T ( T (/ for all > ad T ( (Agai, the s i this recurrece correspod to some machie depedet costat The fial recurrece we cosider is T ( T (7/0 5 T (/5 for all > 5 ad T ( for 5 This recurrece will come up whe we itroduce a algorithm i the ext lecture 3 Methods for Solvig Recurreces Oce we have a recurrece for a fuctio, we wat to determie what the asymptotic bouds are (either Θ or O There are three mai methods for solvig recurreces Recursio Tree Substitutio Method - guess rutime ad check usig iductio 3 Master Theorem 3 Recursio Tree Recursio trees are a visual way to devise a good guess for the solutio to a recurrece, which ca the be formally proved usig the substitutio method I a recursio tree, each ode represets the cost of a sigle subproblem somewhere i the set of recursive fuctio ivocatios Summig the costs withi each level of the tree gives us the cost of each level of recursio To determie the total cost we the sum the costs of all levels of the recursio Below is a example of a recursio tree for the MergeSort algorithm At the root ode we have, represetig the size of the origial istace Sice we have two recursive calls at each step i which we divide the curret size ito two equally sized pieces, each ode as two childre that are each half the size of their paret Level 0: Level : Level : k k k Total over levels i {0,, k log }: O ( log i0 i i Recursio tree for Merge Sort LaTEX credit Mauel Kirsch k Level k: ( log O i0 O( log
If we aalyzed the FidMi recurrece T ( T (/ with the recursio tree method we would get that T ( O(: Level 0: Level : Level : 4 Level k: k Total over levels i {0,, k log }: log i0 i log ( While recursio trees work quite well for uderstadig simpler recurreces, more complex recurreces start to get quite tricky Below is a partial recurrece tree for the recurrece T ( T (7/05T (/5 Note that at each stage the ode is divided ito ueve groups, which will result i some leaf odes beig at differet levels tha others To get the sums i each level oe eeds to carefully calculate the differet ode sizes, ad it ca get quite messy Thus for such recurreces a differet solutio method may be preferred O( 5 7 0 5 5 6 5 7 50 57 0 7 50 49 00 7 Recursio tree for selectio problem from ext lecture We ca evertheless use the recursio tree method to geerate a guess for what T ( T (7/0 5 T (/5 solves to Suppose that we drop the 5 ad terms ad aalyze T ( T (7/0 T (/5 istead We might ot get the right aswer but we ca guess that it will be somethig close to correct With the recursio tree method we see that: I level 0, the algorithm performs work I level, the sum of rutimes is 7/0 /5 9/0 3
I level, the sum uder the ode with 7/0 is 9/0(7/0 ad uder the ode with /5 it s 9/0(/5 Altogether it s (9/0 Iductively we ca show that i level k the algorithm performs (9/0 k work Thus the total amout of work is < i0 (9/0i 0 O( Thus we ca guess that the aswer is O( for T ( T (7/0 5 T (/5 With the ext method we will prove it by iductio 3 Substitutio Method As we saw above, recurreces trees ca get quite messy whe attemptig to solve complex recurreces With the substitutio method we ca guess what the rutime is, plug it i to the recurrece ad see if it works out Give a recurrece T ( cf( k T ( i, we ca guess that the solutio to the recurrece is i { d g(0 if T ( 0 d g( if > 0 for some costats d > 0 ad 0 ad fuctio g( We are essetially guessig that T ( O(g( For our base case we must show that you ca pick some d such that T ( 0 d g( 0, assumig T ( for istace (or whatever the base case of the recurrece is Next we assume that our guess is correct for everythig smaller tha, meaig T ( d g( for all < For our iductive hypothesis we prove the guess for We must pick some d such that f( k d g( i d g(, wheever 0 i Typically the way this works is you first try to prove the iductive step startig from the iductive hypothesis, ad the from this obtai a coditio that d eeds to obey Usig this coditio you try to figure out the base case, ie what 0 should be 3 Solvig T ( T (7/0 5 T (/5 usig Substitutio Method Recall the recurrece T ( c T ( 5 T ( 7 0 5 for > 5 ad T ( for 5 From our recursio tree we make the guess: { d0 if T ( 0 d if > 0 For our iductive hypothesis, we assume that our guess is correct for aythig smaller tha, ad we prove our guess for That is, pick some d such that for all 0, k dg( i d g( i ( d 5 d [ 9d ( 7 0 5 d 0 6d ] d ] d [ 9d 0 6d 4
Replacig the i the deomiator with 0 sice we oly care about 0, we get that we are lookig for d, 0 that satisfy the base case T ( 0 d 0 ad such that ( 9d 0 6d d 0 ( d 0 6 0 The above is true wheever 0 6 0 > 0 ad d 0 60 0 If we pick 0 6 for istace we get that we ca select ay d 60 Thus we cosider the base case ad figure out what T (6 is by brute-forcig the recurrece up to 6 The we select d max{60, T (6/6} The iductio is completed 3 Issues whe usig the Substitutio Method Now we will try out a example where our guess is icorrect Cosider MergeSort, which has the recurrece T ( T ( We will guess that the algorithm is liear { d0 if T ( 0 d if > 0 We try the iductive step We try to pick some d such that for all 0, k dg( i d g( i d d ( d d d d < 0, However, the above ca ever be true, ad there is o choice of d that works! Thus our guess was icorrect This time the guess was icorrect sice MergeSort takes superliear time Sometimes however the guess ca be asymptotically correct but the iductio might ot work out Cosider for istace T ( T (/, the rutime for FidMi We kow that the rutime is O( so let s try to prove it with the substitutio method Let s guess that T ( c for all 0 First we do the iductio step: We assume that T (/ c/ ad cosider T ( We wat that c/ c However this is impossible sice the LHS is c ad the RHS is c This does t mea that T ( is ot O(, but that we chose the wrog liear fuctio We could guess istead that T ( c Now for the iductio we get (c/ c which is true for all c We ca the eve leave the base case T ( 5
Algorithm : Mult(x, y Split x ad y ito x 0 a b ad y 0 c d z Mult(a, c z Mult(a, d z 3 Mult(b, c z 4 Mult(b, d retur z 0 0 (z z 3 z 4 Algorithm : Karatsuba(x, y Split x 0 a b ad y 0 c d z Karatsuba(a, c z Karatsuba(b, d z 3 Karatsuba(a b, c d z 4 z 3 z z retur z 0 z 4 0 z 33 Master Method The Master Method uses the Master Theorem to solve recurreces of a special form Theorem 3 (Master Theorem A time recurrece T ( f( a T (/b, where a, b > are costats solves to: Case : T ( Θ ( log a b if f( O ( log a ɛ b for some costat ɛ > 0 domiated by the leaves of the recursio tree (ituitively: the cost is Case : T ( Θ (f( log if f( Θ ( log b a, or Case 3: T ( Θ(f( if f( Ω ( log b aɛ for some costat ɛ > 0 ad for c <, af(/b cf( (ituitively: cost is domiated by root Iteger Multiplicatio Recall the iteger multiplicatio problem, where we are give two -digit itegers x ad y ad output the product of the two digits I lecture we saw two approaches to solvig this problem Let Add( be the rutime to add two( digit itegers The, the rutime of Mult ca be writte as the recurrece T ( c Add( 4T, ad Karatsuba s rutime ca be writte as the recurrece T ( c Add( 3T ( Addig two digit itegers is a O( operatio, sice for each positio we add at most three digits: the ith digit from each umber ad possibly a carry from the additios due to the i st digits We ca the rewrite our Mult recurrece as ( T ( c 4T, ad Karatsuba as T ( c 3T ( For Mult, f( c, a 4 ad b, so log b a We ca see that f( c O( ɛ for ay ɛ, which meas case of the Master Theorem applies ad T ( Θ( For Karatsuba, f( c, a 3 ad b, so log b a log 3 59 We see that f( c O( 59 ɛ (for istace for ɛ 05, which meas case of the Master Theorem applies ad T ( O( 59 6
Therefore, Karatsuba s algorithm is T ( Θ( 59 wier! For iteger multiplicatio, Karatsuba is the clear 7