Solving Divide-and-Conquer Recurrences

Transcription

1 Solvig Divide-ad-Coquer Recurreces Victor Adamchik A divide-ad-coquer algorithm cosists of three steps: dividig a problem ito smaller subproblems solvig (recursively) each subproblem the combiig solutios to subproblems to get solutio to origial problem We use recurreces to aalyze the ruig time of such algorithms. Suppose T is the umber of steps i the worst case eeded to solve the problem of size. Let us split a problem ito a 1 subproblems, each of which is of the iput size where b>1. b Observe, that the umber of subproblems a is ot ecessarily equal to b. The total umber of steps T is obtaied by all steps eeded to solve smaller subproblems T êb plus the umber eeded to combie solutios ito a fial oe. The followig equatio is called divide-ad-coquer recurrece relatio T = a T êb + fhl As a example, cosider the mergesort: -divide the iput i half -recursively sort the two halves -combie the two sorted subsequeces by mergig them. Let THL be worst-case rutime o a sequece of keys: If = 1, the THL = QH1L costat time If >1, the THL = 2 THê2L + QHL here Q() is time to do the merge. The

2 15-451: Algorithm Desig ad Aalysis 2 T = 2 T ê2 +QHL Other examples of divide ad coquer algorithms: quicksort, iteger multiplicatio, matrix multiplicatio, fast Fourier trsform, fidig cover hull ad more. There are several techiques of solvig such recurrece equatios: the iteratio method the tree method the master-theorem method guess-ad-verify ü Tree method We could visualize the recursio as a tree, where each ode represets a recursive call. The root is the iitial call. Leaves correspod to the exit coditio. We ca ofte solve the recurrece by lookig at the structure of the tree. To illustrate, we take this example THL=2 TK 2 O+2 TH1L=1 Here is a recursio tree that diagrams the recursive fuctio calls T() T(/2) T(/2) T(/4) T(/4) T(/4) T(/4) T(1) Usig a recursio tree we ca model the time of a recursive executio by writig the size of the problem i each ode. T(1)

3 3 Usig a recursio tree we ca model the time of a recursive executio by writig the size of the problem i each ode. The last level correspods to the iitial coditio of the recurrece. Sice the work at each leaf is costat, the total work at all leaves is equal to the umber of leaves, which is 2 h = 2 log 2 = To fid the total time (for the whole tree), we must add up all the terms -1+log 2 THL= = + 2 The sum is easily computed by meas of the geometric series This yeilds h k=0 x k = xh+1-1 x-1, x 1 k=0 1 2 k

4 15-451: Algorithm Desig ad Aalysis 4 THL= = Check with Mathematica RSolveA9T@D == 2 T@ê2D + 2, T@1D ä 1=, T@D, E 88T@D H 1+2L<< Example. Solve the recurrece THL=3 TK 4 O+ The work at all levels is Sice the height is log 4, the tree has 3 log 4 leaves. Hece, the total work is give by -1+log 4 THL= k=0 3 4 k + 3 log 4 TH1L By meas of the geometric series ad takig ito accout 3 log 4 = log 4 3 the above sum yields THL=4-4 log log 4 3 TH1L = OHL ü The Master Theorem The master theorem solves recurreces of the form THL = a T b + fhl for a wide variety of fuctio fhl ad a 1, b>1. I this sectio we will outlie the

5 5 mai idea. Here is the recursive tree for the above equatio It is easy to see that the tree has a log b leaves. Ideed, sice the height is log b, ad the tree brachig factor is a, the umber of leaves is log a h = a log b a log = a a b Summig up values at each level, gives Therefore, the solutio is THL= fhl + a f b + a2 f = 1 log a b b 2 = log b a log b a TH1L -1+log b THL= log b a TH1L+ a k f k=0 b k Now we eed to compare the asymptotic behavior of fhl with log b a. There are three possible cases. QI log b a M if fhl=oi log b a M THL= QI log b log k+1 M if fhl=qi log b a log k M, k 0 QH fhll if fhl=wi log b a M The followig examples demostrate the theorem. Case 1. THL=4 TI M+ 2 We have fhl= ad log b a = log 2 4 = 2, therefore fhl=oi 2 M. The the solutio is THL= QI 2 M by case 1. Case 2. THL=4 TI 2 M+2 I this case fhl= 2 ad fhl=qi 2 M. The THL= QI 2 log M by case 2. Case 3. THL=4 TI 2 M+3 I this case fhl= 3 ad fhl=wi log b a M=WI 2 M. The THL= QI 3 M by case 3.

6 15-451: Algorithm Desig ad Aalysis 6 ü Multiplicatio of large itegers Karatsuba Algorithm The brute force approach ("grammar school" method) We say that multiplicatio of two -digits itegers has time complexity at worst OI 2 M. We develop a algorithm that has better asymptotic complexity. The idea is based o divide-ad-coquer techique. Cosider the above itegers ad split each of them i two parts 123 = 12 * = 4 * ad the multiply them: 123*45 = (12*10 + 3)(4*10 + 5) = H L I geeral, the iteger which has digits ca be represeted as um = x * 10 m + y where m=floork 2 O x=ceiligk 2 O y=floork 2 O Example, = Cosider two -digits umbers

7 7 um 1 = x 1 * 10 p + x 0 um 2 = y 1 * 10 p + y 0 Their product is um 1 * um 2 = x 1 * y 1 * 10 2 p +Hx 1 * y 0 + x 0 * y 1 L*10 p + x 0 * y 0 Just lookig at this geeral formula you ca say that just istead of oe multiplicatio we have 4. Where is the advatage? ü The worst-case complexity umbers x 1, x 0 ad y 1, y 0 have twice less digits. Let THL deote the umber of digit multiplicatios eeded to multiply two -digits umbers. The recurrece (sice the algorithm does 4 multiplicatios o each step) THL=4 TI M+OHL, THcL = 1 2 Note, we igore multiplicatios by a base!!! Its solutio is give by The algorithm is still quadratic! THL = 4 log 2 = 2 ü The Karatsuba Algorithm 1962, Aatolii Karatsuba, Russia. um 1 * um 2 = x 1 * y 1 * 10 2 p +Hx 1 * y 0 + x 0 * y 1 L*10 p + x 0 * y 0 The goal is to decrease the umber of multiplicatios from 4 to 3. We ca do this by observig that It follows that Hx 1 + x 0 L*Hy 1 + y 0 L= x 1 * y 1 + x 0 * y 0 +Hx 1 * y 0 + x 0 * y 1 L um 1 * um 2 = x 1 * y 1 * 10 2 p +JHx 1 + x 0 L* Hy 1 + y 0 L- x 1 * y 1 - x 0 * y 0 N*10 p + x 0 * y 0 ad it is oly 3 multiplicatios (see it?). The total umber of multiplicatios is give by (we igore multiplicatios by a base) Its solutio is THL=3 TI M+OHL, T(c) = 1 2

8 15-451: Algorithm Desig ad Aalysis 8 ü Toom-Cook 3-Way Multiplicatio 1963, A. L. Toom, Russia. 1966, Cook, Harvard, Ph.D Thesis THL = 3 log 2 = log 2 3 = The key idea of the algorithm is to divide a large iteger ito 3 parts (rather tha 2) of size approximately ê 3 ad the multiply those parts. Here is the equatio of for the total umber of multiplicatios THL=9 T 3 + OHL, THcL = 1 ad the solutio TH L=9 log 3 = 2 Let us reduce the umber of multiplicatios by oe THL=8 T 3 + OHL THL=8 log 3 = log 3 8 = No advatage. This does ot improve the previous algorithm, that rus at OI M How may multiplicatio should we elimiate? Let us cosider that equatio i a geeral form, where parameter p>0 is arbitrary THL= p T 3 + OHL THL= p log 3 = log 3 p Therefore, the ew algoritm will be faster tha OI 1.58 M if we reduce the umber of multiplicatios to five This is a improvemet over Karatsuba. THL=5 log 3 = log 3 5 = Is it possible to reduce a umber of multiplicatios to 5? Yes, it follows from this system of equatios: where x 0 y 0 = Z 0 12Hx 1 y 0 + x 0 y 1 L=8 Z 1 - Z 2-8 Z 3 + Z 4 24Hx 2 y 0 + x 1 y 1 + x 0 y 2 L=-30 Z Z 1 - Z Z 3 - Z 4 12Hx 2 y 1 + x 1 y 2 L=-2 Z 1 + Z Z 3 - Z 4 24 x 2 y 2 = 6 Z 0-4 Z 1 + Z 2-4 Z 3 + Z 4

9 9 ü Further Geeralizatio Z 0 = x 0 y 0 Z 1 = Hx 0 + x 1 + x 2 LHy 0 + y 1 + y 2 L Z 2 = Hx x x 2 LHy y y 2 L Z 3 = Hx 0 - x 1 + x 2 LHy 0 - y 1 + y 2 L Z 4 = Hx 0-2 x x 2 LHy 0-2 y y 2 L It is possible to develop a faster algorithm by icreasig the umber of splits. Let us cosider a 4-way splittig. How may multiplicatios should we have o each step so this algorithm will outperform the 3-way splittig? We fid parameter p from which yields THL= p TK 4 O+OHL THL= p log 4 = log 4 p log 4 p log 3 5 p = 7 The followig table demostrates a relatioship betwee splits ad the umber of multiplicatios: Ituitively we see that the k-way split requires 2 k - 1 multiplicatios. This meas that istead of k 2 multiplicatios we do oly 2 k- 1. The recurrece equatio for the total umber of multiplicatio is give by THL=H2 k- 1L T k + OHL ad its solutio is THL=H2 k- 1L log k = log k H2 k-1l Here is the sequece of the k-way splits whe k rus from 2 to 10:

10 15-451: Algorithm Desig ad Aalysis , 1.46, 1.40, 1.36, 1.33, 1.31, 1.30, 1.28, We ca prove that asymptotically multiplicatio of two -digits umbers requires OI 1+e M multiplicatios, where eø0. Note, we will NEVER get a liear performace (prove this!) Is it always possible to fid such 2 k- 1 multiplicatios? Cosider two polyomials of k- 1 degree poly 1 = a k-1 x k-1 + a k-2 * x k a 1 * x+a 0 poly 2 = b k-1 x k-1 + b k-2 * x k b 1 * x+b 0 whe we multiply them we get a polyomial of 2 k- 2 degree poly 1 * poly 2 = a k-1 b k-1 * x 2 k Ha 1 b 0 + b 1 a 0 L* x + a 0 b 0 The above polyomial has exactly 2 k- 1 coefficiets, therefore it's uiquely defied by 2 k- 1 values.