Solving recurrences. CS 4407, Algorithms University College Cork, Gregory M. Provan

Transcription

1 Solving recurrences The analysis of divide and conquer algorithms require us to solve a recurrence. Recurrences are a major tool for analysis of algorithms

2 MergeSort A L G O R I T H M S A L G O R I T H M S divide cn T(n/2) T(n/2)

3 MergeSort A L G O R I T H M S A L G O R I T H M S Divide #1 A L G O R I T H M S Divide #2 cn T(n/2) T(n/2) T(n/4) T(n/4) T(n/4) T(n/4)

4 MergeSort Solve T(n) = T(n/2) + T(n/2) + cn cn (n/2) +c (n/2) +c T(n/4) T(n/4) T(n/4) T(n/4) Recurrence T ( n) = 2T n 2 + cn c n = 1 n > 1

5 Integer Multiplication Let X = A B and Y = C D where A,B,C and D are n/2 bit integers Simple Method: XY = (2 n/2 A+B)(2 n/2 C+D) Running Time Recurrence T(n) < 4T(n/2) + 100n How do we solve it?

6 Substitution method The most general method: 1. Guess the form of the solution. 2. Verify by induction. 3. Solve for constants. Example: T(n) = 4T(n/2) + 100n [Assume that T(1) = Θ(1).] Guess O(n 3 ). (Prove O and Ω separately.) Assume that T(k) ck 3 for k < n. Prove T(n) cn 3 by induction.

7 Example of substitution T ( n) = 4T ( n / 2) + 100n 4c( n / 2) n = ( c / 2) n n = cn3 (( c / 2) n3 100n) desired residual cn 3 desired whenever (c/2)n 3 100n 0, for example, if c 200 and n 1. residual

8 Recursion-tree method A recursion tree models the costs (time) of a recursive execution of an algorithm. The recursion tree method is good for generating guesses for the substitution method. The recursion-tree method can be unreliable, just like any method that uses ellipses ( ). The recursion-tree method promotes intuition, however.

9 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 :

10 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : T(n)

11 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : n 2 T(n/4) T(n/2)

12 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : n 2 (n/4) 2 (n/2) 2 T(n/16) T(n/8) T(n/8) T(n/4)

13 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : n 2 (n/4) 2 (n/2) 2 (n/16) 2 (n/8) 2 (n/8) 2 (n/4) 2 Θ(1)

14 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : n 2 n 2 (n/4) 2 (n/2) 2 (n/16) 2 (n/8) 2 (n/8) 2 (n/4) 2 Θ(1)

15 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : n 2 n2 2 (n/2) 2 5 n 2 (n/4) 16 (n/16) 2 (n/8) 2 (n/8) 2 (n/4) 2 Θ(1)

16 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : n 2 n2 2 5 n 2 (n/4) (n/2) 2 16 (n/16) 2 (n/8) 2 (n/8) 2 (n/4) 2 25 n Θ(1)

17 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : n 2 n2 (n/4) 2 5 (n/2) 2 n 2 16 (n/16) 2 (n/8) 2 (n/8) 2 (n/4) 2 25 n Θ(1) Total = n 2 ( ( ) 2 ( ) 3 ) L = Θ(n 2 ) geometric series

18 Appendix: geometric series x x x = L for x < x x x x x n n = L for x 1 1 x

19 The master method The master method applies to recurrences of the form T(n) = a T(n/b) + f (n), where a 1, b > 1, and f is asymptotically positive.

20 h = log b n Idea of master theorem Recursion tree: f (n) a f (n/b) f (n/b) a f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) f (n/b) f (n) a f (n/b) a 2 f (n/b 2 ) Τ(1) #leaves = a h = a log bn = n log ba n log ba Τ(1)

21 Three common cases Compare f (n) with n log ba : 1. f(n) = O(n log ba ε ) for some constant ε > 0. f (n) grows polynomially slower than n log ba (by an n ε factor). Solution: T(n) = Θ(n log ba ). # leaves in recursion tree

22 h = log b n Idea of master theorem Recursion tree: f (n) a f (n/b) f (n/b) a f (n/b) f (n) a f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) a 2 f (n/b 2 ) Τ(1) CASE 1: The weight increases geometrically from the root to the n log ba Τ(1) leaves. The leaves hold a constant fraction of the total weight. Θ(n log ba )

23 Three common cases Compare f (n) with n log ba : 2. f (n) = Θ(n log ba lg k n) for some constant k 0. f (n) and n log ba grow at similar rates. Solution: T(n) = Θ(n log ba lg k+1 n).

24 h = log b n Idea of master theorem Recursion tree: f (n) a f (n/b) f (n/b) a f (n/b) f (n) a f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) a 2 f (n/b 2 ) Τ(1) CASE 2: (k = 0) The weight is approximately the same on each of the log b n levels. n log ba Τ(1) Θ(n log ba lg n)

25 Three common cases (cont.) Compare f (n) with n log ba : 3. f (n) = Ω(n log ba + ε ) for some constant ε > 0. f (n) grows polynomially faster than n log ba (by an n ε factor), and f (n) satisfies the regularity condition that a f (n/b) c f (n) for some constant c < 1. Solution: T(n) = Θ( f (n)).

26 h = log b n Idea of master theorem Recursion tree: f (n) a f (n/b) f (n/b) a f (n/b) f (n) a f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) a 2 f (n/b 2 ) Τ(1) CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight. n log ba Τ(1) Θ( f (n))

27 Examples Ex. T(n) = 4T(n/2) + n a = 4, b = 2 n log ba = n 2 ; f (n) = n. CASE 1: f(n) = O(n 2 ε ) for ε = 1. T(n) = Θ(n 2 ). Ex. T(n) = 4T(n/2) + n 2 a = 4, b = 2 n log ba = n 2 ; f (n) = n 2. CASE 2: f(n) = Θ(n 2 lg 0 n), that is, k = 0. T(n) = Θ(n 2 lg n).

28 Examples Ex. T(n) = 4T(n/2) + n 3 a = 4, b = 2 n log ba = n 2 ; f (n) = n 3. CASE 3: f(n) = Ω(n 2 + ε ) for ε = 1 and 4(cn/2) 3 cn 3 (reg. cond.) for c = 1/2. T(n) = Θ(n 3 ). Ex. T(n) = 4T(n/2) + n 2 /lgn a = 4, b = 2 n log ba = n 2 ; f (n) = n 2 /lgn. Master method does not apply. In particular, for every constant ε > 0, we have n ε = ω(lgn).