Solving Linear Recurrence Relations Niloufar Shafiei
Review A recursive definition of a sequence specifies Initial conditions Recurrence relation Example: a 0 =0 and a 1 =3 Initial conditions a n = 2a n-1 - a n-2 a n = 3n Recurrence relation Solution 1
Linear recurrences Linear recurrence: Each term of a sequence is a linear function of earlier terms in the sequence. For example: a 0 = 1 a 1 = 6 a 2 = 10 a n = a n-1 + 2a n-2 + 3a n-3 a 3 = a 0 + 2a 1 + 3a 2 = 1 + 2(6) + 3(10) = 43 2
Linear recurrences Linear recurrences 1. Linear homogeneous recurrences 2. Linear non-homogeneous recurrences 3
Linear homogeneous recurrences A linear homogenous recurrence relation of degree k with constant coefficients is a recurrence relation of the form a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k, where c 1, c 2,, c k are real numbers, and c k 0. a n is expressed in terms of the previous k terms of the sequence, so its degree is k. This recurrence includes k initial conditions. a 0 = C 0 a 1 = C 1 a k = C k 4
Example Determine if the following recurrence relations are linear homogeneous recurrence relations with constant coefficients. P n = (1.11)P n-1 a linear homogeneous recurrence relation of degree one a n = a n-1 + a 2 n-2 not linear f n = f n-1 + f n-2 a linear homogeneous recurrence relation of degree two H n = 2H n-1 +1 not homogeneous a n = a n-6 a linear homogeneous recurrence relation of degree six B n = nb n-1 does not have constant coefficient 5
Solving linear homogeneous recurrences Proposition 1: Let a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k be a linear homogeneous recurrence. Assume the sequence a n satisfies the recurrence. Assume the sequence a n also satisfies the recurrence. So, b n = a n + a n and d n = a n are also sequences that satisfy the recurrence. ( is any constant) Proof: b n = a n + a n = (c 1 a n-1 + c 2 a n-2 + + c k a n-k ) + (c 1 a n-1 + c 2 a n-2 + + c k a n-k ) = c 1 (a n-1 + a n-1 ) + c 2 (a n-2 + a n-2 ) + + c k (a n-k + a n-k ) = c 1 b n-1 + c 2 b n-2 + + c k b n-k So, b n is a solution of the recurrence. 6
Solving linear homogeneous recurrences Proposition 1: Let a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k be a linear homogeneous recurrence. Assume the sequence a n satisfies the recurrence. Assume the sequence a n also satisfies the recurrence. So, b n = a n + a n and d n = a n are also sequences that satisfy the recurrence. ( is any constant) Proof: d n = a n = (c 1 a n-1 + c 2 a n-2 + + c k a n-k ) = c 1 ( a n-1 ) + c 2 ( a n-2 ) + + c k ( a n-k ) = c 1 d n-1 + c 2 d n-2 + + c k d n-k So, d n is a solution of the recurrence. 7
Solving linear homogeneous recurrences It follows from the previous proposition, if we find some solutions to a linear homogeneous recurrence, then any linear combination of them will also be a solution to the linear homogeneous recurrence. 8
Solving linear homogeneous recurrences Geometric sequences come up a lot when solving linear homogeneous recurrences. So, try to find any solution of the form a n = r n that satisfies the recurrence relation. 9
Solving linear homogeneous recurrences Recurrence relation a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k Try to find a solution of form r n r n = c 1 r n-1 + c 2 r n-2 + + c k r n-k r n - c 1 r n-1 - c 2 r n-2 - - c k r n-k = 0 r k - c 1 r k-1 - c 2 r k-2 - - c k = 0 (dividing both sides by r n-k ) This equation is called the characteristic equation. 10
Example Example: The Fibonacci recurrence is F n = F n-1 + F n-2 Its characteristic equation is r 2 - r - 1 = 0 11
Solving linear homogeneous recurrences Proposition 2: r is a solution of r k - c 1 r k-1 - c 2 r k-2 - - c k = 0 if and only if r n is a solution of a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k. Example: consider the characteristic equation r 2-4r + 4 = 0. r 2-4r + 4 = (r - 2) 2 = 0 So, r=2. So, 2 n satisfies the recurrence F n = 4F n-1-4f n-2. 2 n = 4. 2 n-1-4. 2 n-2 2 n-2 ( 4-8 + 4) = 0 12
Solving linear homogeneous recurrences Theorem 1: Consider the characteristic equation r k - c 1 r k-1 - c 2 r k-2 - - c k = 0 and the recurrence a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k. Assume r 1, r 2, and r m all satisfy the equation. Let 1, 2,, m be any constants. So, a n = 1 r n 1 + 2 r n 2 + + m r n m satisfies the recurrence. Proof: By Proposition 2, i r i n satisfies the recurrence. So, by Proposition 1, i i r i n satisfies the recurrence. Applying Proposition 1 again, the sequence a n = 1 r 1 n + 2 r 2 n + + m r m n satisfies the recurrence. 13
Example What is the solution of the recurrence relation a n = a n-1 + 2a n-2 with a 0 =2 and a 1 =7? Solution: Since it is linear homogeneous recurrence, first find its characteristic equation r 2 - r - 2 = 0 (r+1)(r-2) = 0 r 1 = 2 and r 2 = -1 So, by theorem a n = 1 2 n + 2 (-1) n is a solution. Now we should find 1 and 2 using initial conditions. a 0 = 1 + 2 = 2 a 1 = 1 2 + 2 (-1) = 7 So, 1 = 3 and 2 = -1. a n = 3. 2 n - (-1) n is a solution. 14
Example What is the solution of the recurrence relation f n = f n-1 + f n-2 with f 0 =0 and f 1 =1? Solution: Since it is linear homogeneous recurrence, first find its characteristic equation r 2 - r - 1 = 0 r 1 = (1+ )/2 and r 2 = (1- )/2 So, by theorem f n = 1 ((1+ )/2) n + 2 ((1- )/2) n is a solution. Now we should find 1 and 2 using initial conditions. f 0 = 1 + 2 = 0 f 1 = 1 (1+ )/2 + 2 (1- )/2 = 1 So, 1 = 1/ and 2 = -1/. a n = 1/. ((1+ )/2) n - 1/ ((1- )/2) n is a solution. 15
Example What is the solution of the recurrence relation a n = -a n-1 + 4a n-2 + 4a n-3 with a 0 =8, a 1 =6 and a 2 =26? Solution: Since it is linear homogeneous recurrence, first find its characteristic equation r 3 + r 2-4r - 4 = 0 (r+1)(r+2)(r-2) = 0 r 1 = -1, r 2 = -2 and r 3 = 2 So, by theorem a n = 1 (-1) n + 2 (-2) n + 3 2 n is a solution. Now we should find 1, 2 and 3 using initial conditions. a 0 = 1 + 2 + 3 = 8 a 1 = - 1-2 2 + 2 3 = 6 a 2 = 1 + 4 2 + 4 3 = 26 So, 1 = 2, 2 = 1 and 3 = 5. a n = 2. (-1) n + (-2) n + 5. 2 n is a solution. 16
Solving linear homogeneous recurrences If the characteristic equation has k distinct solutions r 1, r 2,, r k, it can be written as (r - r 1 )(r - r 2 ) (r - r k ) = 0. If, after factoring, the equation has m+1 factors of (r - r 1 ), for example, r 1 is called a solution of the characteristic equation with multiplicity m+1. When this happens, not only r 1 n is a solution, but also nr 1n, n 2 r 1n, and n m r 1 n are solutions of the recurrence. 17
Solving linear homogeneous recurrences Proposition 3: Assume r 0 is a solution of the characteristic equation with multiplicity at least m+1. So, n m r n 0 is a solution to the recurrence. 18
Solving linear homogeneous recurrences When a characteristic equation has fewer than k distinct solutions: We obtain sequences of the form described in Proposition 3. By Proposition 1, we know any combination of these solutions is also a solution to the recurrence. We can find those that satisfies the initial conditions. 19
Solving linear homogeneous recurrences Theorem 2: Consider the characteristic equation r k - c 1 r k-1 - c 2 r k-2 - - c k = 0 and the recurrence a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k. Assume the characteristic equation has t k distinct solutions. Let i (1 i t) r i with multiplicity m i be a solution of the equation. Let i,j (1 i t and 0 j m i -1) ij be a constant. So, a n = ( 10 + 11 n+ + 1,m1-1 nm 1-1 ) r n 1 + ( 20 + 21 n+ + 2,m2-1 nm 2-1 ) r 2 n + + ( t0 + t1 n+ + t,mt -1 nm t -1 ) r t n satisfies the recurrence. 20
Example What is the solution of the recurrence relation a n = 6a n-1-9a n-2 with a 0 =1 and a 1 =6? Solution: First find its characteristic equation r 2-6r + 9 = 0 (r - 3) 2 = 0 r 1 = 3 (Its multiplicity is 2.) So, by theorem a n = ( 10 + 11 n)(3) n is a solution. Now we should find constants using initial conditions. a 0 = 10 = 1 a 1 = 3 10 + 3 11 = 6 So, 11 = 1 and 10 = 1. a n = 3 n + n3 n is a solution. 21
Example What is the solution of the recurrence relation a n = -3a n-1-3a n-2 - a n-3 with a 0 =1, a 1 =-2 and a 2 =-1? Solution: Find its characteristic equation r 3 + 3r 2 + 3r + 1 = 0 (r + 1) 3 = 0 r 1 = -1 (Its multiplicity is 3.) So, by theorem a n = ( 10 + 11 n + 12 n 2 )(-1) n is a solution. Now we should find constants using initial conditions. a 0 = 10 = 1 a 1 = - 10-11 - 12 = -2 a 2 = 10 + 2 11 + 4 12 = -1 So, 10 = 1, 11 = 3 and 12 = -2. a n = (1 + 3n - 2n 2 ) (-1) n is a solution. 22
Example What is the solution of the recurrence relation a n = 8a n-2-16a n-4, for n 4, with a 0 =1, a 1 =4, a 2 =28 and a 3 =32? Solution: Find its characteristic equation r 4-8r 2 + 16 = 0 (r 2-4) 2 = (r-2) 2 (r+2) 2 = 0 r 1 = 2 r 2 = -2 (Their multiplicities are 2.) So, by theorem a n = ( 10 + 11 n)(2) n + ( 20 + 21 n)(-2) n is a solution. Now we should find constants using initial conditions. a 0 = 10 + 20 = 1 a 1 = 2 10 + 2 11-2 20-2 21 = 4 a 2 = 4 10 + 8 11 + 4 20 + 8 21 = 28 a 3 = 8 10 + 24 11-8 20-24 21 = 32 So, 10 = 1, 11 = 2, 20 = 0 and 21 = 1. a n = (1 + 2n) 2 n + n (-2) n is a solution. 23
Linear non-homogeneous recurrences A linear non-homogenous recurrence relation with constant coefficients is a recurrence relation of the form a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k + f(n), where c 1, c 2,, c k are real numbers, and f(n) is a function depending only on n. The recurrence relation a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k, is called the associated homogeneous recurrence relation. This recurrence includes k initial conditions. a 0 = C 0 a 1 = C 1 a k = C k 24
Example The following recurrence relations are linear nonhomogeneous recurrence relations. a n = a n-1 + 2 n a n = a n-1 + a n-2 + n 2 + n + 1 a n = a n-1 + a n-4 + n! a n = a n-6 + n2 n 25
Linear non-homogeneous recurrences Proposition 4: Let a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k + f(n) be a linear non-homogeneous recurrence. Assume the sequence b n satisfies the recurrence. Another sequence a n satisfies the nonhomogeneous recurrence if and only if h n = a n - b n is also a sequence that satisfies the associated homogeneous recurrence. 26
Linear non-homogeneous recurrences Proof: Part1: if h n satisfies the associated homogeneous recurrence then a n is satisfies the non-homogeneous recurrence. b n = c 1 b n-1 + c 2 b n-2 + + c k b n-k + f(n) h n = c 1 h n-1 + c 2 h n-2 + + c k h n-k b n + h n = c 1 (b n-1 + h n-1 ) + c 2 (b n-2 + h n-2 ) + + c k (b n-k + h n-k ) + f(n) Since a n = b n + h n, a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k + f(n). So, a n is a solution of the non-homogeneous recurrence. 27
Linear non-homogeneous recurrences Proof: Part2: if a n satisfies the non-homogeneous recurrence then h n is satisfies the associated homogeneous recurrence. b n = c 1 b n-1 + c 2 b n-2 + + c k b n-k + f(n) a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k + f(n) a n - b n = c 1 (a n-1 - b n-1 ) + c 2 (a n-2 - b n-2 ) + + c k (a n-k - b n-k ) Since h n = a n - b n, h n = c 1 h n-1 + c 2 h n-2 + + c k h n-k So, h n is a solution of the associated homogeneous recurrence. 28
Linear non-homogeneous recurrences Proposition 4: Let a n = c 1 a n-1 + c 2 a n-2 + + c k a n-k + f(n) be a linear nonhomogeneous recurrence. Assume the sequence b n satisfies the recurrence. Another sequence a n satisfies the non-homogeneous recurrence if and only if h n = a n - b n is also a sequence that satisfies the associated homogeneous recurrence. We already know how to find h n. For many common f(n), a solution b n to the non-homogeneous recurrence is similar to f(n). Then you should find solution a n = b n + h n to the nonhomogeneous recurrence that satisfies both recurrence and initial conditions. 29
Example What is the solution of the recurrence relation a n = a n-1 + a n-2 + 3n + 1 for n 2, with a 0 =2 and a 1 =3? Solution: Since it is linear non-homogeneous recurrence, b n is similar to f(n) Guess: b n = cn + d b n = b n-1 + b n-2 + 3n + 1 cn + d = c(n-1) + d + c(n-2) + d + 3n + 1 cn + d = cn - c + d + cn -2c + d +3n + 1 0 = (3+c)n + (d-3c+1) c = -3 d=-10 So, b n = - 3n - 10. (b n only satisfies the recurrence, it does not satisfy the initial conditions.) 30
Example What is the solution of the recurrence relation a n = a n-1 + a n-2 + 3n + 1 for n 2, with a 0 =2 and a 1 =3? Solution: We are looking for a n that satisfies both recurrence and initial conditions. a n = b n + h n where h n is a solution for the associated homogeneous recurrence: h n = h n-1 + h n-2 By previous example, we know h n = 1 ((1+ )/2) n + 2 ((1- )/2) n. a n = b n + h n = - 3n - 10 + 1 ((1+ )/2) n + 2 ((1- )/2) n Now we should find constants using initial conditions. a 0 = -10 + 1 + 2 = 2 a 1 = -13 + 1 (1+ )/2 + 2 (1- )/2 = 3 1 = 6 + 2 2 = 6-2 So, a n = -3n - 10 + (6 + 2 )((1+ )/2) n + (6-2 )((1- )/2) n. 31
Example What is the solution of the recurrence relation a n = 2a n-1 - a n-2 + 2 n for n 2, with a 0 =1 and a 1 =2? Solution: Since it is linear non-homogeneous recurrence, b n is similar to f(n) Guess: b n = c2 n + d b n = 2b n-1 - b n-2 + 2 n c2 n + d = 2(c2 n-1 + d) - (c2 n-2 + d) + 2 n c2 n + d = c2 n + 2d - c2 n-2 - d + 2 n 0 = (-4c + 4c - c + 4)2 n-2 + (-d + 2d -d) c = 4 d=0 So, b n = 4. 2 n. (b n only satisfies the recurrence, it does not satisfy the initial conditions.) 32
Example What is the solution of the recurrence relation a n = 2a n-1 - a n-2 + 2 n for n 2, with a 0 =1 and a 1 =2? Solution: We are looking for a n that satisfies both recurrence and initial conditions. a n = b n + h n where h n is a solution for the associated homogeneous recurrence: h n = 2h n-1 - h n-2. Find its characteristic equation r 2-2r + 1 = 0 (r - 1) 2 = 0 r 1 = 1 (Its multiplicity is 2.) So, by theorem h n = ( 1 + 2 n)(1) n = 1 + 2 n is a solution. 33
Example What is the solution of the recurrence relation a n = 2a n-1 - a n-2 + 2 n for n 2, with a 0 =1 and a 1 =2? Solution: a n = b n + h n a n = 4. 2 n + 1 + 2 n is a solution. Now we should find constants using initial conditions. a 0 = 4 + 1 = 1 a 1 = 8-1 + 2 = 2 1 = -3 2 = -3 So, a n = 4. 2 n - 3n - 3. 34
Recommended exercises 1,3,15,17,19,21,23,25,31,35 Eric Ruppert s Notes about Solving Recurrences (http://www.cse.yorku.ca/course_archive/2007-08/f/1019/a/recurrence.pdf) 35