Polynomials and the Fast Fourier Transform (FFT) Battle Plan

Transcription

1 Polynomials and the Fast Fourier Transform (FFT) Algorithm Design and Analysis (Wee 7) 1 Polynomials Battle Plan Algorithms to add, multiply and evaluate polynomials Coefficient and point-value representation Fourier Transform Discrete Fourier Transform (DFT) and inverse DFT to translate between polynomial representations A Short Digression on Complex Roots of Unity Fast Fourier Transform (FFT) is a divide-and-conquer algorithm based on properties of complex roots of unity 2

2 Polynomials A polynomial in the variable x is a representation of a function A x = a n 1 x n a 2 x 2 + a 1 x + a 0 as a formal sum A x = n 1 j=0 a j x j. We call the values a 0, a 1,, a n 1 the coefficients of the polynomial A x is said to have degree if its highest nonzero coefficient is a. Any integer strictly greater than the degree of a polynomial is a degree-bound of that polynomial 3 A x = x 3 2x 1 Examples A(x) has degree 3 A(x) has degree-bounds 4, 5, 6, or all values > degree A(x) has coefficients ( 1, 2, 0, 1) B x = x 3 + x B(x) has degree 3 B(x) has degree bounds 4, 5, 6, or all values > degree B(x) has coefficients (1, 0, 1, 1) 4

3 Coefficient Representation A coefficient representation of a polynomial A x = n 1 j=0 a j x j of degree-bound n is a vector of coefficients a = a 0, a 1,, a n 1. More examples A x = 6x 3 + 7x 2 10x + 9 (9, 10, 7, 6) B x = 2x 3 + 4x 5 ( 5, 4, 0, 2) The operation of evaluating the polynomial A(x) at point x 0 consists of computing the value of A x 0. Evaluation taes time Θ(n) using Horner s rule A(x 0 ) = a 0 + x 0 (a 1 + x 0 (a x 0 a n 2 + x 0 a n 1 )) 5 Adding Polynomials Adding two polynomials represented by the coefficient vectors a = (a 0, a 1,, a n 1 ) and b = (b 0, b 1,, b n 1 ) taes time Θ(n). Sum is the coefficient vector c = (c 0, c 1,, c n 1 ), where c j = a j + b j for j = 0,1,, n 1. Example A x = 6x 3 + 7x 2 10x + 9 (9, 10, 7, 6) B x = 2x 3 + 4x 5 ( 5, 4, 0, 2) C(x) = 4x 3 + 7x 2 6x + 4 (4, 6,7,4) 6

4 Multiplying Polynomials For polynomial multiplication, if A(x) and B(x) are polynomials of degree-bound n, we say their product C(x) is a polynomial of degree-bound 2n 1. Example 6x 3 + 7x 2 10x + 9 2x 3 + 4x 5 30x 3 35x x 45 24x x 3 40x x 12x 6 14x x 4 18x 3 12x 6 14x x 4 20x 3 75x x 45 7 Multiplying Polynomials Multiplication of two degree-bound n polynomials A(x) and B(x) taes time Θ n 2, since each coefficient in vector a must be multiplied by each coefficient in vector b. Another way to express the product C(x) is 2n 1 j=0 c j x j j, where c j = =0 a b j. The resulting coefficient vector c = (c 0, c 1, c 2n 1 ) is also called the convolution of the input vectors a and b, denoted as c = a b. 8

5 Point-Value Representation A point-value representation of a polynomial A(x) of degree-bound n is a set of n point-value pairs x 0, y 0, x 1, y 1,, x n 1, y n 1 such that all of the x are distinct and y = A(x ) for = 0, 1,, n 1. Example A x = x 3 2x + 1 x 0, 1, 2, 3 A(x ) 1, 0, 5, 22 Using Horner s method, n-point evaluation taes time Θ(n 2 ). * 0, 1, 1, 0, 2, 5, 3, Point-Value Representation The inverse of evaluation is called interpolation determines coefficient form of polynomial from pointvalue representation For any set * x 0, y 0, x 1, y 1,, x n 1, y n 1 + of n pointvalue pairs such that all the x values are distinct, there is a unique polynomial A(x) of degree-bound n such that y = A(x ) for = 0, 1,, n 1. Lagrange s formula n 1 j (x x j ) A x = y j (x x j ) =0 Using Lagrange s formula, interpolation taes time Θ(n 2 ). 10

6 Example Using Lagrange s formula, we interpolate the pointvalue representation 0, 1, 1, 0, 2, 5, 3, x 1 (x 2)(x 3) = x3 6x 2 +11x 6 (0 1)(0 2)(0 3) 6 0 (x 0)(x 2)(x 3) (1 0)(1 2)(1 3) = 0 = x3 +6x 2 11x (x 0)(x 1)(x 3) = 5 x3 4x 2 +3x = 15x3 +60x 2 45x (2 0)(2 1)(2 3) (x 0)(x 1)(x 2) = 22 x3 3x 2 +2x = 22x3 66x 2 +44x (3 0)(3 1)(3 2) x3 + 0x 2 12x + 6 x 3 2x Adding Polynomials In point-value form, addition C x = A x + B(x) is given by C x = A x + B(x ) for any point x. A: x 0, y 0, x 1, y 1,, x n 1, y n 1 B: x 0, y 0, x 1, y 1,, x n 1, y n 1 C: * x 0, y 0 + y 0, x 1, y 1 + y 1,, x n 1, y n 1 + y n 1 + A and B are evaluated for the same n points. The time to add two polynomials of degree-bound n in point-value form is Θ(n). 12

7 We add C x Example = A x + B(x) in point-value form A x = x 3 2x + 1 B x = x 3 + x x = (0, 1, 2, 3) A: 0, 1, 1, 0, 2, 5, 3, 22 B: * 0, 1, 1, 3, 2, 13, 3, 37 + C: * 0, 2, 1, 3, 2, 18, 3, Multiplying Polynomials In point-value form, multiplication C x = A x B(x) is given by C x = A x B(x ) for any point x. Problem: if A and B are of degree-bound n, then C is of degree-bound 2n. Need to start with extended point-value forms for A and B consisting of 2n point-value pairs each. A: x 0, y 0, x 1, y 1,, x 2n 1, y 2n 1 B: x 0, y 0, x 1, y 1,, x 2n 1, y 2n 1 C: * x 0, y 0 y 0, x 1, y 1 y 1,, x 2n 1, y 2n 1 y 2n 1 + The time to multiply two polynomials of degreebound n in point-value form is Θ(n). 14

8 We multiply C x Example = A x B(x) in point-value form A x = x 3 2x + 1 B x = x 3 + x x = ( 3, 2, 1, 0, 1, 2, 3) We need 7 coefficients! A: 3, 17, 2, 3, 1,1, 0, 1, 1, 0, 2, 5, 3, 22 B: * 3, 20, 2, 3, 1, 2, 0, 1, 1, 3, 2, 13, 3, 37 + C: * 3,340, 2,9, 1,2, 0, 1, 1, 0, 2, 65, 3, The Road So Far a 0, a 1,, a n 1 b 0, b 1,, a n 1 Ordinary multiplication Time Θ(n 2 ) c 0, c 1,, c n 1 Coefficient representations Evaluation Time Θ(n 2 ) Interpolation Time Θ(n 2 ) A x 0, B x 0 A x 1, B x 1 A x 2n 1, B(x 2n 1 ) Pointwise multiplication Time Θ(n) C x 0 C x 1 C(x 2n 1 ) Point-value representations Can we do better? Using Fast Fourier Transform (FFT) and its inverse, we can do evaluation and interpolation in time Θ(n log n). The product of two polynomials of degree-bound n can be computed in time Θ(n log n), with both the input and output in coefficient form. 16

9 Amplitude Weight Fourier Transform Fourier Transforms originate from signal processing Transform signal from time domain to frequency domain Time Frequency Input signal is a function mapping time to amplitude Output is a weighted sum of phase-shifted sinusoids of varying frequencies 17 Fast Multiplication of Polynomials Using complex roots of unity Evaluation by taing the Discrete Fourier Transform (DFT) of a coefficient vector Interpolation by taing the inverse DFT of point-value pairs, yielding a coefficient vector Fast Fourier Transform (FFT) can perform DFT and inverse DFT in time Θ(n log n) Algorithm 1. Add n higher-order zero coefficients to A(x) and B(x) 2. Evaluate A(x) and B(x) using FFT for 2n points 3. Pointwise multiplication of point-value forms 4. Interpolate C(x) using FFT to compute inverse DFT 18

10 Complex Roots of Unity A complex n th root of unity (1) is a complex number ω such that ω n = 1. There are exactly n complex n th root of unity e 2πi n for = 0, 1,, n 1 where e iu = cos u + i sin (u). Here u represents an angle in radians. Using e 2πi n = cos 2π n + i sin( 2π n), we can chec that it is a root e 2πi n n = e 2πi = cos(2π) +i sin (2π) = Examples The complex 4 th roots of unity are 1, 1, i, i where 1 = i. The complex 8 th roots of unity are all of the above, plus four more 1 + i, 1 i, 1 + i, and 1 i For example i 2 2 = i 2 + i2 2 = i 20

11 The value Principal n th Root of Unity ω n = e 2πi n is called the principal n th root of unity. All of the other complex n th roots of unity are powers of ω n. The n complex n th roots of unity, ω n 0, ω n 1,, ω n n 1, form a group under multiplication that has the same structure as (Z n, +) modulo n. ω n n = ω n 0 = 1 implies ω n j ω n = ω n j+ = ω n j+ mod n ω n 1 = ω n n 1 21 Visualizing 8 Complex 8 th Roots of Unity imaginary ω 8 2 = i i ω 8 3 = i 2 ω 8 1 = i 2 ω 8 4 = 1 ω 8 0 = ω 8 8 = real ω 8 5 = 1 2 i 2 ω 8 7 = 1 2 i 2 i ω 8 6 = i 22

12 Cancellation Lemma For any integers n 0, 0, and b > 0, Proof ω d dn = e 2πi dn ω d dn = ω n. d = e 2πi n = ωn n For any even integer n > 0, ω 2 n = ω 2 = 1. Example ω 6 24 = ω 4 ω 6 24 = e 2πi 24 6 = e 2πi 6 24 = e 2πi 4 = ω 4 23 Halving Lemma If n > 0 is even, then the squares of the n complex n th roots of unity are the n 2 complex n 2 th roots of unity. Proof By the cancellation lemma, we have ω 2 n = ωn 2 for any nonnegative integer. If we square all of the complex n th roots of unity, then each n 2 th root of unity is obtained exactly twice ω n +n 2 2 = ωn 2+n = ω n 2 ω n n = ω n 2 = ω n 2 Thus, ω n and ω n +n 2 have the same square 24

13 Summation Lemma For any integer n 1 and nonzero integer not divisible by n, n 1 j=0 ω n j = 0. Proof Geometric series n 1 j=0 x j ω j n = ω n n n 1 1 j=0 ω n 1 = xn 1 x 1 = ω n n 1 = 1 1 = 0 ω n 1 ω n 1 Requiring that not be divisible by n ensures that the denominator is not 0, since ω n = 1 only when is divisible by n 25 Discrete Fourier Transform (DFT) Evaluate a polynomial A(x) of degree-bound n at the n complex n th roots of unity, ω n 0, ω n 1, ω n 2,, ω n n 1. assume that n is a power of 2 assume A is given in coefficient form a = (a 0, a 1,, a n 1 ) We define the results y, for = 0, 1,, n 1, by y = A ω n 1 j n = j=0 a j ω n. The vector y = (y 0, y 1,, y n 1 ) is the Discrete Fourier Transform (DFT) of the coefficient vector a = a 0, a 1,, a n 1, denoted as y = DFT n (a). 26

14 Fast Fourier Transform (FFT) Fast Fourier Transform (FFT) taes advantage of the special properties of the complex roots of unity to compute DFT n (a) in time Θ(n log n). Divide-and-conquer strategy define two new polynomials of degree-bound n 2, using even-index and odd-index coefficients of A(x) separately A 0 x = a 0 + a 2 x + a 4 x a n 2 x n 2 1 A 1 x = a 1 + a 3 x + a 5 x a n 1 x n 2 1 A x = A 0 x 2 + xa 1 (x 2 ) 27 Fast Fourier Transform (FFT) The problem of evaluating A(x) at ω n 0, ω n 1,, ω n n 1 reduces to 1. evaluating the degree-bound n 2 polynomials A 0 (x) and A 1 (x) at the points ω n 0 2, ω n 1 2,, ω n n combining the results by A x = A 0 x 2 + xa 1 (x 2 ) Why bother? The list ω n 0 2, ω n 1 2,, ω n n 1 2 does not contain n distinct values, but n 2 complex n 2 th roots of unity Polynomials A 0 and A 1 are recursively evaluated at the n 2 complex n 2 th roots of unity Subproblems have exactly the same form as the original problem, but are half the size 28

15 Example A x = a 0 + a 1 x + a 2 x 2 + a 3 x 3 of degree-bound 4 A ω 0 4 = A 1 = a 0 + a 1 + a 2 + a 3 A ω 1 4 = A i = a o + a 1 i a 2 a 3 i A ω 2 4 = A 1 = a 0 a 1 + a 2 a 3 A ω 3 4 = A i = a 0 a 1 i + a 2 + a 3 i Using A x = A 0 x 2 + xa 1 x 2 A x = a 0 + a 2 x 2 + x a 1 + a 3 x 2 A ω 4 0 = A 1 = a 0 + a 2 + 1(a 1 + a 3 ) A ω 4 1 = A i = a 0 a 2 + i(a 1 a 3 ) A ω 2 4 = A 1 = a 0 + a 2 1 a 1 + a 3 A ω 3 4 = A i = a 0 a 2 i(a 1 a 3 ) 29 Recursive FFT Algorithm RECURSIVE-FFT a 1 n length,a- n is a power of 2 2 if n = 1 3 then return a basis of recursion 4 ω n e 2πi n ω n is principal n th root of unity 5 ω 1 6 a 0 (a 0, a 2,, a n 2 ) 7 a 1 (a 1, a 3,, a n 1 ) 8 y 0 RECURSIVE-FFT a 0 y 0 = A 0 ωn 2 = A 0 (ω 2 n ) 9 y 1 RECURSIVE-FFT a 1 y 1 = A 1 ωn 2 = A 1 (ω 2 n ) 10 for 0 to n do y y ωy 12 y +( n 2 ) y 0 1 ωy since ω + n = ω n 2 n 13 ω ωω n compute ω n iteratively 14 return y 30

16 Why Does It Wor? For y 0, y 1, yn 2 1 (line 11) y = y 0 + ω 1 n y = A 0 2 ω n + ω n A 1 2 ω n = A ω n For yn 2, yn 2 +1,, y n 1 (line 12) y + n 2 = y 0 ω 1 n y = y 0 + ω n + n 2 y 1 = A 0 ω n 2 + ω n + n 2 A 1 ω n 2 = A 0 ω n 2+n + ω n + n 2 A 1 ω n 2+n = A ω n + n 2 since ω n = ω n + n 2 since ω n 2+n = ω n 2 31 Input Vector Tree of RECURSIVEFFT(a) (a 0, a 1, a 2, a 3, a 4, a 5, a 6, a 7 ) (a 0, a 2, a 4, a 6 ) (a 1, a 3, a 5, a 7 ) (a 0, a 4 ) (a 2, a 6 ) (a 1, a 5 ) (a 3, a 7 ) (a 0 ) (a 4 ) (a 2 ) (a 6 ) (a 1 ) (a 5 ) (a 3 ) (a 7 ) 32

17 Interpolation Interpolation by computing the inverse DFT, denoted by a = DFT n 1 (y). By modifying the FFT algorithm, we can compute DFT n 1 in time Θ(n log n). switch the roles of a and y replace ω n by ω n 1 divide each element of the result by n 33