74 CHAPTER AALYSIS OF DISCRETE-TIME LIEAR TIME-IVARIAT SYSTEMS 4 Fast Fourier Transform (FFT Algorithm Fast Fourier Transform, or FFT, is any algorithm for computing the -point DFT with a computational complexity of O( log It is not a new transform, but simply an efficient method of calculating the DFT of x(n If we assume that is even, we can write the -point DFT of x(n as X ( (k n is even: n is odd: m We make the following substitutions: x(ne j nm,m,,, x(ne j nl,l,,, j x(me m l n n x (m x(m, where m,,, x (l x(l, where l,, j x(l e (l ( Rewriting Eq (, we get X ( (k x (me m j m e j x (le l j l X ( j ( (ke X (k, ( where X ( (k is the -point DFT of the even-numbered samples of x(n and X( (k is the -point DFT of the odd-numbered samples of x(n ote that both of them are -periodic discrete-time functions We have the following algorithm to compute X ( (k for k,, ( : Compute X ( (k for k,, Compute X ( (k for k,, Perform the computation ( with complex multiplications and complex additions Actually, it is possible to use fewer than complex multiplications Let W e j π
Sec 4 Fast Fourier Transform (FFT Algorithm 75 x( x( x( x( x( x( -pt DFT X ( ( X ( ( X ( ( -pt X ( ( DFT X ( ( X ( W W X ( ( W X ( ( X ( ( W X( ( X ( ( X ( X ( ( W X( ( X ( X ( ( Figure 6 The FFT algorithm Then j( π W k e j e W k Therefore, X ( (k X ( (kwx k (k for k,,, ( X ( k X ( (k W k X (k for k,,, as illustrated in Fig 6 This shows that we do not need to actually perform complex multiplications, but only 8 Fig 7 illustrates the recursive implementation of the FFT supposing that M There is a total of M log stages of computation, each requiring complex operations Hence, the total computational complexity is O( log We see that the process ends at a -point DFT A -point DFT is the sample of the original signal: X( n π x(ne j( n x( The following remarks apply to the FFT: 8 Actually, slightly fewer if we do not count multiplications by ± and ±j
76 CHAPTER AALYSIS OF DISCRETE-TIME LIEAR TIME-IVARIAT SYSTEMS -pt DFT M -pt DFT M -pt DFT 4 X (k 4 X (k 4 X (k 4 X (k X (k operations X (k operations M -pt DFT X ( (k operations x x Total number of complex operations per DFT operation Total number of complex operations per level of recursion Figure 7 The recursive implementation of the FFT supposing that M There is a total of M log stages of computation, each requiring complex operations Hence, the total computational complexity is O( log For large, the FFT is much faster than the direct application of the definition of DFT, which is of complexity O( The particular implementation of the FFT described above is called decimationin-time radix- FFT The number of operations required by an FFT algorithm can be approximated as C log, where C is a constant There are many variations of FFT aimed at reducing this constant eg, if M, it may be better to use a radix- FFT 4 ote that { } DFT[x (n] { n n x (ne j( n x(ne j( n } which is the IDFT of x(n Thus, the FFT can also be used to compute the IDFT
Sec 4 Fast Fourier Transform (FFT Algorithm 77 -point DFT of x( x(4 «4-point DFT of B @ x( x( x(4 x(6 C A x( x(4 X (8 ( X (8 ( x( x(6 x( x(5 x( x(7 W 4 «x( -point DFT of x(6 «x( -point DFT of x(5 W 4 W 8 W 8 W 8 X (8 ( X (8 ( X (8 (4 X (8 (5 X (8 (6 X (8 (7 -point DFT of x( x(7 «4-point DFT of B @ x( x( x(4 x(6 C A Figure 8 The 8-point FFT Example 6 The 8-point FFT is depicted in Fig 8 The values of the twiddle factors are: W e j π, W 4 e j π 4 j, W 8 e j π 8
78 CHAPTER AALYSIS OF DISCRETE-TIME LIEAR TIME-IVARIAT SYSTEMS X A ( x W W W W W W A ( A ( x W W W W W W A ( A ( x( x( x( x( x( x( W X ( W W X ( W ( W W ( X ( ( X ( ( Figure 9 The FFT reduces the number of operations required to calculate the DFT by reducing A ( to two A ( that is only half the size of A ( This operation is repeated with every recursion until we reach the -point DFT
Sec 4 Fast Fourier Transform (FFT Algorithm 79 Recall that the DFT is a matrix multiplication (Fig 5 One stage of the FFT essentially reduces the multiplication by an matrix to two multiplications by matrices This reduces the number of operations required to calculate the DFT by almost a factor of two (Fig 9 Another interpretation of FFT involves analyzing the matrix ( A k,l j e L, j e L where k and L are nonnegative integers such that k< L ote that A k,l x,a k,l y (A k,l y H (A k,l x y H A H k,l A k,lx ( y H e j e j ( y H ( x y H x x, y, j e e j x ie, multiplication by A k,l preserves distances and angles roughly speaking, it is a rotation or reflection Continuing the matrix decomposition of Fig 9 further until we get the full FFT, it can be shown that FFT consists of log multiplications by matrices of the form A k,l, each operating on a pair of coordinates 9 Therefore, FFT breaks down the multiplication by the DFT matrix A into elementary planar transformations 4 Fast Computation of Convolution Consider a linear system described by y Sx, ( where x is the input vector, representing an -periodic input signal; S is an matrix; and y is the output vector, representing an -periodic output signal What conditions must the matrix S satisfy in order for the system to be time-invariant, ie, invariant to circular shifts of the input vector? ote that a circular shift by one sample is x( x( x( x( x( x( x( x( x( 9 The same conclusion can be reached by examining an FFT diagram such as Fig 8
8 CHAPTER AALYSIS OF DISCRETE-TIME LIEAR TIME-IVARIAT SYSTEMS Let the first column of S be ote that when and when h x h( h( h( h(, then y h, x then y is the second column of S, which therefore, in order for S to be invariant to circular shifts, must be equal to: h( h( h( h( Similarly, when x,,, then y is the third column of S, etc Thus, the matrix S must have the following structure: h( h( h( h( h( h( h( h( S h( h( h( h( h( h( h( h(
Sec 4 Fast Fourier Transform (FFT Algorithm 8 This is called a circulant matrix We can then write Eq ( as y(n m m x(mh(n m x(mh((n m mod (4 x h(n x h (5 Eq (5 is called a circular convolution or a periodic convolution ote that formula (4 works even when x or h are non-periodic Observe the following: For y(, the sum of the indices of x and h is always mod for every term y( x(h( x(h( x(h( x( h( For y(, the sum of the indices of x and h is always mod for every term y( x(h( x(h( x(h( x( h( This is true for all y(k, k,,, What are the eigenvectors of S? Let us try ej g k ej, where k,,, We have: ej ( y(n h(n g k m m { m h(mg k (n m h(m H(k } {{ } DFT of h ej (nm } j h(me m ej n ej n
8 CHAPTER AALYSIS OF DISCRETE-TIME LIEAR TIME-IVARIAT SYSTEMS Hence we have that Sg k H(kg k where g k is the k-th eigenvector and H(k gives the corresponding eigenvalue Therefore, H( S ( g g g ( H( g g g } {{ } The IDFT matrix B H( Then S can be written as: S B H( H( H( A, where the DFT matrix A is: A B H g H g H g H Complex exponentials are the eigenvectors of circulant matrices They diagonalize circulant matrices Thus, for any x C, H( H( Sx B Ax H( Let us compare two algorithms for computing the circular convolution of x and h Algorithm Directly perform the multiplication Sx This has computational complexity O ( Algorithm Represent x in the eigenbasis of S, ie, the Fourier basis, X Ax This step can be done with FFT whose complexity is O( log
Sec 4 Fast Fourier Transform (FFT Algorithm 8 Step Step Step x(n h(n -point DFT -point DFT X(k H(k Y (k X(kH(k Y (k -point IDFT y(n x h(n Figure 4 An illustration of the FFT implementation of the circular convolution Compute the representation of y in the eigenbasis of S: H( H( Y H( X This computation has complexity O( Reconstruct y from its Fourier coefficients: y BY This has complexity O( log, if done using the FFT This algorithm is summarized in Fig 4 Its total complexity is O( log (ote that the second algorithm does not necessarily perform better for any matrix Example 7 This example explores the relationship between the convolution and the circular convolution Let x and h be -periodic signals, and let { x(n, n x z, otherwise { h(n, n h z, otherwise If we let y z (n x z h z (n y(n x h(n then y(n can be expressed as { yz (ny y(n z ( n, n,,, y(, n
84 CHAPTER AALYSIS OF DISCRETE-TIME LIEAR TIME-IVARIAT SYSTEMS ote that the overlap of y z (n and y z ( n causes temporal aliasing in the resulting y(n This is the main difference between convolution and circular convolution x(n h(n 4 n 4 n (a Convolution y z (n x z h z (n y z (n 5 8 4 5 n (b Circular convolution y(n x h(n y(n 5 8 5 5 5 8 y( y z( y z( y( y z( y z(4 y( y z( y z(5 4 5 n Figure 4 A comparison between circular convolution and convolution Fig 4 illustrates the effect of temporal aliasing To remove or minimize the effect of temporal aliasing, we could zero-pad x and h so that the temporal replicas are spread further apart, and thus, overlapping would not occur