7i x i Felder c9_olie.tex V - Jauary 4, 5 : P.M. Page CHAPTER 9 Fourier Series ad Trasforms (Olie) 9.7 Discrete Fourier Trasforms AFourierseries models a fuctio as a discrete set of umbers (coefficiets of a sum); a Fourier itegral models a fuctio as a cotiuous fuctio (the trasform). I both cases the fuctio you start with is cotiuous. I this sectio we will use Fourier aalysis o a data set that is discrete to begi with: a set of poits, or data, rather tha a algebraically defied fuctio. 9.7. Explaatio: Discrete Fourier Trasforms Throughout this chapter we have used Fourier aalysis (series or trasforms) to see what oscillatios make up a particular fuctio. I practice, though, we rarely measure a fuctio such as f (x) =e x. We measure data poits. Cosider, for example, the Motivatig Exercise (Sectio 9.). Observers repeatedly measure the velocity of a star over may years. A star with o plaets aroud it should have a costat velocity, a star with oe plaet should oscillate like a sie wave, a star with two plaets should follow a superpositio of two sie waves, ad so o. A discrete Fourier trasform starts with idividual velocity measuremets ad fids the oscillatios represeted by those poits, eablig us to determie the presece of plaets. But Figure 9. does t look like a superpositio of sie waves, does it? Real oscillatios are buried uder radom oise caused by limitatios of the measurig istrumets, atmospheric distortio, ad other factors. These radom fluctuatios are ofte may times greater tha the oscillatios themselves. Amazigly, the Fourier trasform ca geerally cut through such oise to fid the uderlyig order. Below we preset the formula for a discrete Fourier trasform, but first let s make sure the letters are clear. Take a look f at Figure 9.9. We begi with a fuctio f (x) represetig N f idividual poits spaced Δx apart. For istace, if we have f (), f (), f (4),adf(6), the Δx = ad N = 4. (The formula f f assumesδx is costat: that is, the poits are sampled at evely f spaced itervals.) We refer to the data poits as f, f, f, f, ad so o. We use the letter r as a idex for this list. I our example above, f Δx Δx Δx x would mea f (6); more geerally, f r meas f (rδx). FIGURE 9.9 With four data Our goal is to express the data as <some coefficiet> times a poits r =,,,. oscillatio of a particular frequecy, plus <a differet coefficiet>
7i x i Felder c9_olie.tex V - Jauary 4, 5 : P.M. Page Chapter 9 Fourier Series ad Trasforms (Olie) times a oscillatio of a differet frequecy, ad so o. We refer to these coefficiets as, ad so o, usig as a idex ito this list. Below, the, we preset the formula for turig the f r list of data poits ito the f list of frequecy coefficiets. The Formula for a Discrete Fourier Trasform Suppose a fuctio f (x) is kow at N discrete poits: f r = f (rδx) for r =,,, N. The discrete Fourier trasform (DFT) a of f is a set of poits f with N N give by the formula: N = f r e πir N (9.7.) r= The umbers f are proportioal to the amplitudes of oscillatios with frequecies give by: p = π N Δx (9.7.) It s easy to glide over those equatios too quickly. To slow dow, cosider the example of geerated from a set with N = data poits. Equatio 9.7. tells us how to fid : it is a sum over all the values i the origial data set. f = f + f e 6πi + f e πi + f e 8πi + +f 99 e 6(99)πi The 4 is a differet sum over the etire origial data set, ad so o. (Such liear combiatios ca be cocisely represeted with matrices; see Problem 9.4.) Equatio 9.7. tells us what represets: it tells us how much of the origial data set oscillates with frequecy 6π (Δx). Note that the deomiator N Δx is the etire spa of the origial data set which meas that π (N Δx), 4π (N Δx), 6π (N Δx), etc. are all the frequecies that fit perfectly ito that domai. As you ca see, calculatig N -values for your N data poits i this way requires summig N terms. But discrete Fourier trasforms are so importat that decades of research have goe ito algorithms that ca quickly calculate the DFT of a list with billios of iput poits. The eed to do so arises more ofte tha you might thik, i applicatios ragig from image maipulatio to electroic sigal processig. Iterpretig a Discrete Fourier Trasform As with so may of these formulas, Equatio 9.7. hides a umber of subtleties. The DFT formula assumes that the fuctio you are modelig is periodic, ad that your data represet a full period. For istace (as we metioed above) if you start with data poits with spacig, the terms i the DFT represet a period of, a period of /, a period of /, ad so o; all of them start over at precisely. If you iverse trasform your DFT you will recreate the origial data poits ad the reproduce them agai ad agai. a The acroym DFT is widely used for Discrete Fourier Trasform, but ufortuately it has a umber of other uses. The most problematic for our readers may be Desity Fuctioal Theory, a model of may-body quatum systems used by physical chemists. There is probably less dager of cofusio with the UK s Departmet For Trasport.
7i x i Felder c9_olie.tex V - Jauary 4, 5 : P.M. Page 9.7 Discrete Fourier Trasforms Equatio 9.7. makes o metio of Δx so the DFT depeds o the values of your data poits but ot o how ofte they were sampled. If you sample poits over the course of a year, tells you how much your data oscillate with a period of a third of a year. If you sample the same poits i a miute you will get the same,thistime represetig a period of secods. Strictly speakig is ot the amplitude of a oscillatio; N is the amplitude. That matters for a iverse discrete Fourier trasform, but for most purposes we oly care about the relative amplitudes of the oscillatios so we igore that detail. We said above that Equatio 9.7. assumes its iput (the f r umbers) are periodic, repeatig the same N umbers over ad over. The output (the umbers) will also be periodic with period N. For istace if you put i data poits you will fid that = ad = ad so o. I most cases the frequecies of iterest are give by N N. 4 A egative just meas a egative expoet i the complex expoetial, but ad both represet oscillatios with the same period, just as they do i regular Fourier series with complex expoetials. Just as with a regular Fourier trasform, if the iput data are real the will always be the complex cojugate of. Sice DFTs are geerally used to model measured data, which is by defiitio real, you ca geerally get away with oly calculatig about half of the amplitudes. (I our example above you could calculate through 5 the hard way ad the you would kow 5 through.) EXAMPLE Discrete Fourier Trasform Problem: Fid a discrete Fourier trasform of the list f () =, f () =4, f (6) =, f (9) =. Solutio: We plug f =, f = 4, f =, f =, ad N = 4 ito Equatio 9.7.. = + 4 + + = = + 4e πi 4 + e πi() 4 + e πi() 4 = + 4( i)+( )+i = i = + 4e πi() 4 + e πi()() 4 + e πi()() 4 = + 4( )+ + ( ) = 4 It s worth goig through that exercise to see the patters i the formula. You ca see the r-values coutig,,, as you move across the terms i ay lie, ad you ca see the -values coutig,, as you cout dow the lies. We could similarly calculate, but the fact that the iputs were all real guaratees that = = + i. (You ca do the calculatio directly ad verify this.) We kow for the same reaso that =, ad we kow from periodicity that =. The oly way those ca both be true is if is real, which we see it is. With N real iputs, N will always be real (assumig N is eve). Ca you see why the same argumet guarateed that would be real as well? So the DFT, from = N ton, is ( 4, + i,, i, 4). Oce you have goe through the process oce or twice, there s o great value i repeatig it. DFTs are meat for extremely large data sets ad are performed by computers. 4 Techically f N = f N so you oly eed to calculate values from = N + ton.
7i x i Felder c9_olie.tex V - Jauary 4, 5 : P.M. Page 4 4 Chapter 9 Fourier Series ad Trasforms (Olie) Questio: What does f = 4represet? Aswer: From Equatio 9.7. it represets the amplitude associated with the frequecy π() (4 ) =π. Iverse Discrete Fourier Trasforms Whe we take the Fourier series of a fuctio we get a list of coefficiets c. If we multiply each of those coefficiets by e πix L ad add them, we recover the origial fuctio. I a similar way, a discrete Fourier trasform takes as iput the value of a fuctio at certai discrete poits ad gives as output the coefficiets, ad the iverse discrete Fourier trasform calculates the origial values f r from those coefficiets. Iverse Discrete Fourier Trasform The iverse discrete Fourier trasform of a set of poits f is f r = N N = N + f e πir N (9.7.) (The sum ca go over ay N cosecutive values of, ad it is ofte writte from to N, but we prefer this way because it focuses o the most physically meaigful values. Clearly for odd N you would have to adjust the limits of the sum.) Recall that the x-value of each f r is rδx. If we plug r = x Δx ito Equatio 9.7. the expoetial becomes e πi (N Δx)x, which is why we said above that each mode has frequecy π (N Δx). We ca also see from this formula that is ot techically the coefficiet of the oscillatio; N is. Remember that the DFT coefficiets are periodic; they are N umbers that repeat forever. You ca geerate a iverse DFT from ay N such umbers i a row ad get a fuctio that perfectly replicates the origial poits you used to create your DFT. Betwee those poits, however, the behavior may surprise you uless you use the rage specified i Equatio 9.7.. See Problem 9.7. With the DFT ad iverse DFT formulas you ca traslate a data set ito Fourier space ad the recover the origial data. That may ot seem very useful, but for may applicatios we ca trasform a set of data ito Fourier space, maipulate it i some way, ad the do a iverse trasform to get back to regular space. This is illustrated i the example below. EXAMPLE Smoothig the Rough Edges Problem: Copy a image from the Web ad elimiate all high frequecy oscillatios from it. Solutio: We chose a image of Rodi s sculpture The Thiker. We imported this photo ito Mathematica, which stored it as a 7 48 grid of umbers. We the asked
7i x i Felder c9_olie.tex V - Jauary 4, 5 : P.M. Page 5 9.7 Discrete Fourier Trasforms 5 Mathematica to take a discrete Fourier trasform of that grid of umbers. (We have t give you the formula for a multivariate discrete Fourier trasform, but Mathematica s built-i DFT routies kew just what to do.) The result was a 7 48 grid of complex umbers represetig the DFT. A iverse Fourier trasform of this grid reproduced the origial picture exactly. We the set about 99% of the umbers i the DFT to zero, leavig oly the low frequecy terms. A iverse Fourier trasform of the resultig frequecies produced a softeed versio of the origial. Fially, we set over 99.99% of the umbers i the DFT to zero. A iverse trasform of the resultig set was still very recogizable, but blurry. Image reproduced from % of the data i Fourier space, image reproduced from % of the data, ad image reproduced from.6% of the data Rafael Ramirez Lee/Shutterstock. This techique ca be used to deliberately softe images, but it also provides powerful data compressio. You ca store pictures i a fractio of the space required for a ucompressed image. The implicatios for dowloadig pictures (i a Web browser for istace) are also importat. If you dowload a picture as pixels the user sees the top row of pixels appear at first, fillig i toward the bottom over time. But if you dowload the same iformatio as frequecies the user sees the etire image almost istatly, sharpeig to full resolutio over time. Where Did Those Formulas Come From? Cosider a regular Fourier series for a fuctio with period L: f (x) = c e πix L where the coefficiets are give by c = L f (x)e πix L dx L If we kow the fuctio f (x) we ca fid the coefficiets c by itegratig, aalytically or umerically. If we oly kow f (x) at N evely spaced itervals x = rδx the we must approximate this itegral with a sum. c N f (rδx)e πirδx L Δx L r=
7i x i Felder c9_olie.tex V - Jauary 4, 5 : P.M. Page 6 6 Chapter 9 Fourier Series ad Trasforms (Olie) The spacig Δx is related to the total period L ad the umber of sample poits N by Δx = L N. Substitutig this i gives c N N f (rδx)e πir N r= We could have left the formula like this, but historically the terms i a DFT are ot defied to make c, but rather Nc, which thus elimiates the N from the formula, givig us Equatio 9.7.. A similar calculatio ca be used to derive Equatio 9.7. for the iverse discrete Fourier trasform. A Few Practical Notes While discrete Fourier trasforms have bee aroud for ceturies, their use did t become commo util Cooley ad Tukey published a DFT algorithm that is i may cases millios of times faster tha straightforwardly applyig Equatio 9.7.. 5 That algorithm is ow kow as a Fast Fourier Trasform. Oe of us (GF) rus simulatios of the early uiverse usig hudreds of processors to take FFTs of datasets with over 6 billio poits, ad such large trasforms are commo i may areas of sciece ad egieerig. As a practical matter, the FFT is most efficiet whe the umber of data poits N is a power of. If you are workig with ay other size data set it is usually worth paddig the ed with extra zeroes to make N a power of. It s also importat to remember that a DFT gives us oscillatios at the frequecies p = π (N Δx), 4π (N Δx) π Δx. That highest frequecy p = π Δx is called the Nyquist frequecy, ad it sets a limit o how high a frequecy we ca fid from our data. The fact that the Nyquist frequecy is iversely proportioal to Δx makes sese: if we sample oce per secod, we wo t fid a oscillatio that happes te times per secod! If a fuctio oscillates much faster tha our samplig rate, our discrete Fourier trasform will be subject to aliasig. The power that is actually i modes above the Nyquist frequecy will appear icorrectly i the lower frequecy modes, causig them to come out higher tha they should. As a practical matter you should either kow ahead of time what rage of oscillatio frequecies your data is likely to have or figure it out by trial ad error, ad make sure you sample frequetly eough that your Nyquist frequecy is higher tha the highest sigificat frequecy of your physical system. 9.7. Problems: Discrete Fourier Trasforms 9.99 Walk-Through: Discrete Fourier Trasforms. You ve measured the followig data poits for a fuctio f (x): f () =, f () =, f (4) = 6, f (6) =. (a) Use Equatio 9.7. to calculate,,ad. (b) Fid without usig Equatio 9.7.. This should take o more tha secods. (c) What are ad? Agai, more tha secods meas you re doig it wrog. (d) What frequecies p are represeted by the terms,,,ad? For Problems 9. 9.5 fid the discrete Fourier trasform of the give list of umbers, goig from = N ton. Give the smallest magitude o-zero frequecy ad the largest frequecy (the Nyquist frequecy) represeted by the DFT. 9. f () =, f (5) = 5 Cooley, James W.; Tukey, Joh W. (965). A Algorithm for the machie calculatio of complex Fourier series. Math. Comput. 9: 97. The algorithm had bee idepedetly discovered umerous times before Cooley ad Tukey, begiig with Gauss i 85. For a discussio of the FFT algorithm itself see Numerical Recipes by Press, et. al.
7i x i Felder c9_olie.tex V - Jauary 4, 5 : P.M. Page 7 9.7 Discrete Fourier Trasforms 7 9. f () =, f (.6) =, f (.) =, f (.8) = 7 9. f () =, f (.) =, f (.4) =, f (.6) = 9. f () =5, f () =4, f () =, f () = 9.4 f () =5 + i, f () =i, f (4) =, f (6) = i 9.5 f () =, f (4) =4, f (8) =, f () = 6, f (6) =, f () = 4 9.6 Calculate the DFT of f () =5, f (5) = 5, f () =5, f (5) = 5. Explai why your results make sese. I other words, how could you have predicted most of the values you foud without havig to calculate aythig? 9.7 Cosider the poits f () =, f () =7, f (6) =, f (9) =, f () =4, f (5) =, f (8) =, f () =. (a) Calculate the DFT coefficiets for 7. (b) O oe plot show the real ad imagiary parts of ( 8) 7 f = e πix. Show the origial 8 poits f r o the same plot. (c) Repeat Part (b) for 4. Hit: you ca easily get all the -values you eed from the oes you already calculated. (d) O oe plot show the origial poits f r ad the real ad imagiary parts of ( 8)[( )( 4 e 8πix + 4 e 8πix + f = e πix ]. (e) You should have foud that all three fuctios you just defied go through all of the origial data poits f r, but behave differetly i betwee them. Lookig at your plots, which of these fuctios is likely to be the most useful for modelig your origial fuctio, ad why? 9.8 Have a computer geerate a table of 56 poits f (x) where x goes from to. At each poit let f (x) =si(x)+si(5x)+r, where R is a radom umber (differet at each poit) from 4 to 4. Plot the data poits o a graph of f (x) vs. x. The take a discrete Fourier trasform of the data ad plot the resultsoagraphoff (p) vs. p. Explai why the results appear the way they do. 9.9 Use x = rδx, p = Δp = π (N Δx), ad = Nc to rewrite Equatio 9.7. so it looks like the usual expressio for a Fourier series. A ormal Fourier series has ifiitely may terms, but this oe will oly have N.All the others ca be take to be zero. 9. Plug Equatio 9.7. ito Equatio 9.7. ad verify that it does simplify to f r.at some poit i the calculatio you should ecouter a term of the form N j= N + eπki N, where k is a iteger. This sum equals zero uless k =. (You ca easily figure outwhatitequalswhek =.) 9. Use Equatio 9.7. to show that +N =. Formally a DFT is periodic i frequecy space, but i practice that really meas all of the frequecies outside the rage p π Δx are meaigless. 9. Cosider the fuctio f (x) =e cos x. (a) Have a computer geerate a table of values of f (x) ragig from x = to x = 99.5. Plot these values o a graph of f vs. x. (Just plot the values i your table, ot the curve for the fuctio.) (b) Take a discrete Fourier trasform of your list of values. What frequecy is associated with the th term i that list? Plot the moduli of the values of the discrete Fourier trasform from = to =, with the horizotal axis showig the associated frequecies. (Leave out =, which just represets the costat term.) (c) What frequecy shows the highest peak i the DFT? What does that tell you about the origial fuctio? (Hit: you could have predicted this peak before doig the calculatios.) (d) What frequecy shows the secod highest peak i the DFT? What does that tell you about the origial fuctio? (You probably could t have predicted this oe.) 9. Cosider the list f r =(,,, 4,,,,,, 4,, ). (a) Plot these data poits, usig the x-values,,, 4, 5, 6,. (b) Take a discrete Fourier trasform of the list, the take a iverse discrete Fourier trasform, ad the plot the resultig poits. This plot should look idetical to the previous oe. (c) Take a discrete Fourier trasform of f r, set = i the resultig list, the take its iverse discrete Fourier trasform ad plot the results. How does this look differet from your origial plot? (d) Take a discrete Fourier trasform of f r, set = =, take a iverse discrete Fourier trasform, ad plot the results. How does this plot look differet from your origial plot? Hit: some computers store the results i the order N,so the compoet may be stored as N.
7i x i Felder c9_olie.tex V - Jauary 4, 5 : P.M. Page 8 8 Chapter 9 Fourier Series ad Trasforms (Olie) 9.4 (This problem requires liear algebra.) (a) Startig from Equatio 9.7. write a matrix for covertig a pair of data poits f r ito a DFT.(Remember that ad r go from to.) (b) Startig from Equatio 9.7. write a matrix for covertig ito f r. (c) Prove that the two matrices you just wrote are iverses. What does this tell you about the two trasformatios? (d) Write a matrix to fid, ad from three data poits f, f, f. 9.5 Look through the Example Smoothig the Rough Edges o Page 4. Repeat this process for aother image. Choose a image, import it ito a computer algebra program, take a discrete Fourier trasform of the image data, set the high frequecy compoets to zero, ad iverse trasform it to get a ew image. You may eed to do some trial ad error to figure out exactly which compoets to elimiate. Be aware that whe you elimiate Fourier coefficiets the iverse discrete Fourier trasform will usually give you complex umbers. The easiest way to deal with this is to just take the real part after you take the IDFT. Hit: If you use a color image the each pixel will most likely be a list of three umbers istead of a sigle umber. You ca still do the same thig. Each compoet i Fourier space will be a list of three umbers ad you ca set them all equal to zero for the high frequecy modes.