Lecture 4: Noise and the Fourier Transform

Methods of Experimental Physics Semester 1 25 Lecture 4: Noise and the Fourier Transform 4.1 Introduction Lets start with some simple definitions. In class I ll give some examples of the various outcomes in an interactive way using Igor. This work may appear arcane but will turn out to be very useful in understanding transfer functions, getting a good method of expressing noise and thinking about filtering. I apologise for the apparent level of complexity but you are going to be surprised by how practical all of this stuff is. Most of this chapter and the next is based on Chapter 12 in Numerical Methods, Chapter 15 in Reif Fundamentals of statistical and thermal physics, Norton Fundamentals of noise and Vibration analysis for engineers as well as Papoulis The Fourier Integral and its Applications, Brigham The Fast Fourier Transform and its Applications as well as some of the help files of Igor and Mathematica. In this lecture feel free to jump all the intermediate results if you are willing to take on face value the various definitions that you find scattered throughout. Perhaps to start I should state that there are four ways to characterise random signals (or noise): 1. mean square values and variance - these provide amplitude information about a signal 2. probability distributions - provide information about statistical properties in amplitude domain 3. correlation functions - provide information about statistical properties in the time domain 4. spectral density functions - provide information about the statistical properties in the frequency domain we have already come across mean square values and variance in a previous lecture. In this lecture we will address correlation functions and spectral densities. In addition, I want to start by giving a few reasonable definitions for a real random/noisy signal. Hopefully, in the next lecture we will look at one example of a probability distribution in relation to white noise. A time history of a random signal is called a sample record, and if we take a collection of these consititues an ensemble average of the process. A random process is called ergodic (or stationary) if all distributions associated with it are time invariant i.e. the distribution of one record, is the same as any other record as it is of the ensemble average. Usually it is a reasonable assumption that the things we will look at are nearly stationary. Throughout this lecture we will use a function h(t) to represent some time series of data points, with t standing for time. Its Fourier transform pair will be denoted H(f) with the f representing frequency. Mathematically these functions can represent anything, and are not limited to time and frequency but in almost this entire course h(t) will be a time series, while H(f) will be a frequency spectrum. 4.2 Probability functions The mean of a function h(t) is given by: Definition 4.1 1 T T h(t) dt = hp(h) dh where p(h) is the probability density function, which specifies the probability, p(h) dh, that the function h(t) lies in the range h to h + dh. For a stationary process h. The mean square value is given by: 4-1

Lecture 4: Noise and the Fourier Transform 4-2 6 5 4 3 2 1-1 2 4 6 (a) some noisy traces (b) The histograms of the noisy traces Figure 1: Definition 4.2 h 2 = 1 T T h(t) 2 dt = h 2 p(h) dh The root-mean-square value is just the positive square root of this value. The standard deviation of h(t) termed σ, and variance σ 2 are defined by: Definition 4.3 σ 2 = h 2 (h) 2 For digitally sampled data (more about this later) the mean and mean square are simply: 1 h = lim N N N i=1 h i h 2 1 = lim N N See figure 1 and Table 1 for explicit examples of this. Consider whether the statisical measures delivered are sensible in each of the cases. Type mean root-mean-square standard deviation N i=1 white noise.58.583 gaussian noise 3 3.5.5 drifting 5.4 5.4.39 Table 1: The parameters of the noisy time series in Figs. 1. h 2 i 4.3 Correlation Functions One can characterise a random signal in terms of how its value at one time, t, depends on the value at some other time, t + τ. This is termed the auto-correlation function; likewise one can also talk about the correlation of one signal against another - this is termed the cross-correlation function. For stationary signals only the time separation between samples matters (the particular time choice should not) and the autocorrelation function is defined as: 1 T K(τ) h(t)h(t + τ) = lim h(t)h(t + τ) dt (1) T 2T T

Lecture 4: Noise and the Fourier Transform 4-3.3.2.1. -.1 -.2 Amplitude (a.u) 12 1 8 6 4 2-2 2 4 6 8 1 Point Number 12-1 -5 5 1 Lag (points) (a) a random noisy function (b) The autocorrelation function of that function Figure 2: It is really painful to work with autocorrelation functions when the mean, h, is non-zero; so during this course we will stick to random functions with a mean of zero. One notes that in this case K() = h 2 = σ 2. Alternatively one can make use of a normalized autocorrelation function: κ(τ) = (h(t) h)(h(t + τ) h) σ 2 (2) which handle the non-zero mean, and have the property that κ() = 1 while κ(t) as t. Anyway, autocorrelations of random signals almost always look like the Fig. 2 i.e. a peak that rapidly falls and then eventually approaches zero very closely as the lag becomes large. The other obvious feature is that the function is symmetric about a lag of zero. The time duration corresponding to the lag where the autocorrelation has fallen substantially from its peak value is referred to the correlation time - this is essentially the memory of the system under study. We will now turn to Fourier transforms and then return to the relationship between the correlation functions and the Fourier transform. 4.4 Defining the Transform pairs Imagine some random function of time that we are going to call h(t), let s imagine furthermore that it is only non-zero over some finite interval, θ < t < θ, but apart from this it is zero. This is perhaps mathematically not very pretty but is probably realistic since if we are going to use these functions practically we are only going to have a finite sample. In addition, we are going to use integrals that we want to be sure actually converge and a finite duration function will satisfy this requirement. Recalling that the complex exponential function is orthogonal (see A.7 from Reif if you don t remember, or see Appendix I from Papoulis): 1 e ıω(t t) dω = δ(t t ) (3) 2π where δ(t t ) is the Dirac delta function. The Dirac function is also called the impulse function for reasons that will become apparent later. This type of definition implies also that we can evaluate the following types of integrals: cos(ωt) dω = 2πδ(t) One needs to examine the theory of distributions (see Appendix I from Papoulis) to be convinced of these types of things...or just take it on faith and use it.

Lecture 4: Noise and the Fourier Transform 4-4 Reminder of the meaning of the Dirac delta function (which strictly speaking isn t a function at all): t+ɛ t ɛ δ() (4) δ(not zero) = (5) δ(t t ) dt = 1 (6) Now lets use a strange method to redefine a function (i.e. select a tiny bit of it and return the value of the selected point): to do something tricky. We substitute Eq. 3 into Eq. 7 to get: 1 2π h(t )δ(t t ) dt (7) h(t ) e ıω(t t) dω dt (8) and if we define then we can rewrite Eq. 8 as: H(ω) 1 h(t )e ıωt dt 2π H(ω)e ıωt dω (9) Let s examine these last two equations more carefully - they could define two functions that have a nearly symmetric relationship: Definition 4.4 H(ω) 1 h(t)e ıωt dt 2π Definition 4.5 H(ω)e ıωt dω H(ω) is called the Fourier transform of h(t), and in turn, h(t) is termed the Inverse Fourier Transform of H(ω). Although we have used t and ω here don t forget that we can actually take Fourier transforms of anything e.g. position transforming to spatial frequency. Unfortunately, because life is never simple, you will find in the literature many variations of these definitions (essentially because they contain many arbitrary choices). For example, the factor of 1/2π can be placed before the Fourier transform or before the Inverse Fourier transform in Definitions 4.4-5. Also, one can reverse the signs of the exponent in the two transforms. Finally, one can choose to work in normal frequency rather than angular i.e. use H(f) instead of H(ω) then one can drop the 1 2π all together, and get e.g.: Definition 4.6 H(f) = h(t)e 2πıft dt Definition 4.7 H(ω)e 2πıft df In some ways this last method is the neatest as we have the highest degree of symmetry between the two transforms. However, perhaps unfortunately, the physics and maths literature almost exclusively uses the asymmetric representations. From the Wolfram website (http://mathworld.wolfram.com/fouriertransform.html) we see that generally speaking the Fourier transform pairs can be written as:

Lecture 4: Noise and the Fourier Transform 4-5 b H(f) = 2π 1 a b 2π 1+a h(t)e ıbft dt H(f)e ıbft df So in our original definitions (Definitions 4.4-4.5) we have used a = 1 and b = 1, while the cleaner version given later (Definitions 4.6-7) uses a = and b = 2π. 4.5 One example Let s substitute A sin(2πf t) into Definition 4.6 to see the result: H(f) = A sin(2πf t)e 2πıft dt = ıa 2 = ıa 2 ( e 2iπtf e 2iπtf) e 2πıft dt (1) e 2iπt(f f) e 2iπt(f+f) dt (11) = ıa 2 [δ(f f ) δ(f + f )] (12) This differs from the results shown in Figure 3 and Figure 4 because Brigham has the opposite signs in his transform expressions. 4.6 Useful Rules about the Transforms We note that since an integral is linear, then the transform process is linear as well i.e. the transform of a sum of functions is equal to the sum of the transforms. In addition there are a whole bunch of symmetries between the two domains. In Table 2 below I give you the most useful of them (where we are using H(f) H(ω) ω=2πf : If... h(t) is real h(t) is even h(t) is odd h(t) is real and even h(t) is real and odd then... H( f) = H(f) H( f) = H(f) H( f) = H(f) H(f) is real and even H(f) is imaginary and odd Table 2: some useful relations between the time and frequency domain To prove these, one breaks the complex exponential function into its real and imaginary parts and then uses the known laws of sine and cosine. As one example: if h(t) is real then we can split the complex exponential function into a real cosine part and imaginary sine part (both multiplied by the real h(t)). Since it is well known that cos( θ) = cos(θ) and sin( θ) = sin(θ) we see that real part is the same for H( f) and H(f) while the imaginary part will be inverted i.e. H( f) = H(f). Here are a couple of other properties that you might find useful (I use to indicate transforming): h(at) 1 ( ) f a H time scaling (13) a ( ) 1 t b h H (bf) frequency scaling (14) b h (t t ) H (f) e 2πıft time shifting (15) e 2πıft h (t) H (f f ) frequency shifting (16)

Lecture 4: Noise and the Fourier Transform 4-6 So this is pretty interesting stuff. Perhaps the first two are obvious - there is a reciprocal relationship between the time and frequency scales. The last two are pretty cool - if we want to move an event in time then we just need to rotate the phase of the Fourier components linearly with Fourier frequency. We will come across this again when we consider the effect of a time delay in the transfer function of some device. On the other hand if we want to move the frequency of some component then we see that we need to multiply its time dependence with a sinusoid that has a frequency equal to the frequency shift we desire. This is the basis of modulation/demodulation techniques that we will also study later in this course. The final thing I wish to remind you of is the Fourier transforms of some quite complex operations. We will note one of the great advantages of the Fourier transform - it converts complex calculus into simple algebra. We will use this in a future lecture to solve complex differential equations with ease, as well as to get better insight into transfer functions. d h(t) dt ı2πfh(f) (17) d n dt n h(t) (ı2πf)n H(f) (18) ( ı2πt)h(t) d H(f) (19) dt h(t)dt 1 H(f) (2) ı2πf 1 H(f)df (21) 2ıπt 4.6.1 Examples of Fourier Transforms There are numerous good places on the web to play about with Fourier transforms - try http://13.191.21.21/multimedia/jiracek/dga/spectralanalysis/examples.html or http://www.seismo.unr.edu/htdocs/students/ichinose/fftlab.html and for a heavy mathematical version you can check out http://mathworld.wolfram.com/fouriertransform.html. Below in Figure 3 and Figure 4 (taken from p24-27 of Brigham) you will find some diagrams of classic Fourier transforms relationships. Of course to properly convey all of the information contained in a Fourier Transform you really need 2 graphs ( for each ) of the conjugate variables i.e. R [H(f)] and I [H(f)] or H(f) 2 = H(f)H (f) and arctan so you need to read the captions carefully. I[H(f)] R[H(f)] The example of the Fourier Transform of the pure sine and cosine function is useful: A sin(2πf t) ıa 2 (δ(f + f ) δ(f f )) (22) A cos(2πf t) A 2 (δ(f f ) + δ(f + f )) (23) As mentioned earlier the Dirac Delta function is zero everywhere except where the argument is zero in which case it is infinity. The integral over this special point however yields 1 and thus is defined. We can t really plot the function of the sine as a result of this pathological behaviour but Brigham has done so by putting arrows on the top to indicate that it is heading to infinity at these two special points. Can we instead plot the average power in a finite interval? i.e. let the energy of the delta function spread over a little frequency window (or bin) and thus get a finite value in that frequency interval (see Figure 5). We will return to this a little later but perhaps you should consider what we would need to do in the time domain in order to have an averaging of energy in the frequency domain? P (f n ) = 1 f In this case it is possible to plot the function: fn+ f/2 f n f/2 H(f) 2 df (24)

Lecture 4: Noise and the Fourier Transform 4-7 Frequency Domain t) 5 h(:a lrl<ro =L trl :rn 2 " A sin(2nlof) <) Htft:2ATo l"rrf =o ltl>ro h(t):zafo# @ ntfi:e l/l <fo A =; l/l =ro : l/l >.fo h(t):k @ mjl:ke(f) r,(r)=ke(r) @ ntfl=x Frequency Domain h(t): j b(r-nr) @ ntfl=+ ; I _ a n = -?) h(t) : Acos(2nfor) @ n<n : W - n *!a1y * 1,1 h(t): Asin(2afor) @ nfn: -j+s11 - "fo) *1 6 * 61 N q Figure 2.12 Catalog of Fourier transform pairs. b= Figure 3: Some commonly used Fourier transform pairs: from Brigham 1 ir

Lecture 4: Noise and the Fourier Transform 4-8 Frequency Domain A h(t)=-zkt+a F> Itl < zro : ltl>2to H(fl:2ATo sin2(2rrinf),"tr- h(t1 = I cos(2tfol) @ lrl <?o = ltl>ro a<n: AzTolQff + fo) +Qff-fd) - sin(2nlol) etl\ ztu ha) = + q(t) @ H : +. i""' (f) Af. / l\ *in\'*a/ Af" / l\ +-7slt-rfl t wjcl sin(2of.t) alll = - ' n t lfl= f, = lfl> f, Frequency Domain.. /trr\ h\t)a+;cosl;, \r./ Itl < : To lrl >Io O strr = )oct.iln('.a) * a(r +)l gr1, = ""tzt!'n h@ ::a exp(-ctlrl) O H(f) =?iff ^r,: (;)"' "^p1-o,'1 O ao: *r (#) Figure 2.12 (cont.) Figure 4: Continued; commonly used Fourier transform pairs from Brigham L

Lecture 4: Noise and the Fourier Transform 4-9 1..2 Intensity (a.u).5. -.5 Intensity (a.u).15.1.5-1. -6-4 -2 2 4 6 Time (a.u.). -2-1 1 2 Frequency (a.u) (a) Sampled Sinewave (b) Power Spectrum of Sampled Sinewave Figure 5: