Fourier Analysis of Stochastic Processes

Transcription

1 Fourier Analysis of Stochastic Processes. Time series Given a discrete time rocess ( n ) nz, with n :! R or n :! C 8n Z, we de ne time series a realization of the rocess, that is to say a series (x n ) nz of real or comlex numbers where x n = n (!)8n Z. In some cases we will use the notation x(n) instead of x n, with the same meaning. For convenience, we introduce the sace l of the time series such that P nz jx nj <. The nite value of the sum can be sometimes interreted as a form of the energy, and the time series belonging to l are called nite energy time series. Another imortant set is the sace l of the time series such that P nz jx nj <. Notice that the assumtion P nz jx nj < imlies P nz jx nj <, because P nz jx nj su nz jx n j P nz jx nj and su nz jx n j is bounded when P nz jx nj converges. Given two time series f(n) and g(n), we de ne the convolution of the two time series as h(n) = (f g)(n) = kz f (n k) g (k). Discrete time Fourier transform Given the realization of a rocess (x n ) nz l, we introduce the discrete time Fourier transform (DTFT), indicated either by the notation bx (!) or F [x] (!) and de ned by bx (!) = F [x] (!) = e i!n x n ;! [; ] : nz Note that the symbol b is used to indicate both DTFT and emirical estimates of rocess arameters; the variable! is used to indicate both the indeendent variable of the DTFT and the outcomes (elementary events) in a set of events. Which of the two meanings is the correct one will be always evident in both cases. The sequence x n can be reconstructed from its DTFT by means of the inverse Fourier transform x n = Z e i!n bx (!) d!: In fact, Z assuming that it is allowed to interchange the in nite summation with the integration, we have e i!n bx (!) d! = Z e i!n x k e i!k d! = Z x k e i!(n k) d! = x k (n k)=x n kz kz kz Remark. In the de nition using i!n as exonent, the increment of the indeendent variable between two consecutive samles is assumed to be. If the indeendent variable is time t and the i

2 ii FOURIER ANALYSIS OF STOCHASTIC PROCESSES time interval between two consecutive samles is t, in order to change the time scale, the exonent becomes i!nt. The quantity t is called samling frequency. Remark. The function bx (!) can be considered for all! R, but it is -eriodic or t -eriodic. In the last case,! is called angular frequency. Remark. Sometimes (in articular by hysicists), DTFT is de ned without the exonent; in this case the sign is obviously resent in the inverse transform. sign at the Remark 4. Sometimes the factor is not included in the de nition; in our de nition the factor is included for symmetry with the inverse transform or the Plancherel formula (without, a factor aears in one of them). Remark 5. Sometimes, it is referable to use the variant bx (f) = e ifn x n ; f [; ] : where f =!. When the factor t is resent in the exonent, f ; nz is called frequency. t In the following, we will use our de nition of DTFT, indeendently of the fact that in certain alications it is customary or convenient to make other choices. The reader is warned!! Of course, in ractical cases, in nite samling times do not exist. Let us therefore introduce a samling window of width ( N contaning N + samles from time N to time N (also called boxcar if N n N window) [ N;N] (n) = and the truncation x N (n) of the time series x n de ned otherwise ( x n if N n N as x N (n) = x n [ N;N] (n) =. The DTFT of the truncated time series is otherwise de ned as bx N (!) = e i!n x n : Remark 6. In order to de ne roerly the DTFT for a rocess, we must assure that it is ossible to calculate the DTFT of every of its realization. Therefore the above de nition, involving only a nite summation, can be used also to de ne the truncated DTFT of a rocess: b N (!) = e i!n n : In fact, each realization bx N of a truncated rocess b N is a nite-energy time series that admits DTFT. We underline that b N (!) is a random variable for every! [; ], while bx N (!) is a function of!. The de nition of truncated DTFT for a rocess will be used in the roof of Wiener- Khinchine theorem. Processes whose realizations are l or l time series are never stationary.

3 . DISCRETE TIME FOURIER TRANSFORM iii Proerties of DTFT ) The L -theory of Fourier series guarantees that the series P nz e i!n x n converges in mean square with resect to!, namely, there exists a square integrable function bx (!) such that ) Plancherel formula Proof. Z n;mz jx n j assuming that it is correct to interchange the in nite summation with the inte- x n x n = nz nz gration. jbx (!)j d! = lim N! Z Z jx n j = nz x n x me i!n e i!m A d! = jbx N (!) bx (!)j d! = : Z bx (!) (bx (!)) d! = n;mz x n x m jbx (!)j d! Z Z! e i!n x n nz e i!(n m) d! = n;mz! e i!m x m d! = mz x n x m(n m) = The meaning of Plancherel formula is that the energy contained in a time series and the energy contained in its DTFT is the same (a factor can aear in one of the two members when a di ernt detinition of DTFT is used). ) If the time series g(n) is real, then bg (!) = bg (!).! Proof. bg (!)= g(n)e i(!)n = g(n) e i!n = g(n)e i!n =g (!) nz 4) The DTFT of the convolution of two time series corresonds (a factor can be resent,deending on the de nition of DTFT) to the roduct of the DTFT of the two time series Proof. F[f g] (!) = nz F [f g] (!) = b f (!) bg (!) : (f g)(n)e i!n = nz f(n k)e i!(n k) = nz nz f(n kz k)g(k)! e i!n = g(k)e i!k b f(!) = g(k)e i!k g(k)e i!k f(m)e i!m = kz nz kz mz kz f(!)bg(!), b assuming that it is correct to commute the two in nite summations. The is not resent when the de nition of DTFT without is used. " 6) Combining roerties # 4) and " 5), we obtain # F f (n + k) g (k) (!) = F f (n k) g ( k) (!) = f b (!) bg (!) = f b (!) bg (!), where kz kz

4 iv FOURIER ANALYSIS OF STOCHASTIC PROCESSES the last equality is valid only for a real time series g. This roerty is used in the calculation of DTFT of correlation function. Again, the resence of factor deends on the de nition of DTFT adoted. 7) When jx n j <, then the series P nz e i!n x n is absolutely convergent, uniformly in! nz [; ], simly because su nz![;] e i!n x n = nz su![;] e i!n jx n j = nz jx n j < : In this case, we may also say that bx (!) is a bounded continuous function, not only square integrable.. Generalized discrete time Fourier transform One can do the DTFT also for sequences which do not satisfy the assumtion P nz jx nj <, in secial cases. Consider for instance the sequence Comute the DFTT of the truncated sequence Recall that Hence sin (! n) = ei! n e i! n i x n = a sin (! n) : bx N (!) = e i!n a sin (! n) = i sin t = eit e i!n a sin (! n) : e it : i e i(!! )n i e i(!+! )n : The next lemma makes use of the concet of generalized function or distribution, which is outside the scoe of these notes. We still given the result, to be understood in some intuitive sense. We use the generalized function (t) called Dirac delta (not to be confused with the dircrete time de ned in examle for white noise), which is characterized by the roerty (.) Z (t t ) f (t) dt = f (t ) for all continuous comact suort functions f. No usual function has this roerty. A way to get intuition is the following one. Consider a function n (t) which is equal to zero for t outside n ; n, interval of length n around the origin; and equal to n in ; n. Hence n (t t ) is equal to zero for t outside t n ; t + n, equal to n in t n ; t + n. We have Z n (t) dt = :

5 . GENERALIZED DISCRETE TIME FOURIER TRANSFORM v Now, Z Z t + n n (t t ) f (t) dt = n f (t) dt t n which is the average of f around t. As n!, this average converges to f (t ) when f is continuous. Namely. we have Z lim n (t t ) f (t) dt = f (t ) n! which is the analog of identity (.), but exressed by means of traditional concets. In a sense, thus, the generalized function (t) is the limit of the traditional functions n (t). But we see that n (t) converges to zero for all t 6=, and to for t =. So, in a sense, (t) is equal to zero for t 6=, and to for t = ; but this is a very oor information, because it does not allow to deduce identity (.) (the way n (t) goes to in nity is essential, not only the fact that (t) is for t = ). Lemma. Denote by (t) the generalized function such that Z (t t ) f (t) dt = f (t ) for all continuous comact suort functions f (it is called the delta Dirac distribution). Then lim e itn = (t) : N! From this lemma it follows that lim e i!n a sin (! n) = i (!! ) In other words, N! Corollary. The sequence has a generalized DTFT bx (!) = lim N! bx N (!) = x n = a sin (! n) i (! +! ) : i ( (!! ) (! +! )) : This is only one examle of the ossibility to extend the de nition and meaning of DTFT outside the assumtion P nz jx nj <. It is also very interesting for the interretation of the concet of DTFT. If the signal x n has a eriodic comonent (notice that DTFT is linear) with angular frequency!, then its DTFT has two symmetric eaks (delta Dirac comonents) at!. This way, the DTFT reveals the eriodic comonents of the signal. Exercise. Prove that the sequence has a generalized DTFT bx (!) = lim N! bx N (!) = x n = a cos (! n) ( (!! ) + (! +! )) :

6 vi FOURIER ANALYSIS OF STOCHASTIC PROCESSES 4. Power sectral density Given a stationary rocess ( n ) nz with correlation function R (n) = E [ n ], n Z, we call ower sectral density (PSD) the function S (!) = e i!n R (n) ;! [; ] : Alternatively, one can use the exression S (f) = e ifn R (n) ; f [; ] nz nz which roduces easier visualizations because we catch more easily the fractions of the interval [; ]. Remark 7. In rincile, to de ne roerly the PSD, the condition P nz jr (n)j < should be required, or at least P nz jr (n)j <. In ractice, on a side the convergence may haen also in unexected cases due to cancellations, on the other side it may be accetable to use a nite-time variant, something like P e i!n R (n), for ractical uroses or from the comutational viewoint. A riori, one may think that S (f) may be not real valued. However, the function R (n) is nonnegative de nite (this means P n i= R (t i t j ) a i a j for all t ; :::; t n and a ; :::; a n ) and a theorem states that the Fourier transform of non-negative de nite function is a non-negative function. Thus, at the end, it turns out that S (f) is real and also non-negative. We do not give the details of this fact here because it will be a consequence of the fundamental theorem below. hence 4.. Examle: white noise. We have S (!) = R (n) = (n) ;! R: The sectra density is constant. This is the origin of the name, white noise. 4.. Examle: erturbed eriodic time series. This examle is numeric only. Produce with R software the following time series: t <- : y<- sin(t/)+.*rnorm() ts.lot(y)

7 The emirical autocorrelation function, obtained by acf(y), is 4. POWER SPECTRAL DENSITY vii and the ower sectral density, suitable smoothed, obtained by sectrum(y,san=c(,)), is 4.. Pink, Brown, Blue, Violet noise. In certain alications one meets PSD of secial tye which have been given names similarly to white noise. Recall that white noise has a constant PSD. Pink noise has PSD of the form Brown noise: S (f) f : Blue noise S (f) f : S (f) f : Violet noise S (f) f : if f where, chosen such that < <, (f) = otherwise

8 viii FOURIER ANALYSIS OF STOCHASTIC PROCESSES 5. A fundamental theorem on PSD The following theorem is often stated without assumtions in the alied literature. One of the reasons is that it can be roved under various level of generality, with di erent meanings of the limit oeration (it is a limit of functions). We shall give a rigorous statement under a very recise assumtion on the autocorrelation function R (n); the convergence we rove is rather strong. The assumtion (a little bit strange, but satis ed in all our examles) is (5.) R (n) < for some (; ) : nn This is just a little bit more restrictive than the condition P nn jr (n)j < which is natural to imose if we want uniform convergence of PnZ e i!n R (n) to S (!). Theorem (Wiener-Khinchin). If ( (n)) nz is a wide-sense stationary rocess satisfying assumtion (5.), then S (!) = lim N! N + E N b (!) : The limit is uniform for! [; ]. Here N is the truncated rocess [ N;N]. In articular, it follows that S (!) is real an non-negative. Proof. Ste. Let us rove the following main identity: (5.) S (!) = where the remainder r N is given by r N (!) = N + F N + E 4 b N (!) + r N (!) n(n;t) E [ (t + n) (n)] 5 (!) and (N; t) is a subinterval of [ N; N] to be de ned later. Since R (t) = E [ (t + n) (n)] for all n, we obviously have, for every t, R (t) = E [ (t + n) (n)] : N + Thus (5.) S (!)= b R (!)= N + F 4 E [ (t + n) (n)] 5(!)= N + [E [ (t + n) (n)]] e i!t tz To invert the in nite sum over t with the calculation of mean value, we need to introduce truncated rocess N (n) = (n) [ N;N] (n), for which we have, for t N N (t + n) N (n) = nz N t (t + n) (n) ; n= N

9 5. A FUNDAMENTAL THEOREM ON PSD ix for N t < ve have N N (t + n) N (n) = (t + n) (n) ; nz n= N t for t > N or t < with N, we have N (t + n) N (n) = 8n. In general, N (t + n) N (n) = (t + n) (n) : nz n[n t ;N t + ] 8 ; if t < N >< [N t ; N t + ] = [ N t; N] if N t < [ N; N t] if t N >: ; if t > N Therefore, remembering roerty 6) of DTFT, N + " # t F 4 (t + n) (n) 5 (!) = F N (t + n) N (n) (!) = b N (!) b N (!) = b N (!) : n=n t nz Taking the mean value of rst and last term, we obtain N t + N t + F 4 E [ (t + n) (n)] 5 (!) = E 4F 4 (t + n) (n) 5 (!) 5 = E N b (!) : n=n t n=n t Now, it is ossible to decomose the sum running from N to +N into a sum running from N t to N t + and a remainder r N containing the other terms, from N to N t or from N t + to N. S(!) = N + F 4 N E [ (t + n) (n)] 5 t + (!) = N + F 4 E [ (t + n) (n)] 5 (!) + n=n t N + F 4 E [ (t + n) (n)] 5 (!) = N + E N b (!) + r N (!) ; with n(n;t) 8 >< (N; t) = >: [ N; N] if t < N [ N; N t ] if N t < ; if t = [N t + ; N] if < t N [ N; N] if t > N

10 x FOURIER ANALYSIS OF STOCHASTIC PROCESSES Ste. The roof is comlete if we show that lim N! r N (!) = uniformly in! [; ]. Let " n = R (n). As P nn R (n) <, we have R (n)!, and therefore also " n!. Moreover, R (n) = R (n) < : " n nn We can write n(n;t) nn E [ (t + n) (n)] = n(n;t) R (t) = " jtj R (t) " jtj j (N; t)j where j (N; t)j denotes the cardinality of (N; t). If (N + )^jtj denotes the smallest value between (N + ) and jtj, we have j (N; t)j = (N + ) ^ jtj hence N + E [ (t + n) (n)] jr (t)j ((N + ) ^ jtj) " jtj = : " jtj N + n(n;t) t N+ Given >, let t be such that " jtj for all t t. Then take N t such that N N. It is not restrictive to assume " jtj for all t. Then, for N N, if t t then ((N + ) ^ jtj) " jtj N + t " jtj N + t N + and if t t then ((N + ) ^ jtj) " jtj ((N + ) ^ jtj) : N + N + We have roved the following statement: for all > there exists N such that ((N + ) ^ jtj) " jtj N + for all N N, uniformly in t. Then also N + E [ (t + n) (n)] n(n;t) jr (t)j " jtj for all N N, uniformly in t. Therefore jr N (!)j = e i!t 4 E [ (t + n) (n)] 5 N + tz n(n;t) N + E [ (t + n) (n)] jr (t)j = C " jtj jr(t)j tz n(n;t) tz for all where C = P tz " jtj <. This is the de nition of lim N! r N (!) = uniformly in! [; ]. The roof is comlete.

11 5. A FUNDAMENTAL THEOREM ON PSD xi This theorem gives us the interretation of PSD. The Fourier transform b T (!) identi es the frequency structure of the signal. The square b T (!) dros the information about the hase and kees the information about the amlitude, but in the sense of energy (a square). It gives us the energy sectrum, in a sense. So the PSD is the average amlitude of the oscillatory comonent at frequency f =!. Thus PSD is a very useful tool if you want to identify oscillatory signals in your time series data and want to know their amlitude. By PSD, one can get a "feel" of data at an early stage of time series analysis. PSD tells us at which frequency ranges variations are strong. Remark 8. A riori one could think that it were more natural to comute the Fourier transform b (!) = P nz ei!n n without a cut-o of size N. But the rocess ( n ) is stationary. Therefore, it does not satisfy the assumtion P nz n < or similar ones which require a decay at in nity. Stationarity is in contradiction with a decay at in nity (it can be roved, but we leave it at the obvious intuitive level). Remark 9. Under more assumtions (in articular a strong ergodicity one) it is ossible to rove that S (!) = lim N! N + b N (!) without exectation. Notice that N+ b N (!) is a random quantity, but the limit is deterministic.