Introduction of Fourier Analysis and Time-frequency Analysis March 1, 2016
Fourier Series Fourier transform Fourier analysis Mathematics compares the most diverse phenomena and discovers the secret analogies that unite them. Joseph Fourier (1768 1830)
Fourier Series Fourier transform Frequency and spectrum Signal model: a signal f (t) R contains components of different frequencies For physical meaning, the term frequency is defined in terms of sinusoidal function cos(ωt) and sin(ωt), where ω is the angular frequency (rad/s) The spectrum shows the intensity and phase of the sinusoidal functions composing f (t) Fourier analysis facilitates this model Fourier analysis is a classic method of retrieving the spectrum of a signal
Fourier Series Fourier transform Fourier series A periodic signal f (t) R can be represented in terms of an infinite sum of sines and cosines. f (t) = a 0 + a n cos(ω n t) + b n sin(ω n t) (1) a 0 = π π f (t)dt, a n = n=1 π π n=1 f (t) cos(ω n t)dt, b n = π π f (t) sin(ω n t)dt The frequencies of these sines and cosines are constructed by a harmonic series, containing a fundamental frequency (i.e., ω 0 ) and its integer multiples (harmonics); ω n = nω 0. These sines and cosines form an orthogonal basis, and the Fourier coefficients are the projection of f (t) on the basis.
Fourier Series Fourier transform Illustration A synthetic viewpoint:
Fourier Series Fourier transform Example: a note C4 played on a piano Fundamental frequency f 0 = 261.6 Hz
Fourier Series Fourier transform Physical interpretation For a bounded vibrating string with length L: Fundamental mode with wavelength λ 0 = L 2 Fundamental frequency f 0 = 2v L 1st harmonic frequency 2f 0 2nd harmonic frequency 3f 0 Pitched musical signal f 0 + higher order harmonics periodic
Fourier Series Fourier transform More examples (a) Piano (b) Trumpet (c) Violin (d) Flute
Fourier Series Fourier transform Phase We can write Equation (1) as f (t) = a 0 + n=1 a 2 n + b 2 n cos(ω n t + φ n ) (2) where φ n = tan 1 b n a n (3)
Fourier Series Fourier transform Phasor, complex number, and negative frequency We can also write Equation (1) as f (t) = a 0 + n=1 c n 2 ejωnt + c n 2 e jωnt (4) where e jωnt = cos(ω n t) + j sin(ω n t), c n = a n + jb n When written in phasor forms, a real-valued signal is decomposed into components with positive and negative frequencies For real-valued signals (e.g., most of the audio signals), negative frequencies are redundant For complex-valued signals, negative frequencies are informative
Fourier Series Fourier transform Fourier transform The Fourier transform is a generalization of the Fourier series, by changing the sum to an integral. A signal f (t) R can be represented as F (ω) = The inverse Fourier transform f (ω) = Denote F (ω) = Ff (t) and f (t) = F 1 F (ω) f (t)e jωt dt, (5) F (ω)e jωt dω. (6)
Fourier Series Fourier transform Example (1): impulse The Dirac delta function f (t) = δ(t) = { 1, t = 0, 0, t 0. (7) F (ω) = δ(t)e jωt dt = e jω 0 = 1. (8)
Fourier Series Fourier transform Example (2): cosine f (t) = cos(ηt) (9) F (ω) = = j cos(ηt)e jωt dt cos(ηt) cos(ωt)dt ( = 1 2 cos(ηt) sin(ωt)dt (= 0) ) for ω = ±η = 1 2 δ(ω η) + 1 δ(ω + η) (10) 2 It is also evident that Fe jηt = δ(ω η)
Fourier Series Fourier transform Example (3): rectangular pulse f (t) = { 1, T 2 < t < T 2, 0, others (11) T 2 F (ω) = e jωt dt = 1 T jω e jωt T 2 T 2 2 = sin T ω ω (12) In a weak sense, lim T F (ω) = δ(ω) lim T 0 f (t) = δ(t)
Fourier Series Fourier transform Example (4): Gaussian f (t) =e at2 F (ω) = = e at2 e jωt dt e at2 cos(ωt)dt j π e at2 sin(ωt)dt = (odd function!) 2a e ω 2 4a. (13)
Fourier Series Fourier transform Example (4): Gaussian (cont d) The Fourier transform of a Gaussian is still a Gaussian f (t) = e t2 2 is an eigenfunction of the Fourier transform We also have lim T F (ω) = δ(ω) and lim T 0 f (t) = δ(t)
Fourier Series Fourier transform Translation and modulation Shifted by t 0 in the time domain multiplied by a phase shift in the frequency domain Modulated by e jω 0t in the time domain shifted by ω 0 in the frequency domaiin f (t t 0 )e jωt dt = f (t)e jω 0t e jωt dt = f (u)e jω(t 0+u) dt = e jωt 0 F (ω)(14) f (t)e j(ω ω 0)t dt = F (ω ω 0 )(15)
Fourier Series Fourier transform Convolution Convolution of f (t) and g(t): f (t) g(t) = f (τ)g(t τ)dτ (16) The convolution theorem: for F (ω) = Ff (t) and G(ω) = Fg(t) then F (f (t) g(t)) = F (ω)g(ω) (17)
Fourier Series Fourier transform Magnitude spectrum, additivity and interference Magnitude spectrum: F (ω) If f (t) = f A (t) + f B (t), then F (ω) = F A (ω) + F B (ω) But F (ω) F A (ω) + F B (ω)! Constructive interference and destructive interference
Sampling Quantization Spectral leakage Sampling Sampling: make a continuous-time signal be discrete-time A discrete-time signal x(n), n Z, sampled from f (t), t R, by a sampling period T x(n) = f (nt ) (18) Sampling frequency f s = 1 T, we have n = tf s Audio signals: f s = 8 khz, 16 khz, 22.05 khz, 44.1 khz, etc. T =? f s =?
Sampling Quantization Spectral leakage Aliasing Limited sampling rate implies limited information of the spectrum Nyquist-Shannon sampling theorem: if the highest frequency of a signal f (t) is f h, then the signal can be perfectly reconstructed by a sampled signal with sampling rate of at least 2f h If sampling rate smaller than 2f h : aliasing
Sampling Quantization Spectral leakage Aliasing effects in image, video and sound Video example and Audio example
Sampling Quantization Spectral leakage Quantization Quantization make a analog-value signal digital Bit depth: number of bits of information in each sample Example: bit depth = 4 Audio signals: 16-bit, 24-bit or more For an audio signal with 2-channel, 16-bit and sampling rate 44.1 khz, what is its bit rate?
Sampling Quantization Spectral leakage Types of signal Can you tell the difference among the following terms? Continuous ( 連續 ) Analog ( 類比 ) Discrete ( 離散 ) Digital ( 數位 )
Sampling Quantization Spectral leakage Spectral leakage We are never able to measure a signal of infinite length Fourier transform on a finite support [T A, T B ]: F (ω) = TB T A f (t)e jωt dt Spectral leakage: the spectrum of a sinusoidal function is never a impulse Heisenberg s uncertainty: better localization in time implies worse localization in frequency
Interpretation of DFT Fast Fourier transform (FFT) DFT For a discrete-time signal (usually sampled from a continuous-time signal) x R N, x = [x(0), x(1),..., x(n 1)] X (k) = N 1 0 x(n)e j2πkn N (19) Let X R N, X = [X (0), X (1),..., X (N 1)] and ω = j2π N 1 1 1 1 1 ω ω 2 ω (N 1) X = Fx, F =....... (20) 1 ω N 1 ω 2(N 1) ω (N 1)(N 1)
Interpretation of DFT Fast Fourier transform (FFT) Interpretation of DFT (1) The discrete Fourier transform approximates the continuous Fourier transform (Riemann sum approximation) ˆx(ω) = n Z x(n)e jωn = f (nt )e jωn n Z f (nt )e jωt dt t R = 1 f (nt )e jωt T dt T t R = 1 ( ω ) T F (21) T
Interpretation of DFT Fast Fourier transform (FFT) Interpretation of DFT (2) Frequency resolution = fs N When N is even: (the distance between two bins) X N 1 X (0) = x(n) f =0 n=0 N 1 N 1 X (1) = x(n)e j2πn N = x(n)e j2π( fs t) N n=0 n=0 f = f s N. ( ) N 1 N 2 1 = x(n)e j2πn ( N 1) N 2 f = f s 2 f s N n=0
Interpretation of DFT Fast Fourier transform (FFT) Interpretation of DFT (3) X ( ) N 1 N X = x(n)e j2πn ( N N 2 ) f = f s 2 2 n=0 ( ) N 1 N 2 + 1 = x(n)e j2πn ( N 1) N 2 f = f s 2 + f s N. n=0 N 1 X (N 1) = x(n)e j2πn ( N 1) N 2 f = f s N n=0
Interpretation of DFT Fast Fourier transform (FFT) Example (1): normal case A chirp signal f (t) := sin(0.003πt 2 ) for t 0
Interpretation of DFT Fast Fourier transform (FFT) Example (2): aliasing A chirp signal f (t) := sin(0.004πt 2 ) for t 0
Interpretation of DFT Fast Fourier transform (FFT) Example (3): approximation property
Interpretation of DFT Fast Fourier transform (FFT) Fast Fourier transform (FFT) Complexity of N-point DFT: O(N 2 ) Cooley-Tukey FFT algorithm: O(N log N) Utilizing the redundancy of the DFT matrix F
Interpretation of DFT Fast Fourier transform (FFT) Algorithm
Non-stationary signals Implementation Non-stationarity of signals Fourier analysis assumes that the amplitude/frequency/phase of a signal do not change over time But this never happens in real world Music is non-stationary: onset, offset, pitch change, etc. f (t) = m A m (t) cos(ω m (t)t + φ m (t)) (22) How to model the temporal dynamics of a non-stationary signal? How to model time and frequency information at the same time?
Non-stationary signals Implementation Example
Non-stationary signals Implementation Dennis Gabor (1900 1979) Nobel Prize in Physics (1971) Known for Holography, Time-frequency analysis, etc. Google Doodle on June 5, 2010
Non-stationary signals Implementation Short-time Fourier transform (STFT): capture local information through a sliding window function h(t) S x (h) (t, ω) = x(τ)h(τ t)e jωτ dτ, (23) Also have magnitude and phase Spectrogram: S (h) x 2
Non-stationary signals Implementation Sliding window and short-time spectrum
Non-stationary signals Implementation Window function For better temporal continuity, better control of spectral leakage and less ripple artifacts (than using rectangular window) ), n = N 2,..., N 2 Hann window: h(t) = 1 2 + 1 2 cos ( 2πn N Comparison of rectangular, triangular and Hann windows:
Non-stationary signals Implementation Illustration (1): window type Hann window (up) and rectangular window (bottom)
Non-stationary signals Implementation Illustration (2): window length Short Hann window (32 ms, up) and long Hann window (128 ms, bottom)
Non-stationary signals Implementation Illustration (3): scaling Spectrogram and db-scaled spectrogram of a scale played with the piano
Non-stationary signals Implementation Implementation of STFT (1) Slice the signal into frames of segments Multiply the short segments by a window function Do FFT for each segment!
Non-stationary signals Implementation Implementation of STFT (2) Discrete STFT: X (n, k) = H: time step (hop size) N 1 m=0 x(m + nh)h(m)e j2πkm N (24) The index k corresponds to the frequency f (k) := kfs N The index n corresponds to the time t(n) := nh f s Example: for an audio signal sampled at f s = 44.1 khz, use window size N = 4096 samples and hop size H = 1024 samples. Could you specify the time-frequency grid in the STFT representation?
Fourier Analysis Non-stationary signals Implementation Example (1): piano solo fs = 44.1 khz, H = 441, N = 4096 (left) and N = 1024 (right) Play sound
Fourier Analysis Non-stationary signals Implementation Example (2): voice fs = 44.1 khz, H = 441, N = 4096 (left) and N = 1024 (right) Play sound
Fourier Analysis Non-stationary signals Implementation Example (3): rock fs = 44.1 khz, H = 441, N = 4096 (left) and N = 1024 (right) Play sound
Fourier Analysis Non-stationary signals Implementation Example (4): symphony fs = 44.1 khz, H = 441, N = 4096 (left) and N = 1024 (right) Play sound