Sound Modeling: signal-based approaches (part 2)

Transcription

1 Sound Modeling: signal-based approaches (part 2) Sound Analysis, Synthesis and Processing Paolo Bestagini

2 Summary What does signal based approach mean? Models representing the sound without referring to the mechanism used for its generation Possible approaches: Time-segment based models Spectral models Source-filter models Subtractive synthesis Speech modeling Non-linear models Non-linear distortion Modulations

3 Source-filter models

4 Source-filter Model A spectrally rich excitation signal shaped in the spectrum by linear system (filter) that acts as a resonator In computer music, source-filter models are traditionally grouped under the label subtractive synthesis often used in an analysis-synthesis framework: the source signal and the filter parameters are estimated from a target sound signal, that can be subsequently resynthesized through the identified model

5 Digression: z-transform Z-transform and filters: Time domain Z domain Examples:

6 Source-filter Model Filter block to be linear and time-invariant defined as: x[n] : excitation signal in the z-domain Generic form of a filter Features of the source and of the filter are combined: the spectral fine structure of the excitation signal is multiplied by the spectral envelope of the filter, which has a shaping effect on the source spectrum Source Signal Rich spectrum that extends to a relevant portion of the audible frequency range: Noise Signal Non-smooth periodic waveforms, whose spectral energy is concentrated in a (large) set of discrete spectral lines (square waves, sawtooth waves, triangle waves)

7 Source-filter Model Square Waveform Square Wave: An ideal square wave alternates periodically and instantaneously between two levels where

8 Source-filter Model Square Waveform Triangular Wave An ideal triangular wave alternates periodically between a linearly rising portion and a linearly decreasing portion

9 Source-filter Model Square Waveform Sawtooth Wave An ideal sawtooth wave is a periodic series of linear ramps where is the floor function.

10 Source-filter Model Source signals Impulse Train A sequence of unit impulses spaced by the desired fundamental period Stochastic noise Another simple generator for stochastic sources is the random noise generator, which produces a flat spectrum noise (white noise, pink noise)

11 Source-filter Model - Filters

12 Source-filter Model Filters Resonant Filter l The second-order IIR filter is the simplest one, and is described by a transfer function l where r and ±ω c are the magnitude and phases of the poles, and the condition r < 1 must hold in order for the filter to be stable

13 Source-filter Model Speech Modeling Speech is an acoustic pressure wave created when air is expelled from the lungs through the trachea and vocal tract Vocal tract: throat, nose, mouth and lips As the acoustic wave passes through the vocal tract, its spectrum is altered by the resonances of the vocal tract (the formants)

14 Source-filter Model Speech Modeling Voiced sounds (vowels or nasals): result from a quasi-periodic excitation of the vocal tract caused by oscillation of the vocal folds in a quasi-periodic fashion Unvoiced sounds: do not involve vocal fold oscillations and are typically associated to turbulent flow generated when air passes through narrow restrictions of the vocal tract During voiced signals: the quasi-periodic nature of the oscillations gives rise to an harmonic signal the frequency associated with the first harmonic partial is commonly termed the pitch of the voiced signal

15 Source-filter Model Speech Modeling Concatenative synthesis Connect pre-recorded natural phonetic units Pros: Easiest way and the most popular approach to produce intelligible and natural sounding synthetic speech Cons: Are usually limited to one speaker and one voice and usually require much memory Not flexible Formant Synthesis Formant synthesis is based on the source-filter modeling approach Source: acoustic flow Filter: vocal tract The transfer function of the vocal tract is typically represented as a series of resonant filters, each accounting for one formant Articulatory synthesis Model the human speech production system directly Parameters associated to vocal folds: glottal aperture, fold tension, lung pressure, etc. Pros: promise high quality synthesis Cons: computational costs are high, parametric control is arduous

16 Source-filter Model Speech Modeling

17 Source-filter Model Speech Modeling

18 Source-filter Model Speech Modeling Formant Synthesis Formant synthesis of speech realizes a source-filter model: a broadband source signal undergoes multiple filtering transformations that are associated to the action of different elements of the phonatory system If s[n] is a voiced speech signal, it can be expressed in the z-domain as: the source signal X(z) is a periodic pulse train whose period coincides with the pitch of the signal and g v is a constant gain term G(z) is a filter associated to the response of the glottis (the vocal folds) to pitch pulses V (z) is the vocal tract filter R(z) simulates the radiation effect of the lips If s[n] is a unvoiced speech signal The turbulence can be modeled as white noise, so X(z) is a white noise sequence

19 Source-filter Model Speech Modeling Formant Synthesis G(z) shapes the glottal pulses since the input x[n] is a pulse train, the output is the impulse response g[n] of this filter A model is a IIR low pass filter

20 Source-filter Model Speech Modeling Formant Synthesis R(z) is a load that converts the airflow signal at the lips into an outgoing pressure wave can be approximated by a differentiator (high pass) filter where ρ is a lip radiation coefficient The vocal tract filter V(z) models vocal tract formants a single formant can be modeled with a two-pole resonator V i (z) the filter associated to the i-th formant is V i (z), having center frequency f i and bandwidth B i at least 3 formants are needed (5 for high quality)

21 Source-filter Model Speech Modeling Formant Synthesis Two possible structure which are used combined (cascade and parallel) A cascade formant synthesizer consists of band-pass resonators connected in series A parallel formant synthesizer consists of resonators connected in parallel, i.e. the same input is applied to each formant filter and the outputs are summed A cascade model of the vocal tract is considered to provide good quality in the synthesis of vowels, but is less flexible than a parallel structure, which enables controlling of bandwidth and gain for each formant individually

22 Source-filter Model Speech Modeling Linear Prediction It s possible to use the analysis-synthesis technique The problem is to extract a spectral envelope from a signal spectrum Linear prediction estimates an all-pole filter that matches the spectral content of a sound. When the order of this filter is low, only the formants are taken, hence the spectral envelope

23 Source-filter Model Speech Modeling Example: The frequencies of the source and the frequencies of the filter are independent This is why it is sometimes difficult to understand the vowels of a soprano singing at the top of her range.

24 Non linear models

25 Non linear Models The transformations seen until now are linear (a): frequency does not change Using non linear (b) transformations: frequencies can be drastically changed new components are created It is possible to vary substantially the nature of the input sound

26 Non linear Models Two main effects: Spectrum enrichment: due to non linear distortion allows for controlling the brightness of a sound nonlinearities and saturations found on real systems e.g. analog amplifiers, electronic valves Spectrum shift: due to multiplication of the signal by a sinusoid moves the spectrum, altering the harmonic relationship between the modulating signal lines used in electronic music and it is a new metaphor for computer musicians

27 Non linear Models Non linear distortion (Waveshaping) A sinusoidal input x[n] = A cos(ω 0 n) which passes through a LTI system (Linear Time Invariant) produces an output signal y[n] which is still a sinusoid with the same frequency ω 0 and amplitude and phase modified according to the transfer function values If the signal is processed through a non-linear system, more substantial modifications of the spectrum occur Typical output: y[n] = Total Harmonic Distortion: NX A k cos(kω 0 n), k=0 X v u THD = N tp k=2 A2 k P N k=1 A2 k.

28 Non linear Models Non linear distortion (Waveshaping) We define the distortion block as a non-linear memory-less system: With a sinusoidal signal as input: N y[n]= F ( A cos(ω 0 n))= k = 0 A k cos( k ω 0 n) Harmonic distortion If we consider F(X[n]) as a polynomial with degree N (Taylor expansion) the first N harmonics

29 Non linear Models Non linear distortion (Waveshaping) Overdrive and distortion guitar effects Analog guitar effects, based either on vacuum tubes (valves like diods, triods, pentods) or solid-state devices, provide a good example of nonlinear processing Overdrive: refers to a nearly linear audio effect device which can be driven into the non-linear region of its distortion curve only by high input levels. The transition from the operating linear region to the nonlinear region is smooth. Distortion: refers to a similar effect, with the difference that the device operates mainly in the non-linear region of the distortion curve

30 Non linear Models Non linear distortion (Waveshaping) Symmetric distortion is based on static non-linearities that are odd with respect to the origin, are approximately linear for low input values q in the second equation controls the amount of clipping (higher values provide faster saturation) Asymmetric overdrive effects are based on distortion curves that clip positive and negative input values in different ways The parameter q scales the range of linear behavior (more negative values increase the linear region of operation) and d controls the smoothness of the transition to clipping (higher values provide stronger distortions)

31 Non linear Models Multiplicative Synthesis It is most simple technique for spectrum shift and in analog domain it s called Ring Modulation (RM) Let x 1 [n] and x 2 [n] be two input signals s[n]= x 1 [n] x 2 [n] The spectrum is convolution S(ω d )= [X 1 X 2 ](ω d ) Carrier Signal c[n] is a sinusoid with frequency ω c Modulation Signal the second signal is the input that will be transformed by the ring modulation and is called the modulating signal m[n] x 1 [n]= c 1 [n]= cos(ω c n+ ϕ c ) The Spectrum is x 2 [n]= m[n] i.e. S(ω d ) is composed of two copies of the spectrum of M(ω d ), symmetric around ω c : a lower side- band (LSB), reversed in frequency, and an upper sideband (USB)

32 l l Non linear Models Multiplicative Synthesis If we consider fundamental harmonic partials in this case multiplicative synthesis causes every spectral line kω m to be replaced by two spectral lines, one in the LSB and the other one in the USB, with frequencies ω c kω m and ω c + kω m l l l l The resulting spectrum has partials at frequencies ω c ± kω m with k = 1,..., N Spectra of this kind can be characterized through the ratio ω c /ω m When this ratio is rational (i.e. ω c /ω m = N1/N2 with N1, N2 N and mutually prime) the sound is periodic. When this ratio is irrational the sound is inharmonic

33 l Non linear Models Multiplicative Synthesis Amplitude Modulation Of particular interest is the case of an ω c /ω m ratio approximating a simple rational value l In this case the fundamental frequency is still ω 0 = ω m /N2, but partials are shifted from the harmonic series by ±εω m, so that the spectrum becomes slightly inharmonic. Amplitude Modulation l where α is the amplitude modulation index. In this case the spectrum S(ω d ) contains also the carrier spectral line, plus side-bands of the form. From the expression for S(ω d ) one can see that α controls the amplitude of the sidebands

34 Non linear Models Frequency Modulation l Frequency modulation l They are not derived from models of sound signals or sound production, and are instead based on abstract mathematical descriptions l Pros l versatile methods for producing many types of sounds l great timbral variability l very limited number of control parameters l low computational costs l Cons l It can t be used analysis-synthesis scheme in which parameters of the synthesis model are derived from analysis of real sounds. No intuitive interpretation can be given to the parameter choice

35 Non linear Models - Modulation Synthesis by Frequency modulation (FM) l The definition of synthesis by frequency modulation (FM) encompasses an entire family of techniques in which the instantaneous frequency of a periodic signal (carrier) is itself a signal that varies at audio rate (modulating). l The general formulation of FM is: where l a[n] is the amplitude signal, l ω c [n] is the carrier frequency, l Φ[n] is the modulating signal.

36 Non linear Models - Modulation Basic FM Scheme l It s used a sinusoidal modulating signal φ[n] with amplitude I[n] (called modulation index) and frequency ω m [n] where both I[n] and ω m [n] vary at frame rate l This modulation produce the signal where J k (I[n]) is the k-th order Bessel function of first kind, computed in I[n]

37 Non linear Models - Modulation Basic FM Scheme l we can see that the resulting spectrum is composed of partials at frequencies ω c ± kω m, each with amplitude J k (I) l Note that an infinite number of partials is generated, so that the signal bandwidth is not limited. In practice however only a few low-order Bessel functions take significantly non-null values for small values of I l As I increases, the number of significantly non-null Bessel functions increases too. So we can control the bandwidth around ω c l we can control inharmonic factor through the ratio ω c / ω m

38 Non linear Models - Modulation Compound modulation l If the modulating signal is composed of two sinusoids l s[n] possesses the partials with frequencies ω c ± k 1 ω 1 ± k 2 ω 2 with amplitudes given by l Simplification: consider ω 1 >ω 2 and consider only the sinusoid with ω 1. We obtain partials with frequencies ω c ± k 1 ω 1. Adding the second sinusoid, each partial of the first one become a carry for the second one l If ω c is the greatest common divider for ω 1 and ω 2 then the spectrum is ω c ± kω m similar to the basic case, but with a more rich spectrum l Otherwise we produce inharmonic components

39 Non linear Models - Modulation Compound modulation general case l If the modulating signal is composed of N sinusoids l s[n] possesses all the partials with frequencies ω c ± k 1 ω m,1 ± ± k N ω m,n with amplitudes given by the product of N Bessel functions

40 Non linear Models - Modulation Compound modulation general case

41 Non linear Models - Modulation Nested modulation l A sinusoidal modulator is itself modulated by a second one l The result can be interpreted as if each partial produced by the modulating frequency ω m,1 were modulated by ω m,2 with modulation index ki 2 l The spectral structure is similar to that produced by two sinusoidal modulators, but with larger bandwidth

42 Non linear Models - Modulation Nested modulation

43 Non linear Models - Modulation Feedback modulation l Past values of the output signal are used as a modulating signal l With n 0 =1 l and β (called the feedback factor) acts as a scale factor or feedback modulation index. l For increasing values of β the resulting signal is periodic of frequency ω c and changes smoothly from a sinusoid to a sawtooth waveform. Moreover one may vary the delay n 0 in the feedback, and observe emergence of chaotic behaviors for suitable combinations of the parameters n 0 and β.