Digital Speech Processing Lectures 7-8. Time Domain Methods in Speech Processing



Similar documents
Sampling and Interpolation. Yao Wang Polytechnic University, Brooklyn, NY11201

L9: Cepstral analysis

Lecture 1-6: Noise and Filters

Probability and Random Variables. Generation of random variables (r.v.)

Design of Efficient Digital Interpolation Filters for Integer Upsampling. Daniel B. Turek

Department of Electrical and Computer Engineering Ben-Gurion University of the Negev. LAB 1 - Introduction to USRP

Analog and Digital Signals, Time and Frequency Representation of Signals

Design of FIR Filters

Short-time FFT, Multi-taper analysis & Filtering in SPM12

SWISS ARMY KNIFE INDICATOR John F. Ehlers

Controller Design in Frequency Domain

Time series analysis Matlab tutorial. Joachim Gross

Analysis/resynthesis with the short time Fourier transform

TTT4110 Information and Signal Theory Solution to exam

Frequency Response of FIR Filters

Auto-Tuning Using Fourier Coefficients

TTT4120 Digital Signal Processing Suggested Solution to Exam Fall 2008

Implementation of Digital Signal Processing: Some Background on GFSK Modulation

Appendix D Digital Modulation and GMSK

Lecture 1-10: Spectrograms

ANALYZER BASICS WHAT IS AN FFT SPECTRUM ANALYZER? 2-1

PHYS 331: Junior Physics Laboratory I Notes on Noise Reduction

TCOM 370 NOTES 99-4 BANDWIDTH, FREQUENCY RESPONSE, AND CAPACITY OF COMMUNICATION LINKS

FFT Algorithms. Chapter 6. Contents 6.1

Spike-Based Sensing and Processing: What are spikes good for? John G. Harris Electrical and Computer Engineering Dept

QAM Demodulation. Performance Conclusion. o o o o o. (Nyquist shaping, Clock & Carrier Recovery, AGC, Adaptive Equaliser) o o. Wireless Communications

T = 1 f. Phase. Measure of relative position in time within a single period of a signal For a periodic signal f(t), phase is fractional part t p

Frequency Response of Filters

CBS RECORDS PROFESSIONAL SERIES CBS RECORDS CD-1 STANDARD TEST DISC

(Refer Slide Time: 01:11-01:27)

A Sound Analysis and Synthesis System for Generating an Instrumental Piri Song

Broadband Networks. Prof. Dr. Abhay Karandikar. Electrical Engineering Department. Indian Institute of Technology, Bombay. Lecture - 29.

Filter Comparison. Match #1: Analog vs. Digital Filters

Thirukkural - A Text-to-Speech Synthesis System

The front end of the receiver performs the frequency translation, channel selection and amplification of the signal.

PHASE ESTIMATION ALGORITHM FOR FREQUENCY HOPPED BINARY PSK AND DPSK WAVEFORMS WITH SMALL NUMBER OF REFERENCE SYMBOLS

Understanding Poles and Zeros

Solutions to Exam in Speech Signal Processing EN2300

Digital Speech Coding

Linear Predictive Coding

SGN-1158 Introduction to Signal Processing Test. Solutions

Using the Impedance Method

Lock - in Amplifier and Applications

3. Regression & Exponential Smoothing

CIRCUITS LABORATORY EXPERIMENT 3. AC Circuit Analysis

Implementation of the LMS Algorithm for Noise Cancellation on Speech Using the ARM LPC2378 Processor.

1995 Mixed-Signal Products SLAA013

Electronic Communications Committee (ECC) within the European Conference of Postal and Telecommunications Administrations (CEPT)

Basic Acoustics and Acoustic Filters

Enhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm

CHAPTER 6 Frequency Response, Bode Plots, and Resonance

Analog Filters. A common instrumentation filter application is the attenuation of high frequencies to avoid frequency aliasing in the sampled data.

S. Boyd EE102. Lecture 1 Signals. notation and meaning. common signals. size of a signal. qualitative properties of signals.

Trigonometric functions and sound

Speech Signal Processing: An Overview

Computer Networks and Internets, 5e Chapter 6 Information Sources and Signals. Introduction

Optical Fibres. Introduction. Safety precautions. For your safety. For the safety of the apparatus

Introduction to IQ-demodulation of RF-data

Chapter 8 - Power Density Spectrum

14: FM Radio Receiver

Workshop Perceptual Effects of Filtering and Masking Introduction to Filtering and Masking

Signal Detection C H A P T E R SIGNAL DETECTION AS HYPOTHESIS TESTING

B3. Short Time Fourier Transform (STFT)

ELEN E4810: Digital Signal Processing Topic 8: Filter Design: IIR

LABORATORY 10 TIME AVERAGES, RMS VALUES AND THE BRIDGE RECTIFIER. Bridge Rectifier

HYBRID FIR-IIR FILTERS By John F. Ehlers

Impedance 50 (75 connectors via adapters)

T = 10-' s. p(t)= ( (t-nt), T= 3. n=-oo. Figure P16.2

Lecture 5: Variants of the LMS algorithm

RANDOM VIBRATION AN OVERVIEW by Barry Controls, Hopkinton, MA

POWER SYSTEM HARMONICS. A Reference Guide to Causes, Effects and Corrective Measures AN ALLEN-BRADLEY SERIES OF ISSUES AND ANSWERS

Voice---is analog in character and moves in the form of waves. 3-important wave-characteristics:

DESIGN AND SIMULATION OF TWO CHANNEL QMF FILTER BANK FOR ALMOST PERFECT RECONSTRUCTION

PCM Encoding and Decoding:

The Algorithms of Speech Recognition, Programming and Simulating in MATLAB

Current-Transformer Phase-Shift Compensation and Calibration

Digital Signal Processing IIR Filter Design via Impulse Invariance

Sampling Theorem Notes. Recall: That a time sampled signal is like taking a snap shot or picture of signal periodically.

Lecture 9. Poles, Zeros & Filters (Lathi 4.10) Effects of Poles & Zeros on Frequency Response (1) Effects of Poles & Zeros on Frequency Response (3)

Em bedded DSP : I ntroduction to Digital Filters

Luigi Piroddi Active Noise Control course notes (January 2015)

Aliasing, Image Sampling and Reconstruction

Lab 1. The Fourier Transform

Introduction to Digital Audio

Emotion Detection from Speech

Power Spectral Density

This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.

Transition Bandwidth Analysis of Infinite Impulse Response Filters

MPEG Unified Speech and Audio Coding Enabling Efficient Coding of both Speech and Music

SUMMARY. Additional Digital/Software filters are included in Chart and filter the data after it has been sampled and recorded by the PowerLab.

Lecture 14. Point Spread Function (PSF)

Time Series Analysis

Time Series Analysis: Introduction to Signal Processing Concepts. Liam Kilmartin Discipline of Electrical & Electronic Engineering, NUI, Galway

This unit will lay the groundwork for later units where the students will extend this knowledge to quadratic and exponential functions.

The Fourier Analysis Tool in Microsoft Excel

Rectifier circuits & DC power supplies

Time-series filtering techniques in Stata

Continued Fractions and the Euclidean Algorithm

MODULATION Systems (part 1)

Digital Modulation. David Tipper. Department of Information Science and Telecommunications University of Pittsburgh. Typical Communication System

Transcription:

Digital Speech Processing Lectures 7-8 Time Domain Methods in Speech Processing 1

General Synthesis Model voiced sound amplitude Log Areas, Reflection Coefficients, Formants, Vocal Tract Polynomial, Articulatory Parameters, T 1 T 2 Rz ( ) = 1 α z 1 unvoiced sound amplitude Pitch Detection, Voiced/Unvoiced/Silence Detection, Gain Estimation, Vocal Tract Parameter Estimation, Glottal Pulse Shape, Radiation Model 2

General Analysis Model s[n] Speech Analysis Model Pitch Period, T[n] Glottal Pulse Shape, g[n] Voiced Amplitude, A V [n] V/U/S[n] Switch Unvoiced Amplitude, A U [n] Vocal Tract IR, v[n] Radiation Characteristic, r[n] All analysis parameters are time-varying at rates commensurate with information in the parameters; We need algorithms for estimating the analysis parameters and their variations over time 3

speech, x[n] Overview Signal Processing representation of speech speech or music A(x,t) formants reflection coefficients voiced-unvoiced-silence pitch sounds of language speaker identification emotions time domain processing => direct operations on the speech waveform frequency domain processing => direct operations on a spectral representation of the signal x[n] simple processing system enables various types of feature estimation zero crossing rate level crossing rate energy autocorrelation 4

P 1 P 2 V U/S Basics 8 khz sampled speech (bandwidth < 4 khz) properties of speech change with time excitation goes from voiced to unvoiced peak amplitude varies with the sound being produced pitch varies within and across voiced sounds periods of silence where background signals are seen the key issue is whether we can create simple time-domain processing methods that enable us to measure/estimate speech representations reliably and accurately 5

Fundamental Assumptions properties of the speech signal change relatively slowly with time (5-10 sounds per second) over very short (5-20 msec) intervals => uncertainty due to small amount of data, varying pitch, varying amplitude over medium length (20-100 msec) intervals => uncertainty due to changes in sound quality, transitions between sounds, rapid transients in speech over long (100-500 msec) intervals => uncertainty due to large amount of sound changes there is always uncertainty in short time measurements and estimates from speech signals 6

Compromise Solution short-time processing methods => short segments of the speech signal are isolated and processed as if they were short segments from a sustained sound with fixed (non-time-varying) properties this short-time processing is periodically repeated for the duration of the waveform these short analysis segments, or analysis frames almost always overlap one another the results of short-time processing can be a single number (e.g., an estimate of the pitch period within the frame), or a set of numbers (an estimate of the formant frequencies for the analysis frame) the end result of the processing is a new, time-varying sequence that serves as a new representation of the speech signal 7

Frame-by-Frame Processing in Successive Windows Frame 1 Frame 2 Frame 3 Frame 4 Frame 5 75% frame overlap => frame length=l, frame shift=r=l/4 Frame1={x[0],x[1],,x[L-1]} Frame2={x[R],x[R+1],,x[R+L-1]} Frame3={x[2R],x[2R+1],,x[2R+L-1]} 8

Frame 1: samples 0,1,..., L 1 Frame 2: samples RR, + 1,..., R+ L 1 Frame 3: samples 2 R,2R+ 1,...,2R+ L 1 Frame 4: samples 3 R,3R+ 1,...,3R+ L 1

Frame-by-Frame Processing in Successive Windows Frame 1 Frame 2 Frame 3 Frame 4 50% frame overlap Speech is processed frame-by-frame in overlapping intervals until entire region of speech is covered by at least one such frame Results of analysis of individual frames used to derive model parameters in some manner Representation goes from time sample x[ n], n = L,0,1, 2,L to parameter vector f[ m], m = 0,1, 2,L where n is the time index and m is the frame index. 10

Frames and Windows F S = 16, 000 samples/second L = 641 samples (equivalent to 40 msec frame (window) length) R = 240 samples (equivalent to 15 msec frame (window) shift) Frame rate of 66.7 frames/second 11

Short-Time Processing speech waveform, x[n] short-time processing speech representation, f[m] x[ n] = samples at 8000/sec rate; (e.g. 2 seconds of 4 khz bandlimited speech, xn [ ], 0 n 16000) r f[ m] = f [ m], f [ m],..., f [ m] = vectors at 100/sec rate, 1 m 200, { } 1 2 L L is the size of the analysis vector (e.g., 1 for pitch period estimate, 12 for autocorrelation estimates, etc) 12

Generic Short-Time Processing Qnˆ = T([ x m]) w% [ n m] m= n= nˆ x[n] T(x[n]) T( ) w[n] ~ Qˆn linear or non-linear transformation window sequence (usually finite length) Qˆn is a sequence of local weighted average values of the sequence T(x[n]) at time ˆ n = n 13

Short-Time Energy 2 E = x m m= [ ] -- this is the long term definition of signal energy -- there is little or no utility of this definition for time-varying signals nˆ 2 Enˆ = x m m= nˆ L+ 1 [ ] -- short-time energy in vicinity of time nˆ 2 T( x) = x wn % [ ] = 1 0 n L 1 = 0 otherwise 2 2 = x [ nˆ L+ 1] +... + x [ nˆ] 14

Computation of Short-Time Energy wn %[ m] n L+ 1 n window jumps/slides across sequence of squared values, selecting interval for processing what happens to Eˆn as sequence jumps by 2,4,8,...,L samples ( is a lowpass function so it can be decimated without lost of information; why is lowpass?) effects of decimation depend on L; if L is small, then is a lot more variable than if L is large (window bandwidth changes with L!) E ˆn E ˆn E ˆn 15

Effects of Window Q = T( x[ n]) w% [ n] nˆ = x [ n] w% [ n] n= nˆ n= nˆ wn %[ ] serves as a lowpass filter on T([]) x n which often has a lot of high frequencies (most non-linearities introduce significant high frequency energy think of what ( x[ n] x[ n] ) does in frequency) often we extend the definition of Qˆn to include a pre-filtering term so that x[ n] itself is filtered to a region of interest xˆ[ n] x[ n ] T([]) x n nˆ n n nˆ T ( ) wn %[ ] Linear Filter Q = Q = 16

Short-Time Energy serves to differentiate voiced and unvoiced sounds in speech from silence (background signal) natural definition of energy of weighted signal is: 2 E ˆ nˆ = x[ m] w% [ n m] (sum or squares of portion of signal) m= -- concentrates measurement at sample nˆ, using weighting w% [ n-m ˆ ] x[n] FS 2 2 2 E ˆ ˆ nˆ = x m w% n m = x m h n m m= m= 2 hn [ ] = w% [ n] [ ] [ ] [ ] [ ] x 2 [n] S short time energy Enˆ = E n n = nˆ ( ) 2 h[n] F F / R S 17

Short-Time Energy Properties depends on choice of h[n], or equivalently, ~ window w[n] if w[n] duration very long and constant amplitude ~ (w[n]=1, n=0,1,...,l-1), E n would not change much over time, and would not reflect the short-time amplitudes of the sounds of the speech very long duration windows correspond to narrowband lowpass filters want E n to change at a rate comparable to the changing sounds of the speech => this is the essential conflict in all speech processing, namely we need short duration window to be responsive to rapid sound changes, but short windows will not provide sufficient averaging to give smooth and reliable energy function 18

Windows consider two windows, w[n] rectangular window: h[n]=1, 0 n L-1 and 0 otherwise Hamming window (raised cosine window): ~ h[n]=0.54-0.46 cos(2πn/(l-1)), 0 n L-1 and 0 otherwise rectangular window gives equal weight to all L samples in the window (n,...,n-l+1) Hamming window gives most weight to middle samples and tapers off strongly at the beginning and the end of the window 19

Rectangular and Hamming Windows L = 21 samples 20

Window Frequency Responses rectangular window He ( ) = sin( ΩLT / 2) e sin( ΩT / 2) jωt jωt( L 1)/ 2 first zero occurs at f=f s /L=1/(LT) (or Ω=(2π)/(LT)) => nominal cutoff frequency of the equivalent lowpass filter Hamming window w% [ ] = 0.54 % [ ] 0.46*cos(2 π / ( 1)) % H n wr n n L wr[ n] can decompose Hamming Window FR into combination of three terms 21

RW and HW Frequency Responses log magnitude response of RW and HW bandwidth of HW is approximately twice the bandwidth of RW attenuation of more than 40 db for HW outside passband, versus 14 db for RW stopband attenuation is essentially independent of L, the window duration => increasing L simply decreases window bandwidth L needs to be larger than a pitch period (or severe fluctuations will occur in E n ), but smaller than a sound duration (or E n will not adequately reflect the changes in the speech signal) There is no perfect value of L, since a pitch period can be as short as 20 samples (500 Hz at a 10 khz sampling rate) for a high pitch child or female, and up to 250 samples (40 Hz pitch at a 10 khz sampling rate) for a low pitch male; a compromise value of L on the order of 100-200 samples for a 10 khz sampling 22 rate is often used in practice

Window Frequency Responses Rectangular Windows, L=21,41,61,81,101 Hamming Windows, L=21,41,61,81,101 23

Short-Time Energy Short-time energy computation: E = ([ x m][ w nˆ m]) nˆ nˆ m= = xm wn % ˆ m m= nˆ m= nˆ L+ 1 2 ([ ]) [ ] For L-point rectangular window, wm %[ ] = 1, m= 0,1,..., L 1 giving E = ([ x m]) 2 2 24

Short-Time Energy using RW/HW E ˆn L=51 E ˆn L=51 L=101 L=101 L=201 L=201 L=401 L=401 as L increases, the plots tend to converge (however you are smoothing sound energies) short-time energy provides the basis for distinguishing voiced from unvoiced speech regions, and for medium-to-high SNR recordings, can even be used to find regions of silence/background signal 25

Short-Time Energy for AGC Can use an IIR filter to define short-time energy, e.g., time-dependent energy definition 2 2 σ [ n] = x [ m] h[ n m]/ h[ m] m= m= 0 consider impulse response of filter of form n 1 n 1 hn [ ] = α un [ 1] = α n 1 = 0 n < 1 2 2 n m 1 σ [ n] = ( 1 α) x [ m] α u[ n m 1] m= 26

Recursive Short-Time Energy un [ m 1] implies the condition n m 1 0 or m n 1 giving n 1 2 2 n m 1 2 2 σ [ n] = ( 1 α) x [ m] α = ( 1 α)( x [ n 1] + αx [ n 2] +...) m= for the index n 1 we have n 2 2 2 n m 2 2 2 σ [ n 1] = ( 1 α) x [ m] α = ( 1 α)( x [ n 2] + αx [ n 3] +...) m= thus giving the relationship 2 2 2 σ [ n] = α σ [ n 1] + x [ n 1]( 1 α) and defines an Automatic Gain Control (AGC) of the form Gn [ ] = G 0 σ [ n] 27

Recursive Short-Time Energy 2 2 σ [ n] = x [ n] h[ n] hn n 1 [ ] = (1 αα ) un [ 1] 2 2 σ ( z) = X ( z) H( z) n n 1 n ( ) = [ ] = (1 αα ) [ 1] Hz hnz un z n= = (1 αα ) n= 1 m = n 1 n 1 n= n m ( m+ 1) 1 m m Hz ( ) = (1 αα ) z = (1 α) z α z m= 0 m= 0 z 1 2 = (1 α) z α z = (1 α) z = σ ( z)/ X ( z) 1 m m 1 2 1 m= 0 1 αz 2 2 2 σ [ n] = ασ [ n 1] + (1 α) x ( n 1) 28

Recursive Short-Time Energy x[n] ( 2 ) 2 n x [ ] z 1 + ( 1 α) σ 2 [ n 1] z 1 σ 2 [ n ] α 2 2 2 σ [ n] = α σ [ n 1] + x [ n 1]( 1 α) 29

Recursive Short-Time Energy 30

Use of Short-Time Energy for AGC 31

Use of Short-Time Energy for AGC α = 0.9 α = 0.99 32

Short-Time Magnitude short-time energy is very sensitive to large signal levels due to x 2 [n] terms consider a new definition of pseudo-energy based on average signal magnitude (rather than energy) M ˆ = [ ] %[ ˆ n xm wn m] m= weighted sum of magnitudes, rather than weighted sum of squares x[n] FS x[n] F F / Mnˆ = M n n = nˆ computation avoids multiplications of signal with itself (the squared term) S wn %[ ] S R 33

Short-Time Magnitudes M ˆn M ˆn L=51 L=51 L=101 L=101 L=201 L=201 L=401 L=401 differences between E n and M n noticeable in unvoiced regions dynamic range of M n ~ square root (dynamic range of E n ) => level differences between voiced and unvoiced segments are smaller E n and M n can be sampled at a rate of 100/sec for window durations of 20 msec or so => efficient representation of signal energy/magnitude 34

Short Time Energy and Magnitude Rectangular Window E ˆn M ˆn L=51 L=51 L=101 L=101 L=201 L=201 L=401 L=401 35

Short Time Energy and Magnitude Hamming Window E ˆn L=51 M ˆn L=51 L=101 L=101 L=201 L=201 L=401 L=401 36

Other Lowpass Windows can replace RW or HW with any lowpass filer window should be positive since this guarantees E n and M n will be positive FIR windows are efficient computationally since they can slide by R samples for efficiency with no loss of information (what should R be?) can even use an infinite duration window if its z-transform is a rational function, i.e., n hn [ ] = a, n 0, 0< a< 1 = 0 n < 0 1 Hz ( ) = 1 1 az z > a 37

Other Lowpass Windows this simple lowpass filter can be used to implement E n and M n recursively as: E = ae + a x n n n 1 n 1 2 ( 1 ) [ ] short-time energy M = am + ( 1 a) x[ n] short-time magnitude n need to compute E n or M n every sample and then down-sample to 100/sec rate recursive computation has a non-linear phase, so delay cannot be compensated exactly 38

Short-Time Average ZC Rate zero crossing => successive samples have different algebraic signs zero crossings zero crossing rate is a simple measure of the frequency content of a signal especially true for narrowband signals (e.g., sinusoids) sinusoid at frequency F 0 with sampling rate F S has F S /F 0 samples per cycle with two zero crossings per cycle, giving an average zero crossing rate of z 1 =(2) crossings/cycle x (F 0 / F S ) cycles/sample z 1 =2F 0 / F S crossings/sample (i.e., z 1 proportional to F 0 ) z M =M (2F 0 /F S ) crossings/(m samples) 39

Sinusoid Zero Crossing Rates Assume the sampling rate is F = 10, 000 Hz 1. F = 100 Hz sinusoid has F / F = 10, 000 / 100 = 100 samples/cycle; 0 S 0 or z = 2 / 100 crossings/sample, or z = 2 / 100 * 100 = 1 100 2 crossings/10 msec interval S 2. Hz sinusoid has / = 10, 000 / 1000 = 10 samples/cycle; F0 = 1000 FS F0 or z = 2 / 10 crossings/sample, or z = 2 / 10 * 100 = 1 100 20 crossings/10 msec interval 3. F = 5000 Hz sinusoid has F / F = 10, 000 / 5000 = 2 samples/cycle; 0 S 0 or z = 2/ 2 crossings/sample, or z100 = 2 / 2 * 100 = 1 100 crossings/10 msec interval 40

Zero Crossing for Sinusoids 1 offset:0.75, 100 Hz sinewave, ZC:9, offset sinewave, ZC:8 100 Hz sinewave 0.5 0 ZC=9-0.5-1 0 50 100 150 200 Offset=0.75 1.5 100 Hz sinewave with dc offset 1 ZC=8 0.5 0 0 50 100 150 200 41

Zero Crossings for Noise offseet:0.75, random noise, ZC:252, offset noise, ZC:122 3 random gaussian noise 2 1 ZC=252 0-1 -2 0 50 100 150 200 6 Offset=0.75 random gaussian noise with dc offset 4 2 ZC=122 0-2 0 50 100 150 200 250 42

ZC Rate Definitions nˆ 1 Zˆ = sgn( [ ]) sgn( [ 1]) % [ ˆ n x m x m w n m] 2L eff m= nˆ L+ 1 sgn( xn [ ]) = 1 xn [ ] 0 = 1 xn [ ] < 0 simple rectangular window: wn % [ ] = 1 0 n L 1 = 0 otherwise L eff = L Same form for Z as for E or M nˆ nˆ nˆ 43

ZC Normalization The formal definition of Z is: nˆ 1 Znˆ = z1 = sgn( xm [ ]) sgn( xm [ 1]) 2L 1 1 m= nˆ L+ 1 nˆ is interpreted as the number of zero crossings per sample. For most practical applications, we need the rate of zero crossings per fixed interval of M samples, which is z = z M = rate of zero crossings per M sample interval M Thus, for an interval of τ sec., corresponding to M samples we get z = z M; M = τ F M S = τ / T F = 10, 000 Hz; T = 100 μsec; τ= 10 msec; M = 100 samples S F = 8, 000 Hz; T = 125 μsec; τ= 10 msec; M = 80 samples S F = 16, 000 Hz; T = 62. 5 μsec; τ= 10 msec; M = 160 samples S Zero crossings/10 msec interval as a function of sampling rate 44

ZC Normalization For a 1000 Hz sinewave as input, using a 40 msec window length ( L), with various values of sampling rate ( F ), we get the following: S F L z M z S 8000 320 1/ 4 80 20 10000 400 1 / 5 100 20 16000 640 1 / 8 160 20 1 M Thus we see that the normalized (per interval) zero crossing rate, z M, is independent of the sampling rate and can be used as a measure of the dominant energy in a band. 45

ZC and Energy Computation Hamming window with duration L=201 samples (12.5 msec at Fs=16 khz) Hamming window with duration L=401 samples (25 msec at Fs=16 khz) 46

ZC Rate Distributions Unvoiced Speech: the dominant energy component is at about 2.5 khz 1 KHz 2KHz 3KHz 4KHz Voiced Speech: the dominant energy component is at about 700 Hz for voiced speech, energy is mainly below 1.5 khz for unvoiced speech, energy is mainly above 1.5 khz mean ZC rate for unvoiced speech is 49 per 10 msec interval mean ZC rate for voiced speech is 14 per 10 msec interval 47

ZC Rates for Speech 15 msec windows 100/sec sampling rate on ZC computation 48

Short-Time Energy, Magnitude, ZC 49

Issues in ZC Rate Computation for zero crossing rate to be accurate, need zero DC in signal => need to remove offsets, hum, noise => use bandpass filter to eliminate DC and hum can quantize the signal to 1-bit for computation of ZC rate can apply the concept of ZC rate to bandpass filtered speech to give a crude spectral estimate in narrow bands of speech (kind of gives an estimate of the strongest frequency in each narrow band of speech) 50

Summary of Simple Time Domain Measures sn ( ) Q = T( x[ m]) w% [ nˆ m] nˆ 1. Energy: nˆ 2 E ˆ nˆ = x m w% n m m= nˆ L+ 1 nˆ Linear Filter m= nˆ m= nˆ L+ 1 [ ] [ ] can downsample Enˆ at rate commensurate with window bandwidth 2. Magnitude: x( n) M = x[ m] w% [ nˆ m] T[ ] T( x[ n]) 3. Zero Crossing Rate: nˆ 1 Zn ˆ = z1 = sgn( xm [ ]) sgn( xm [ 1]) wn % [ ˆ m] 2L m= nˆ L+ 1 where sgn( xm [ ]) = 1 xm [ ] 0 = 1 xm [ ] < 0 wn %[ ] Qˆn 51

Short-Time Autocorrelation -for a deterministic signal, the autocorrelation function is defined as: Φ[ k] = x[ m] x[ m+ k] m= -for a random or periodic signal, the autocorrelation function is: L 1 Φ[ k] = lim x[ m] x[ m+ k] L ( 2L + 1) m= L - if x[ n] = x[ n+ P], then Φ[ k] = Φ[ k + P], => the autocorrelation function preserves periodicity -properties of Φ[ k] : 1. Φ[ k] is even, Φ[ k] = Φ[ k] 2. Φ[ k] is maximum at k = 0, Φ[ k] Φ[ 0], k 3. Φ[ 0] is the signal energy or power (for random signals) 52

Periodic Signals for a periodic signal we have (at least in theory) Φ[P]=Φ[0] so the period of a periodic signal can be estimated as the first non-zero maximum of Φ[k] this means that the autocorrelation function is a good candidate for speech pitch detection algorithms it also means that we need a good way of measuring the short-time autocorrelation function for speech signals 53

Short-Time Autocorrelation - a reasonable definition for the short-time autocorrelation is: R ˆ ˆ ˆ [ ] = [ ] %[ ] [ + ] % n k x m w n m x m k w[ n k m] m= 1. select a segment of speech by windowing 2. compute deterministic autocorrelation of the windowed speech R [ k] = R [ k] - symmetry nˆ nˆ = xmxm [ ] [ k] wn %[ ˆ mwn ] %[ ˆ + k m] - define filter of the form w% [ nˆ] = w% [ nˆ] w% [ nˆ + k] k m= - this enables us to write the short-time autocorrelation in the form: R ˆ ˆ[ ] = [ ] [ ] % n k x m x m k wk [ n m] m= th - the value of w% [ k] at time nˆ for the k lag is obtained by filtering nˆ the sequence xn [ ˆ] xn [ ˆ k] with a filter with impulse response w [ nˆ ] % k 54

Short-Time Autocorrelation R ˆ ˆ nˆ [ k] = x[ mw ] %[ n m] x[ m+ k] w% [ n+ k m] m= ~ ~ L n-l+1 n+k-l+1 points used to compute R [ 0]; L k points used to compute R [ k]; nˆ nˆ 55

Short-Time Autocorrelation 56

Examples of Autocorrelations autocorrelation peaks occur at k=72, 144,... => 140 Hz pitch Φ(P)<Φ(0) since windowed speech is not perfectly periodic over a 401 sample window (40 msec of signal), pitch period changes occur, so P is not perfectly defined much less clear estimates of periodicity since HW tapers signal so strongly, making it look like a non-periodic signal 57 no strong peak for unvoiced speech

Voiced (female) L=401 (magnitude) T0 = N0T t = nt T 0 F = 0 1 T 0 F / 58 F s

Voiced (female) L=401 (log mag) T 0 t = nt T 0 F = 0 1 T 0 F / 59 F s

Voiced (male) L=401 T 0 T 0 3F = 0 3 T 0 60

Unvoiced L=401 61

Unvoiced L=401 62

Effects of Window Size choice of L, window duration small L so pitch period almost constant in window L=401 L=251 L=125 large L so clear periodicity seen in window as k increases, the number of window points decrease, reducing the accuracy and size of R n (k) for large k => have a taper of the type R(k)=1-k/L, k <L shaping of autocorrelation (this is the autocorrelation of size L rectangular window) allow L to vary with detected pitch periods (so that at least 2 full periods are included) 63

Modified Autocorrelation want to solve problem of differing number of samples for each different k term in Rn ˆ[ k], so modify definition as follows: Rˆ ˆ[ ] = [ ] % ˆ ˆ 1[ ] [ + ] % n k x m w n m x m k w2[ n m k] m= - where w% is standard L-point window, and w% is extended window 1 2 of duration L+ K samples, where K is the largest lag of interest - we can rewrite modified autocorrelation as: Rˆ [ k] = x[ nˆ + m] wˆ [ m] x[ nˆ + m+ k] wˆ [ m+ k] nˆ 1 1 2 - where wˆ [ m] = w% [ m] and wˆ [ m] = w% [ m] 1 1 2 2 - for rectangular windows we choose the following: -giving Rˆ nˆ m= wˆ [ m] = 1, 0 m L 1 wˆ [ m] = 1, 0 m L 1+ K 2 1 L m= 0 [ k] = x[ nˆ + m] x[ nˆ + m+ k], 0 k K - always use L samples in computation of Rˆ [ k] k nˆ 64

Examples of Modified Autocorrelation L-1 L+K-1 - Rˆ [ k] is a cross-correlation, not an auto-correlation n - Rˆ [ k] Rˆ [ k] n n - Rˆ [ k] will have a strong peak at k = P for periodic signals n and will not fall off for large k 65

Examples of Modified Autocorrelation 66

Examples of Modified AC L=401 L=401 L=401 L=251 L=401 L=125 Modified Autocorrelations fixed value of L=401 Modified Autocorrelations values of L=401,251,125 67

Waterfall Examples 68

Waterfall Examples 69

Waterfall Examples 70

Short-Time AMDF - belief that for periodic signals of period P, the difference function dn [ ] = xn [ ] xn [ k] - will be approximately zero for k = 0, ± P, ± 2P,... For realistic speech signals, dn [ ] will be small at k= P--but not zero. Based on this reasoning. the short-time Average Magnitude Difference Function (AMDF) is defined as: γ ˆ[ ] = [ ˆ + ] % ˆ 1[ ] [ + ] % n k x n m w m x n m k w2[ m k] m= - with w% [ m] and w% [ m] being rectangular windows. If both are the same 1 2 length, then γ [ k] is similar to the short-time autocorrelation, whereas nˆ if w% [ m] is longer than w% [ m], then γ [ k] is similar to the modified short-time 2 1 autocorrelation (or covariance) function. In fact it can be shown that 12 / nˆ γ ˆ[ ] 2β[ ] ˆ ˆ[ 0] ˆ ˆ[ ] n k k Rn Rn k - where β[ k] varies between 0.6 and 1.0 for different segments of speech. 71

AMDF for Speech Segments 72

Summary Short-time parameters in the time domain: short-time energy short-time average magnitude short-time zero crossing rate short-time autocorrelation modified short-time autocorrelation Short-time average magnitude difference function 73