ONE MICROPHONE SINGING VOICE SEPARATION USING SOURCE-ADAPTED MODELS Alexey Ozerov,Pierrick Philippe, Remi Gribonval, Frederic

Transcription

1 ONE MICROPHONE SINGING VOICE SEPARATION USING SOURCE-ADAPTED MODELS Alexey Ozerov,Pierrick Philippe, Remi Gribonval, Frederic Bimbot Presented by Orly Kohn Feldman

2 Main Idea Problem nature - Source Separation # of sensors - one microphone # of sources SINGING VOICE & background music (2 sources) Goal singing voice extraction

3 Method GMM (Gaussian Mixture Models) of short time spectra of two sources. Source Model Adaptation INOVATION adaptation method consisting in a filter adaptation technique (MLLR)

4 Assumptions a priori probabilistic models for two sources Each recording (mixture) is a simple sum of voice and music x(n) - recording or mixture v(n) - singing voice m(n) - background music xn ( ) = vn ( ) + mn ( )

5 Goal Estimate the voice contribution vn ˆ( ) In the observed signal x( n)

6 A priori probabilistic models for two sources These models are learned from training sources Problem in similar methods Usage of general models (training sources issued from recording different from those to be separated) yield poor results Solution approach adapted models

7 Basic Algorithm Segmentation of signal to vocal & non vocal parts Learn adapted music model from non vocal parts Learn adapted voice model using learned adapted music model as a priori data (pullout singing voice from music background)

8 Innovation Usage of adaptation technique for singing voice extraction (modifications) Voice model adaptation assuming that the adapted voice model is obtained from the general voice model by a linear transformation of the feature space (short time spectra)

9 GMM based source separation

10 GMM model for singing voice The short time Fourier spectra Vt at time t of the voice signal v are modeled with a GMM, where Vt(f) represents the local Power spectral Density at frequency f at state i of GMM

11 GMM model for background music Mt (short time Fourier spectra of the background music ) is defined in a similar way to the singing voice

12 Separation by adaptive Wiener filtering Separation is performed in STFT domain with MMSE estimator

13 Model learning GMM models (covariance matrices) are learned by maximization of the likelihoods : pv ( / Σ ) & p(m/ Σ ) v m where V & M are the STFT of the training signals Maximization is achieved using the Expectation Maximization (EM) algorithm

14 Model Adaptation Let voc denote the indices of the frames where voice is present in X. Learn music model from the non vocal frames. Estimate voice model from vocal frames in a maximum likelihood manner as follow (using EM)

15 Adaptation Procedure

16 Wiener Adaptive Filter Formulation TOC - # of vocal frames

17 Filter Invariant Modeling REMEMBER General voice model is a model derived from different singing recordings There are several variability factors in model (specific microphone, specific acoustics) Suggested Solution model variability using a global causal linear time invariant filter instead of building GMMs for inter-recording recording variability Use GMMs to model internal dynamics of the generic vocal source

18 Voice Modeling vr = hr or Vrt ( f ) - STFT of v r v - voice recording r h - a global filter r o - "original voice" r H( f )- STFT of h r O ( f ) - STFT of o rt r r

19 Probability density of the recorded voice STFT Vr

20 Filter Adaptation via MLLR The full adaptation of all voice model parameters is replaced by the only adaptation of the global filter

21 MLLR framework This optimization problem is solved by Maximum Likelihood Linear Regression framework.

22 Full Model Approach versus Filter Adaptation via MLLR

23 Replace EM algorithm for H

24 Filter Adapted Training The above filter adaptation technique is also applied to general voice model training. Thus the general voice model and the unknown filters are jointly estimated as follow

25 SAGE algorithm used to solve the joint maximization problem Space Alternating Generalized EM algorithm is used to estimate both generalized voice model and filters. Each algorithm iteration is combined of two separate iterations each time one parameter is optimized and the other is held fixed

26 Sage Formulation

27 Results Data Description Training data base for general voice includes 34 samples of singing men s s voices from popular music. General music model is trained on 30 samples of popular music free of voice Samples are of one minute length Samples come from different artists Test data five songs of the same genre, where voice and music tracks are available separately Item are segmented manually Sampling rate 11025Hz

28 Results Performance Measure Quality criteria chosen is Source to Distortion Ratio (SDR)

29 Results - Simulations

30 Preview Music model learning on the non vocal parts Filter adapted learning of the general voice model Filter adaptation of the voice model at the separation stage

31 Comparison to Full Voice Model Adaptation Usage of Full Voice Model gives a close results GNSDR 9.9dB, However this techniques sometimes leads to certain listening impairments

32 Possible problems & withdraws Manual segmentation Complexity (time & convergence) Demonstrated at low sampling frequency Can a linear transformation of a generalized voice model really model all voice variability?!