BLIND speech separation (BSS) aims to recover source

Transcription

1 A Convex Speech Extraction Model and Fast Comptation by the Split Bregman Method Meng Y, Wenye Ma, Jack Xin, and Stanley Osher. Abstract A fast speech extraction (FSE) method is presented sing convex optimization made possible by pase detection of the speech sorces. Sparse nmixing filters are soght by l norm reglarization and the split Bregman method. A sbdivided split Bregman method is developed for efficiently estimating long reverberations in real room recordings. The speech pase detection is based on a binary mask sorce separation method. The FSE method is evalated objectively and sbjectively, and fond to otperform existing blind speech separation approaches on both synthetic and room recorded data in terms of the overall comptational speed and separation qality. Index Terms convexity, sparse filters, split Bregman method, fast blind speech extraction. I. INTRODUCTION BLIND speech separation (BSS) aims to recover sorce signals from their mixtres withot detailed knowledge of the mixing process []. However, it remains a challenge to retrieve sond sorces recorded in real-world environment sch as in clttered rooms. The physical reason is that sond reflections (reverberations) in enclosed rooms case signal mixing at crrent time to depend on sorce signals and their long delays (history dependent). Mathematically, the mixing process is convoltive in time and the nknowns are high dimensional. Varios efforts have been made to separate convoltive mixtres. Three major approaches are: time-domain BSS, freqency domain BSS, and time-freqency (TF) domain BSS. Time domain BSS is based on optimizing a cost fnction (measring entropy or non-gassianity) for time domain signals, for example independent component analysis (ICA). The approach is theoretically reasonable, and achieves a good separation if the optimization can be done accrately as is the case for mixtres with minimal time delay (low reverberation). However at the fndamental level, all time domain methods based on ICA attempt to optimize non-convex objectives, for which no global convergence is mathematically garanteed. This weakness poses a difficlt problem for actal convergence and robstness of approximation in real-world settings where high dimensional (on the order of thosands) optimization nder measrement noise is encontered. The lack of robstness The athors M. Y and W. Ma contribted eqally to this work. M. Y and J. Xin were partially spported by NSF DMS-9277 and DMS-7288; W. Ma and S. Osher were partially spported by NSF DMS-9456, NIH G54RR283 and the Department of Defense. M. Y and J. Xin are with the Department of Mathematics, University of California, Irvine, CA, USA ( [email protected]; [email protected]). W. Ma and S. Osher are with Department of Mathematics, University of California, Los Angeles, CA, 995, USA ( [email protected]; [email protected]). nder pertrbations may be explained by potentially many local minima of a non-convex objective where approximating seqences can get stck in. Even if local convergence of optimizing seqence occrs, it may be comptationally expensive. For example, the time domain scaled natral gradient method [] is typically time consming in the regime of long reverberations. Moreover, small divisors and divergence may occr in silent drations of mixtre signals, in other words, the method is not stable in the presence of a small piece of silent dration. Thogh a nonlocally weighted soft constrained natral gradient method [2] resolves sch isses and renders the method asymptotically consistent, convergence is still rather slow. In freqency domain BSS, the observed time domain signals are converted into freqency domain time series signals by the short-time (windowed) Forier transform (STFT). The convoltive mixtre can be approximated with mltiple instantaneos mixtres (no time delay), each of which is defined in a freqency bin. The approximation is however only valid if the window size is mch larger than the length of delay. Under sch condition, one can then employ any instantaneos BSS algorithm to separate the mixtres bin by bin. However, the permtation and scaling ambigity of a BSS soltion trns into a serios problem in reconstrcting time domain otpt. The order of the otpt in each freqency bin mst be determined correctly so that the separated freqency components that originate from the same sorce are groped together before taking inverse STFT. Large window size, permtation and scaling isses limit the effectiveness of freqency domain BSS in reverberant conditions. In contrast, the time-freqency domain methods ([3], [4]) by spectral data clstering are both simple and efficient partly becase they do not resolve room implse responses. The basic working assmption is that at most one sorce signal is dominant at each time-freqency point of the mixtre spectrogram. In other words, the Forier spectra of the sorce signals rarely overlap in time. Sch a nonoverlapping property in the TF domain deteriorates however in reverberant conditions ([3], [4]), casing clstering errors and msical noise in the otpt. In this paper, a novel fast time domain speech extraction (FSE) method is proposed based on the assmption that intelligible speech signals contain pases. Pase detection is a problem of independent interest, which we handle here by processing the otpt of a modified TF domain clstering method. Becase we only detect silence drations from the initial separation, tolerance of artifacts in TF domain clstering is higher. Dring silent drations of the target speech signal, information of the interference (backgrond) is collected and

2 2 allows s to formlate a convex optimization problem for finding part of the implse response fnctions which sffice to estimate the target speech. A sparse soltion is then compted by l norm reglarization and the split Bregman method for which fast convergence was recently stdied [5]. The proposed time domain approach is free from the permtation problem in freqency domain BSS and relaxes the TF domain non-overlap hypothesis. It also does not rely on speech data statistics, and so enjoys high efficiency in data sage and comptation. This paper is organized as follows. In section II, the convex optimization problem for FSE is introdced. In section III, comptational framework by l norm reglarization is shown. The Bregman method and the split Bregman method are explained with algorithmic schemes and convergence proofs. In sbsections III-C and III-D, algorithms for moderately and highly reverberant acostic environments are illstrated. The sbdivided split Bregman method is proposed for FSE with long reverberations and large nmber of sorces. In section IV, an onset-offset detection method of speech is otlined. In section V, the length of selected silent speech dration is stdied, and the comparison between the split Bregman method and the sbdivided split Bregman method is illstrated nder different lengths of the filters. Evalations of FSE show its merits in both speed and separation qality in comparison with existing methods. Discssion and conclsions are in section VI. Or method also applies to non-speech signal extraction from convoltive mixtres as long as pase detection of target signal is possible. II. FAST SPEECH EXTRACTION MODEL Let s consider two sensors and two sond sorces which can be either two speech signals or one speech signal and one non-speech backgrond interference (msic or other ambient noises). FSE method shall seqentially extract speech signals if there are more than one speech sorces. Let s denote one of the two sorces as the target speech signal s T, and the other one as backgrond interference s B. The mixing model is x i (t) = h i s B (t) + h i2 s T (t) () where t is time; i =, 2; and is linear convoltion. Instead of finding an nmixing filter W sch that W (x, x 2 ) recovers (s T, s B ), we extract speech signal s T by eliminating (not recovering) interference s B. Sppose that the target speech contains pases. Then there is a nion D of disjoint time intervals where s T, while interference s B is active. It follows from () that h 2 x (t) h x 2 (t) for t D. The elimination by cross mltiplication was known in blind channel identification [6] and backgrond sppression [7]. Inside D, we seek a pair of sparse filters i (i =, 2) to minimize the energy of 2 x x 2 in the region D. Ideally, h and 2 h 2, that is the soltions are expected to be a pair of sparse acostic room implse response (RIR). The sparse RIR model is theoretically sond [8], and has been shown sefl for estimating RIRs in real acostic environments [9]. Filter sparseness is achieved by l -norm reglarization. The reslting convex optimization problem for t D is: (, 2) (, 2) 2 2 x x η2 2 2 ( i () ) 2 + µ( + 2 ) (2) i= where the second term is to fix scaling and prevent zero (trivial) soltion. Denote the length of D by L D and that of i by L. D can be as short as even.25 s dration, which makes FSE method efficient on the data sage and different from other BSS methods that are based on the high order statistics of data. In matrix form, convex objective (2) becomes: 2 A f µ (3) where is formed by stacking p and 2 ; vector f = (,,,, η) T with length L D + ; and (L D + ) 2L matrix A (T is transpose) is: T A = x () x (2) x (L D ) x (L D ) η x () x (L D 2) x (L D ) x ()... x (L D L+) x 2() x 2(2) x 2(L D ) x 2(L D ) η x 2() x 2(L D 2) x 2(L D ) x 2()... x 2(L D L+) When t D, cross mltiplication of () shows that ŝ T = 2 x x 2 h 2 x h x 2 = (h 2 h 2 h h 22 ) s T. Interference s B is eliminated and ŝ T sonds same as s T to hman ear. Here we assmed that the acostic environment does not change mch so that estimates of h and h 2 dring D still apply when t D. For a convex objective with nonnegativity filter constraints for sparsity, see [7]. Extraction of a speech sorce from M 3 mixtres of N sorces (N = M) is similar. Let a sorce s n ( n N) be silent in t D, for proper vale of (η, µ) >, we minimize: M 2 M jn x j η2 2 ( M jn () ) 2 + µ( jn ), j= j= and estimate s n by ŝ n = M jn x j. j= j= III. MINIMIZATION BY BREGMAN METHOD In this section, we introdce Bregman distance [], Bregman and split Bregman methods of non-smooth convex optimization. We show that the split Bregman method boils down to simple operations sch as shrinkage, matrix mltiplication, and one-time matrix inversion. Then we adapt the split Bregman method and apply it to the convex speech extraction model (3) in reverberant conditions. A. Bregman iterative reglarization The Bregman method was first applied [] to the image denoising model of Rdin-Osher-Fatemi [2] with the nonsmooth total variation (TV) reglarization: µ + 2 f 2 2 (4)

3 3 where f is the observed noisy data and µ is a positive parameter related to signal to noise ratio. The Bregman distance is D p J (, v) = J() J(v) p, v where J() = µ and p J is a sbgradient of J at the point v. The Bregman distance is not a distance in the sal sense becase D p J (, v) Dp J (v, ) in general. However, it measres the closeness of two points since D p J (, v) for all and v, and D p J (, v) Dp J (w, v) for all w on the line segment connecting and v. The Bregman iterative reglarization procedre [] is to solve a seqence of nconstrained sbproblems k+ D pk J (, k ) + 2 f 2 2 (5) for k =,,..., starting with = and p =. Since J() = µ is not differentiable everywhere, the sbgradient of J may not be niqe. However, it follows from the optimality of k+ in (5) that the inclsion J( k+ ) p k + k+ f holds or: p k+ = p k + f k+. (6) Bregman iteration refers to the mapping from ( k, p k ) ( k+, p k+ ). Now consider a more general constrained minimization problem: min J(), s.t. H() = (7) where J is convex bt not necessarily differentiable, sch as the l norm or TV norm, and H is convex and differentiable with zero as its minimm vale. Traditionally, this problem may be solved by contination methods. One solves a seqence of nconstrained problems min J() + λ k H(). (8) By choosing a seqence of positive nmbers λ k with λ k, one gets the soltion of the constrained problem (7). Instead of solving (8), one solves a seqence of sbproblems sing the iterative reglarization procedre as above: k+ D pk J (, k ) + H() (9) p k+ = p k H( k+ ). () with = and p =. In [], the athors analyzed the convergence of Bregman iterative scheme (9)-() and showed that nder fairly weak assmptions on J and H, H( k ) as k. For some cases, it is shown later [3] that this procedre solves the original problem (7). Here we restate two particlar convergence reslts in []. Theorem III.. Assme that J and H are convex fnctionals and H is differentiable, and that the soltions to the sbproblem in (9) exist. Let be a minimizer of H, we then have () monotonic decrease in H: H( k+ ) H( k ), (2) convergence to a minimizer of H: H( k ) H( ) + J( )/k. Theorem III. shows that H( k ) converges to H( ). In particlar, if H has minimal vale, then k gets arbitrarily close to the soltion of the constraint (7). If H() = 2 A f 2 2 and A = f has a soltion, then H( k ) converges to in finitely many steps [3]. The advantage of Bregman method is that it transforms a constrained problem into a seqence of nconstrained sbproblems. It is different from the contination methods since the parameter λ k = (niform) for all sbproblems. These sbproblems are solvable in closed form when J is l norm, as we show below in the context of the so called split Bregman method. B. Split Bregman method The split Bregman method was introdced by Goldstein and Osher [5] for solving l, TV, and related reglarized problems in imaging. It has connections to Lagrangian-based alternating direction methods in convex optimization [4]. Consider the nconstrained problem: min J(Φ) + H(), where J and H are as in (7), and Φ is linear operator. In case of (3), J() = µ, H() = 2 A f 2 2, and Φ = I. The key idea is to introdce an axiliary variable d = Φ, and solve the constrained problem or min J(d) + H(), s.t. d = Φ () d, λ min E(d, ), s.t. d, 2 d Φ 2 2 = where E(d, ) = J(d) + H() and λ is a positive constant. Then we can Bregmanize the problem as in (9). We replace E(d, ) by its associated Bregman distance and pdate the sbgradients p k d and pk respectively. Given that =, d =, p d =, and p =, we have the iterations: ( k+, d k+ ),d J(d) + H() pk d, d d k p k, k + λ 2 d Φ 2 2 p k+ d =p k d λ(d k+ Φ k+ ) p k+ =p k λφ T (Φ k+ d k+ ) For simplicity, we introdce a new variable b k = p k d /λ. And we notice that p k d = λbk and p k = λφ T b k, and ths the iterations become: ( k+, d k+ ),d J(d) + H() λ bk, d d k + λ b k, Φ( k ) + λ 2 d Φ 2 2 b k+ =b k d k+ + Φ k+ with =, d = and b =. The iterates d k+ and k+ can be pdated alternatively. We first fix k to pdate d k+ and then fix d k+ to pdate k+. The general split Bregman

4 4 iteration with initial vales d =, =, b =, is: d k+ d λ J(d) bk, d d k + 2 d Φk 2 2 (2) k+ λ H() + bk, Φ( k ) + 2 dk+ Φ 2 2 (3) b k+ =b k (d k+ Φ k+ ) (4) If J is the l norm, the sbproblem (2) has explicit soltions. The sbproblem (3) is also easy to solve since the objective is differentiable. Convergence of the split Bregman method for the case of J() = µ is analyzed [5], and the reslt is: Theorem III.2. Assme that there exists at least one soltion of (). Then we have the following properties for the split Bregman iterations (2),(3), and (4): lim k µ Φk + H( k ) = µ Φ + H( ) Frthermore, lim k k 2 = if is the niqe soltion. C. Implementation of FSE for moderate reverberations In this sbsection, we implement or proposed FSE method for the moderate reverberation case. Let J() = µ, Φ = I, and H() = 2 A f 2 2. Applying the split Bregman method and setting d =, =, and b =, we have the iterations: d k+ µ d λ d b k, d d k + 2 d k 2 2 (5) k+ 2λ A f b k, k + 2 dk+ 2 2 (6) b k+ =b k (d k+ k+ ) (7) Explicitly solving (5) and (6) gives the simple algorithm Initialize =, d =, b = While k+ k 2 / k+ 2 > ɛ () d k+ = shrink( k + b k, µ λ ) (2) k+ = (λi + A T A) (A T f + λ(d k+ b k )) (3) b k+ = b k d k+ + k+ end While Here shrink is the soft threshold fnction defined by shrink(v, t) = (τ t (v ), τ t (v 2 ),, τ t (v n )) with τ t (x) = sign(x) max{ x t, }, see Fig.. Noting that the matrix A is fixed, we can precalclate (λi + A T A), then the iterations only involve matrix mltiplication and are extremely fast as a reslt. For moderate reverberation, the length of room implse response (RIR) is not too long. The size of matrix λi+a T A is NL NL, N being the nmber of sorces. The comptational cost for matrix inversion is not high. The above algorithm rns fast for the prpose of FSE. Fig.. Demonstration of shrink operator in sbsection III-C D. Sbdivided Split Bregman for Long Reverberations In the strong reverberation regime, RIR length is on the order of thosands. In order to have a more accrate soltion, the length of shold be large accordingly. The length of also goes p when N 3. To redce cost of matrix inversion when is high dimensional, we sbdivide into r parts: = (, 2,, r ) T with i R NL r. Correspondingly A = [A, A 2,, A r ]. The minimization problem is: r 2 A i i f µ i= r i. i= The split Bregman method is applied to pdate each sbdivided part of seqentially (pdate i by fixing the other r j s). Initialize =, d =, b = While k+ k 2 / k+ 2 > ɛ () d k+ = shrink( k + b k, µ λ ) (2) For i from to r k+ i = (λi + A T i A i ) (A T i (f j i A j j ) end For + λ(d k+ i b k i )) (3) b k+ = b k d k+ + k+ end While where d i and b i are the sbdivided parts of d and b. We precalclate inverse matrices (λi +A T i A i), each NL r dimensional. With proper choice of the nmber r, the comptation speed can be improved significantly, as shown in section V. IV. SOURCE ACTIVITY DETECTION The necessary preparation for FSE is silence detection of the speech sorces. To maintain the overall speed of the proposed method, silence detection is based on the binary mask (BM) separation method DUET, the Degenerate Unmixing Estimation Techniqe [3], a fast method of blind speech separation withot resolving RIRs. Thogh msical noise may occr de to binary operation in TF domain, DUET appears reliable for identifying silence periods of a target speech from a mixtre (a robst speech featre). A brief review of DUET algorithm is given here. The standard mixing model for two

5 5 receivers and mltiple sorces is x j (t) = N k= h jk s k, where j =, 2, is the convoltion and h jk represents the implse response from sorce s k to sensor j. The time-domain signals x j (t), j =, 2, sampled at freqency f s are first converted into freqency-domain time-series signals X j (f, τ) with STFT. To grop TF points into N clsters sch that the points within each clster are dominated by a single sorce signal, the featre parameters associated with each TF point are defined as a(f, τ) = r(f, τ) and δ(f, τ) = f r(f, τ), where r(f, τ) = X2(f,τ) X (f,τ), denotes the magnitde and denotes the phase angle of a complex nmber. Sfficient vales of a(f, τ) and δ(f, τ) generate a smooth two dimensional histogram. The K-means clstering algorithm finds the N most prominent peaks in the histogram. Each peak corresponds to one sorce in the mixtre and the vale for a(f, τ) and δ(f, τ) at that peak are the featre parameters for that sorce. Once the featre parameters for each sorce have been estimated, DUET assigns the energy in each TF point to the sorce whose peak lies closest to that point in the featre space of a and δ. The individal separated signal spectrogram Y n (f, τ) is estimated based on the clstering reslt. The TF binary mask for the n-th sorce signal is: { (f, τ) clster C k M n (f, τ) = (8) otherwise Then Y n (f, τ) = M n (f, τ)x J (f, τ), where n =,..., N and J is a selected sensor index. Finally, inverse STFT (istft) is applied to Y n (f, τ) with overlap-add method [6] to recover the waveform y n (t). The ratio R n (τ) = Yn(,τ) 2 2 Y B is sed for detecting the (,τ) 2 2 silence part of sorce n, where Y B is the sm of backgrond sorces. Thogh the separation qality may degrade if reverberation is long, the onset-offset featre is robst and detectable if we delete certain fzzy points and redce binary masking errors. Specifically, at each TF point (f, τ), the confidence coefficient of (f, τ) C n is defined by CC(f, τ) = d n min j n d j, where d j is the distance between the vale of a and δ at (f, τ) and that at j-th peak. The mask is redefined for some ρ > as { (f, τ) C n & CC(f, τ) ρ M n (f, τ) = (9) otherwise The ρ is sally set to be /2 to alleviate clstering error. We check the mean and variance of the ratio R n frame by frame with proper frame size and overlapping. The time intervals with small mean and variance vales are selected as the region where sorce n is almost silent. The entire FSE algorithm is: V. EVALUATION AND COMPARISON The implementation is in Matlab 29b and the evalation is done in the Windows 7 Home Premim operation system with Intel Core i5-m GHz CPU and 3. GB memory. The parameters for FSE are chosen as µ = ɛ = 3, η =, and λ = 2µ throghot the evalation. Fig. 2. Sorce activity detection (mixtre of speech and msic). Top: ratio R(τ); middle: mean of R(τ); bottom: variance of R(τ). Detection frame size is with shift as 2. The range of detection frame is half of time frame. Segments marked by the shadows are selected regions for D where the target speech signal is weak. Algorithm : FSE Overall Scheme Inpt: Acostic mixing signals, x j, j =,..., M (M 2) Otpt: Extracted speech sorce ŝ n, n [, N]. Activity Detection: Find drations of total length L D where speech sorce n is either weak or silent if Room reverberation and nmber of sorces are low then Apply split Bregman method directly to obtain filters jn, j =,..., M else Apply sbdivided split Bregman method to obtain filters jn, j =,..., M Speech Extraction: Calclate ŝ n = M jn x j. j= We first evalate the proposed FSE method, stdy the relation between the length of selected silent speech dration and the extraction qality, and compare the split Bregman algorithm with sbdivided split Bregman algorithm sing synthetically mixed data (two sensors and two sorces). [Setp ]: The room size is m, and the implse responses are measred by two omni-directional microphones (middle of the room and.5 m above the floor) with the spacing 5 cm. The sorces are m away from the sensors with the azimth 3 and 9, and the same height as sensors. The reverberation times of implse responses are from s (anechoic) to. s. In order to illstrate the separation qality and speed of or proposed method, we simplify the detection step by knowing roghly abot.5 s silent dration D (e.g. 2.3 s s for the speech sorce in the p-left panel of Fig. 4) of target speech sorce ahead of time. The other sorce (e.g. lower-left panel in Fig. 4) is either speech or backgrond msic. The dration of the sorces is 5 s and the sampling rate is 6 Hz. Two mixtres (e.g. two right panels in Fig.

6 Extracted sorce.4 Extracted sorce Otpt SIR [db] 5 5 Anechoic T 6 = 5 ms T 6 = 58 ms Fig Extracted two speech sorces from the two mixtres in Fig. 4 by FSE 5 T 6 = 78 ms T 6 = ms Inpt SIR [db] Fig. 3. Otpt SIR vs. inpt SIR for the proposed FSE method with different reverberation times Clean sorce Clean sorce Mixtre Mixtre Fig. 4. Clean speech sorce (p-left panel), backgrond interference (lowerleft panel) and two corresponding synthetical mixtres at T 6 = 5 ms (p-right and lower-right pannels). 4) are synthesized by measred RIRs according to (). As the reverberation time goes p, the length of soltion (e.g. sparse soltion with 4 taps in Fig. 6) increases accordingly from 4 taps to 2 taps. Shown in Fig. 3 are the average otpt signal to interference ratios (SIRs) achieved by FSE for the varios reverberation times and inpt SIRs. Extracted speech sorces in two channels are shown in Fig. 5, corresponding to the two sorces in Fig. 4. With different lengths of selected silent speech drations, FSE achieves varios separation qalities, seen in Fig. 7. Basically, the separation effect is consistently improved with the increasing size of the silence region D (.5 s,.3 s,.45 s,.6 s and.75 s). The separation reaches a platea at.5 s. Length of.5 s total silence is an idea choice, which balances the comptational speed and separation qality. Table I illstrates the average iterations, comptation time [s] and SIR improvement (SIRI [db]) of the split Bregman algorithm and the sbdivided split Bregman algorithm by different lengths of nmixing filters. The data are synthetic mixtres of two sorces same as in [Setp ] with however the reverberation time T 6 = 78 ms and the inpt SIR 5.9 db. The comparison indicates that the sbdivided split Bregman (r = 2 here) performs better than the split Bregman if the length of nmixing filters is larger than 8 taps. When the length L is above 2, the split Bregman rns ot of memory. There is a trade-off between improved separation and Filter length Filter length Filter length Filter length Fig. 6. Sparse filters s with 4 taps, and 2 ( 2 and 22 ) are sed to estimate sorce (sorce 2) in Fig. 5. comptation costs. From Table I, L = 8 already achieves a good separation. TABLE I Comparison of the (divided) split Bregman algorithms Split Bregman Sbdivided Split Bregman L Iteration Time SIRI Iteration Time SIRI The comparison of a list of existing BSS methods is shown in Table II in terms of comptation time, SIR, signal to distortion ratio (SDR) and signal to artifact ratio (SAR). The data are synthetic mixtres of two speech sorces as in [Setp ] with reverberation time T 6 = 5 ms and inpt SIR 5.9 db. To compare the comptation time of the algorithms directly, the proposed FSE method extracts two speech sorces seqentially with the silent nions for the two speech sorces known ahead of time. Table II indicates that the proposed FSE achieves the best separation qality in objective measres at almost the speed of FastICA. Room recorded mixtre data are sed to evalate and compare the above BSS methods by the Perceptal Evalation

7 7 Average SIR improvement [db] Anechoic T 6 = 5 ms T 6 = 58 ms T 6 = 78 ms T 6 = ms Average length of silent dration [s] Reverberation time: 3 ms Sampling rate: 8 Hz Mic Sorce: male and female speeches, or speeches and msic Mic 2 with 7s dration.2m Room height: 2.5 m Mic 3 Lodspeaker.4 m height 4.5 m Omni-directional microphones.4 m height with 4 cm spacing S 5 o S 2 o S 3 7 o 3.6 m Fig. 7. The relationship between average separation effect SIR improvement (SIRI) and the length of selected silent speech dration with different reverberation times. The inpt SIR is abot -5.9 db. TABLE II Comparison of BSS methods on synthetic mixtre data Fig. 8. Configration and parameters of the room recording. TABLE IV Sbjective evalation on blind speech separation. Here > (<) means the otpt of or method is perceived better (worse) than the other method in terms of separation qality and voice clearness respectively, while means hard to distingish. Time [s] SIR [db] SDR [db] SAR [db] Parra [7] IVA [8] SNGTD [] FastICA [] FSE Method Test Category > < FSE vs IVA FSE vs DUET Separation 7.5% 4.8% 23.7% Clearness 53.3% 5.5% 4.2% Separation 65.3% 5.8% 28.9% Clearness 45.5% 2.4% 42.% of Speech Qality (PESQ) [9]. [Setp 2]: The room size is m with reverberation time T 6 = 3 ms. The lodspeakers and omni-directional microphones are.4 m high from the floor. The sensors are set in the middle of the room with 4 cm spacing linearly arranged. For the two sensors and two sorces case, sorces come from speaker S and S 2, and Mic 2 and Mic 3 are trned on, see Fig. 8. For the case of three sensors and three sorces, all the speakers and microphones in Fig. 8 are inclded. The mixtre data are male and female speeches with the dration abot 7 s and sampling rate 8 Hz. Now with the sorce activity detection added, the separation qality of the proposed FSE exceeds those of the known methods, as seen from Table III. The speech sorces activity detection is done within 2 to 3 seconds, and does not affect the efficiency of the FSE method. DUET BSS method [2] is inclded in Table III as the microphone spacing is small enogh so that there is no phase-wrap ambigity to degrade its performance. TABLE III Average PESQ of BSS methods on real recording mixtre data. PRE PESQ is the average PESQ of the mixtre data. Time for FSE is shown as detection time + speech extraction time. 2 sorces (time[s]) 3 sorces (time[s]) PRE PESQ.37. Parra.57 (7.9).44 (6.) FastICA.9 (2.).7 (3.3) SNGTD 2.7 (2).88 (265) IVA 2.35 (49.) 2.2 (52.2) DUET 2.36 (2.2) 2. (4.3) FSE 2.58 (.9+2.4) 2.5 ( ) In the above objective evalations, IVA, DUET and FSE lead other approaches. For frther stdy, we evalate these three approaches by sbjective test. Mixtre data are collected in the same environment as [Setp 2], which contains both two sorces and three sorces cases. At lease one sorce is speech. Extracted speech sorces by three different methods are evalated by hman sbjects with normal hearing. We tilized the paired comparison (PC) test, which reqires each listener to rank the three methods according to the performance of separation qality and sond clearness. The preference percentages of or method to the other two methods is shown in Table IV, and they are calclated as # of pairs where F SE is better P C > = (2) # of all pairs in the test # of pairs where F SE is worse P C < = (2) # of all pairs in the test # of pairs where difference is not significant P C = # of all pairs in the test (22) Hman perception test confirms that the proposed FSE method otperforms the other BSS methods in terms of speech separation qality and clarity. VI. DISCUSSION AND CONCLUSION We proposed and evalated a fast and efficient blind speech extraction method as long as target speeches make pases. A convex optimization problem is formlated and solved by the split Bregman method to yield sparse nmixing filters. Binary mask blind speech separation method is modified to detect the speech sorce onset-offset activity. Experimental reslts

8 8 indicate that the proposed method otperforms conventional blind speech separation methods in terms of the overall comptation speed and separation qality. The limitation of the proposed method is that it relies on a robst silence detection in a long reverberation mlti-talker environment which will be stdied frther in ftre work. Meng Y received his B.S in scientific & engineering compting at Peking University in 27 and M.S in comptational and applied mathematics at University of California, Irvine in 29. He is a Ph.D. candidate in acostic speech and voice signal processing. ACKNOWLEDGMENT The athors wold like to thank Yang Wang for helpfl discssions. REFERENCES [] S. Makino et al. (eds.), Blind Speech Separation, Springer 27. [2] J. Xin, M. Y, Y. Qi, H. Yang, and F-G Zeng, A nonlocally weighted soft-constrained natral gradient algorithm for blind sorce separation of reverberant speech, IEEE Workshop on Application of Signal Processing to Adio and Acostics, 8-84, Oct. 29, New Paltz, NY, USA. [3] O. Yilmaz and S. Rickard, Blind separation of speech mixtres via timefreqency masking, IEEE Trans. Signal Processing, vol. 52, no. 7, pp , Jly 24. [4] S. Araki, H. Sawada, R. Mkai and S. Makino, Underdetermined blind sparse sorce separation for arbitrarily arranged mltiple sensors, Signal Processing, 87, , 27. [5] T. Goldstein and S. Osher, The split Bregman algorithm for L reglarized problems, SIAM J. Imaging Sci., 2(2), , 29. [6] L. Tong, G. X, T. Kailath, Blind identification and eqalization based on second order statistics: A time domain approach, IEEE Information Theory, 4(2):34-349, 994. [7] Y. Wang, Z. Zho, Backgrond sppression in adio throgh learning, in preparation, 2. [8] J. Allen, D. Berkley. Image method for efficiently simlating small-room acostics. J. Acostical Society America, 65:943-95, 979. [9] D. Dttweiler. Proportionate normalized least-mean-sqares adaptation in echo cancelers. IEEE Trans. Speech Adio Processing, 8:58-58, 2. [] L. Bregman, The relaxation method of finding the common points of convex sets and its application to the soltion of problems in convex programming, USSR Compt Math and Math. Phys., v7:2-27, 967. [] S. Osher, M. Brger, D. Goldfarb, J. X, and W. Yin, An iterative reglarization method for total variation based image restoration, SIAM Mltiscale Model. and Sim., 4:46-489, 25. [2] L. Rdin, S. Osher, E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D, 6, , 992. [3] W. Yin, S. Osher, D. Goldfarb, and J. Darbon, Bregman iterativealgorithms for l -minimization with application to compressed sensing, SIAM J. Imaging Sci., ():43-68, 28. [4] E. Esser, Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman, CAM report, 9-3, UCLA, 29. [5] J. Cai, S. Osher and Z. Shen, Split Bregman Methods and Frame Based Image Restoration, Mltiscale Model. Siml. 8(2): , 29. [6] S. Araki, S. Makino, H. Sawada, and R. Mkai, Redcing msical noise by a fine-shift overlap-add method applied to sorce separation sing a time-freqency mask, in Proc. ICASSP25, Mar. 25, vol. III, pp [7] L. Parra and C. Spence, Convoltive blind separation of non-stationary sorces, IEEE Trans. Speech Adio Processing, vol. 8, no. 3, , May 2. [8] T. Kim, H. Attias, S-Y Lee, and T-W Lee, Blind sorce separation exploiting higher-order freqency dependencies, IEEE Trans. Adio, Speech Langage Processing, vol. 5, no., pp 7-79, 27. [9] ITU-T Rec. P. 862, Perceptal evalation of speech qality (PESQ), an objective method for end-to-end speech qality assessment of narrowband telephone networks and speech codecs, International Telecommnication Union, Geneva, 2. [2] A. Jorjine, S. Rickard, and O. Yilmaz, Blind separation of disjoint orthogonal signals: Demixing N sorces from 2 mixtres, in Proc. ICASSP 2, vol. 2, , 2. Wenye Ma received the B.S. and M.S. degree in mathematics from University of Science and Technology of China, in 24 and 27, and the M.A. in mathematics from University of California, Los Angeles in 29. He is crrently working towards the Ph.D. degree in mathematics at Unversity of California, Los Angeles. His research interests inclde optimization and its applications to image and signal analysis. Jack Xin received his B.S in comptational mathematics at Peking University in 985, M.S and Ph.D in applied mathematics at New York University in 988 and 99. He was a postdoctoral fellow at Berkeley and Princeton in 99 and 992. He was assistant and associate professor of mathematics at the University of Arizona from 99 to 999. He was a professor of mathematics from 999 to 25 at the University of Texas at Astin. He has been a professor of mathematics in the Department of Mathematics, Center for Hearing Research, Institte for Mathematical Behavioral Sciences, and Center for Mathematical and Comptational Biology at UC Irvine since 25. He is a fellow of the John S. Gggenheim Fondation. His research interests inclde applied analysis and comptation in nonlinear and mltiscale problems, mathematical modeling in speech and hearing sciences, and sond signal processing. Stanley Osher received his Phd degree in 966 from New York University s Corant Institte of Mathematical Sciences. He is a Professor of Mathematics, Compter Science and Electrical Engineering at UCLA. He is also an Associate Director of the NSF fnded Institte for Pre and Applied Mathematics. He is a member of the National Academy of Sciences, the American Academy of Arts and Sciences and is one of the top 25 most highly cited researchers in mathematics and compter sciences. He has received nmeros academic honors and has co-fonded three sccessfl companies, each based largely on his own (joint) research. His crrent interests mainly involve image science.