An Approach For: Iprovng Voce Coand processor Based On Better Features and Classfers Selecton Mohaed FEZARI 1, Al Al-Dhoud 1.Laboratory of Autoatc and Sgnals, Annaba, BP.1, Annaba, 3, ALGERIA. Faculty of IT Al-Zaytoonah Unversty, Aan, Jordan Mohaed.fezar@uwe.ac.uk,aldahoud@alzaytoonah.edu.j Abstract: Detals of desgnng a voce coand syste based on cobnng features and ppelnng classfers s presented n ths work. Based on research and experental results, ore features wll ncrease the rate of recognton n ASR (autoatc Speech recognton). Thus cobnng classcal coponents used n autoatc speech recognton syste such as Crossng Zero, energy, MFCC wth wavelet transfor ( to extract eanngful forants paraeters) followed by a ppelnng ordered classfers GMM and HMM has contrbuted n reducng the error rate consderably. To pleent the approach on a real-te applcaton, a Personal Coputer nterface was desgned to control the oveents of a four degree of freedo robot ar by transttng the orders va rado frequency crcuts. The voce coand syste for the robot ar s desgned and tests showed an Iproveent by cobnng technques. Keywords: Speech recognton, wavelet transfor, GMM/HMM, ppelnng and hybrd technques, Huan-achne Councaton, wreless coand. 1. Introducton Robot ars, or anpulators, coprse a bllon dollar ndustry. Bolted at ts shoulder to a specfc poston n the assebly lne, the robot ar can ove wth great speed and accuracy to perfor repettve tasks such as spot weldng and pantng. In the electroncs ndustry, anpulators place surfaceounted coponents wth superhuan precson, akng the portable telephone and laptop coputer possble, teleoperated Robot ar are wdely used n space robotcs, bob dsposal, and reotely operated vehcles. [1] and []. Yet, for all of ther successes, these coercal robots suffer fro a fundaental dsadvantage: lack of huan voce control. Huan-robot voce nterface s an nterestng applcaton for elderly people and pared users. A fxed anpulator has a lted range of coands provded by a anpulator. Manly usng a joystck, keyboard or a ouse. Ths paper proposes a new voce coand syste (VCS) for teleoperatng a robot ar. Based on the recognton of solated words, usng a set of tradtonal pattern recognton approaches and a dscrnaton approach based ppelnng classcal ethods [][5] and [7] n order to ncrease the rate of recognton. The ncrease n coplexty as copared to the use of only tradtonal approach s neglgble, but the syste acheves consderable proveent n the atchng phase, thus facltatng the fnal decson and reducng the nuber of errors n decson taken by the VCS. It s only recently that the felds of robot ar control and AGV (Autoatc Guded Vehcle) navgaton have started to port soe of the exstng technques developed n AI for dealng wth nonstatonary nforaton. Hybrd ethod s a sple, robust technque developed to allow the groupng of soe basc technques advantages. It therefore ncreases the rate of recognton. The selected features are: Zero Crossng and Extrees (CZEXM), contnuous Wavelet transfor CWT specally Gabor wavelets, Energy Segents (ES), and cepstral coeffcents ( MFCC: Mel Frequency Cepstral coeffcents)[6]. The applcaton uses a set of coands n Arabc words n order to control the drectons of four DOF robot ar. It has to be pleented on prograable ntegrated crcuts [8] and has to be robust to As applcaton, a voce coand for a four DOF robot ar s chosen. The robot ar s TERGANE - TR45. There have been any research projects dealng wth robot control, aong these projects, there are soe projects that buld ntellgent systes [1-1]. Voce coand syste needs the recognton of solated words fro a lted vocabulary used n gudng a Robot Ar Control syste (RACS)[14]. the paper s presented as follow: n secton, we present the selected feature coputaton, n secton 3 the speech recognton agent s detaled based on GMM and HMM, the hardware part s presented n secton 4, fnally n secton 5 and 6 we present the results of tests sulaton and a concluson.. Gabor Wavelets and Speech Analyss In recent years, the wavelet transfor has been successfully appled to age and sgnal processng.1 Unlke the Fourer transfor, the wavelet transfor 31 can detect the characterstcs of local sgnals.
Therefore, the wavelet transfor s partcularly helpful for the analyss and processng of speech sgnal where local forants nforaton s portant. Gabor wavelets, whch are slar to the Mexcan hat wavelets provde a ore powerful tool for analyzng speech and usc. We shall frst go over ther defnton, and then llustrate ther use wth soe exaples. A Gabor wavelet, wth wdth paraeter w and frequency paraeter v_, s the followng analyzng wavelet: 1/ ( x / w) ** * x / w ( x) e.1 Ths wavelet s coplex valued. Its real part R(x) and agnary part I(x) are. ( x) w R ( x) w R 1/ 1/ e e ( x / w) ( x / w) cos(vx / w) sn(vx / w)..3 The wdth paraeter w plays the sae role as for the Mexcan hat wavelet; t controls the wdth of the regon over whch ost of the energy of (x) s concentrated. The value v/w s called the base frequency for a Gabor CWT. One advantage that Gabor wavelets have when analyzng sound sgnals s that they contan factors of cosnes and snus, as shown n (.) and (.3). These cosne and sne factors allow the Gabor wavelets to create easly nterpretable scalogras of those sgnals whch are cobnatons of cosnes and snus the ost coon nstances of such sgnals are recorded usc and speech. ore detals on ths s n [6-8], but frst we need to say a lttle ore about the CWT defned by a Gabor analyzng wavelet. Because a Gabor wavelet s coplex valued, t produces a coplex-valued CWT. For any sgnals, t s often suffcent to just exane the agntudes of the Gabor CWT values and take the as features. CWT were used to select soe prncples features wthn the uttered sgnal that can not be extracted by classcal technques these coponents wll ndeed ncrease the true postve counter however they wll add on calculaton perod. Alternatvely, Artfcal Neural Networks (ANN) based fraework has also been proposed [], but despte ther hgh dscrnaton ablty n short-te classfcaton tasks, they have proved neffcent when dealng wth long-ter speech segents. Wth the scope of solvng the proble of long te odelng of the ANN fraework, one of the ost successful alternatves to HMM/GMM was later proposed, coonly known as hybrd ANN/HMM or connectonst paradg []. In general, hybrd archtectures seek to ntegrate ANN ablty for estaton of Bayesan posteror probabltes nto a classcal HMM structure that pert odelng long-ter speech evoluton. 3.1 GMM technque The GMM can be vewed as a hybrd between paraetrc and non- paraetrc densty odels. Lke a paraetrc odel, t has structure and paraeters that control the behavor of densty n known ways. Lke non-paraetrc odel t has any degrees of freedo to allow arbtrary densty odelng. The GMM densty s defned as weghted su of Gaussan denstes: P ( x) W g( x,, C ).4 G, M the total nuber of Gaussan coponents. W are the coponent probabltes (_w = 1), also called weghts. We consder K-densonal denstes, so the arguent s a vector x = (x1,..., xk)t. The coponent probablty densty functon (pdf), g(x, _, C), s a K-densonal Gaussan probablty densty functon (pdf) gven n Equaton 7 as follows: g ( x, Where, C T 1 / ( x ) ) e.5 s the ean vector, and C s the covarance atrx. Organzng the data for nput to the GMM s portant snce the coponents of GMM play a vtal role n akng the word odels. For ths purpose, we use K- eans clusterng technque to break the data nto 56 cluster centrods. These centrods are then grouped nto sets of 3 and then passed nto each coponent of GMM. As a result, we obtan a set of 8 coponents of GMM. Once the coponent nputs are decded, the GMM odelng can be pleented ore detals are n [1]. GMM Models HMM Models Fgure1. a-mexcan hat CWT (scalogra) of a test sgnal wth two an frequences, b- Magntudes of Gabor scalogra of test sgnal. 3. Hybrde Technques Regardng the dfferent ASR paradgs proposed durng these years, Hdden Markov Models of Gaussan xtures (HMM/GMM) [19] s doubtless the ost wdely accepted approach. Features extracton ( EN,MFCC,CZEM) + CWT paraeters Input sgnal GMM N-class separator HMM N-class separator Decson 3 Fgure.a Pattern Matchng GMM then HMM And Weghtng vector
3. HMM bascs Over the past years, Hdden Markov Models have been wdely appled n several odels lke pattern [6], or speech recognton [6][9]. To use a HMM, we need a tranng phase and a test phase. For the tranng stage, we usually work wth the Bau-Welch algorth to estate the paraeters (π,a,b) for the HMM [7, 9]. Ths ethod s based on the axu lkelhood crteron. To copute the ost probable state sequence, the Vterb algorth s the ost sutable. A HMM odel s bascally a stochastc fnte state autoaton, whch generates an observaton strng, that s, the sequence of observaton vectors, O=O1,..Ot,,OT. Thus, a HMM odel conssts of a nuber of N states S={S} and of the observaton strng produced as a result of ettng a vector Ot for each successve transtons fro one state S to a state Sj. Ot s d denson and n the dscrete case takes ts values n a lbrary of M sybols. The state transton probablty dstrbuton between state S to Sj s A={aj}, and the observaton probablty dstrbuton of ettng any vector Ot at state Sj s gven by B={bj(Ot)}. The probablty dstrbuton of ntal state s Π={ π}. a P( q S q S ) (3.1) j t1 j t B={bj(Ot)}. (3.) P( q S ) (3.3) Gven an observaton O and a HMM odel λ=(a,b, ), the probablty of the observed sequence by the forward-backward procedure P(O/λ) can be coputed [1]. Consequently, the forward varable s defned as the probablty of the partal observaton sequence O,... 1O O t (untl te t) and the state S at te t, wth the odel λ as α(). and the backward varable s defned as the probablty of the partal observaton sequence fro t+1 to the end, gven state S at te t and the odel λ as β(). the probablty of the observaton sequence s coputed as follow: N P ( O / ) ( ) * ( ) ( ) t t T 1 1 and the probablty of beng n state I at te t, gven the observaton sequence O and the odel λ s coputed as follow: P( q S ) (3.5) 4. Proposed technque: Ppelnng Recognton Syste Models The an eleents are shown n the block dagra of Fgure 1. The pre-processng block s used to adapt the characterstcs of the nput sgnal to the recognton syste. It s essentally a set of flters, whose task s to enhance the characterstcs of the speech sgnal and N (3.4) 33 nze the effects of the background nose produced by the external condtons and the otor. Preprocessng Input Word SL Features Extracton ES, MFCC, CZEXM, CWT Decson Pattern Matchng Block GMM then HMM And Weghtng ppelnng Recognton vector Syste Block (PRS) Acton : 8 bts to transt by RF Reference Words pattern Fgure.b: BLOCK DIAGRAM OF THE PROPOSED DESIGN The SL pleented s based on analyss of crossng zero ponts and energy of the sgnal, the lnear predcton ean square error coputaton helps n ltng the begnnng and the end of a word; ths akes t coputatonally qute sple. The paraeter extracton block analyses the sgnal, extractng a set of paraeters wth whch to perfor the recognton process. Frst, the sgnal s analysed as a block, the sgnal s analysed over s fraes, at 56 saples per frae. Fve types of paraeters are extracted: Noralzed Extrees Rate wth Noralzed Zero Crossng Rate (CZEM),, Gabor wavelets coeffcents, Energy Segents (ES) and MFCC s [4]. These paraeters were chosen for coputatonal splcty reasons (CZEXM, ES), robustness to background nose (1 Cepstral paraeters) and robustness to speaker rhyth varaton (DTWE) [7], Gabor wavelets for extractng useful paraeters whch are forants. The reference pattern block s created durng the tranng phase of the applcaton, where the user s asked to enter 5 tes each coand word. For each word and based on the ten repettons, ten vectors of paraeters are extracted fro each segent and stored. The atchng block, by applcaton of GMM classfer then HMM odels n order, t copares the reference patterns and those extracted fro the nput sgnal n both ethods. The atchng decson ntegrate: a hybrd recognton block based on fve ethods, and a weghtng vector. The value of ths vector s then taken fro the results of theses ethods. Exaple : f GMM and and then HMM recognse the frst word n table 1 then decson block wll recognse the frst word as uttered. Tests were ade usng each ethod GMM and HMM separately. Fro the results obtaned, a weghtng vector s extracted based on the rate of recognton for each ethod. Fgure 1 shows the eleents akng up the an blocks for the hybrd recognton syste (HRS). A voce coand syste and an nterface to the robot ar are pleented. The voce coands conssts of Arabc words used n oble robot control
5. Experents on the Syste The corpus s based on 5 students fro 1 st and nd year Master Degree of our departent, they have already study a course on robotcs and have a lecture on Huan- Machne Councaton, 4 students were used for tranng phases and 1 students were used for tests. The developed syste has been tested wthn the laboratory of L.A.S.A. The tests were done only on the fve coand words. Three dfferent condtons were tested: 1-The rate of recognton usng the classcal ethods wth classcal paraeters used n ASR (ES, MFCC)and GMM as classfer.. - The rate of recognton usng the classcal ethods wth classcal paraeters used n ASR (ES, MFCC)and HMM as classfer. 3- The rate of recognton wth enhanceent of paraeters used n ASR (ES, MFCC, CWT) and GMM as classfer 4-The rate of recognton wth enhanceent of paraeters used n ASR (ES, MFCC, CWT) and HMM as classfer 5-The rate of recognton wth enhanceent of paraeters used n ASR (ES, MFCC, CWT) and cobng GMM and HMM as classfer 6-And the effect of coputaton te due to cobnaton of dfferent technques. For the four frst tests, each coand word s uttered 5 tes. The recognton rate for each word s presented n Table 1 Fg.4.q presents the proveent usng cobned features and classfers. Fg. 4.b, represents the coputaton te for each cobnaton (hybrdzng classfers), t s clear that the coputaton te s long by usng ore data paraeters to process or ore classfers n ppelne to select Table1. results of test on dfferent cobnatons Technques Secon d Fourt h Words Frst Thrd ffth Aa e 6 7 7 75 87 Wara 65 71 74 8 87 Yane 7 75 75 76 86 Yassar 68 7 76 76 89 Kf 75 78 8 84 9 9 91 9 89 88 87 86 85 84 83 results of ppelnng technques and cobnng features Aae Wara Yane Yassar Kf EN+MFCC+WCT, GMM+HMM Fgure 4.a Results of Cobnng technques 3,5 1,5 1,5 1 EN+MFCC, GMM EN+MFCC,HMM EN+MFCC+CWT,GMM EN+MFCC+CWT,HMM Fgure. 4.b The effect coputaton te EN+MFCC+CWT,GMM+HM M 6. Concluson And Future Work Evaluatons show the nterest of.teleoperatng ar anpulator for users' even handcapped people. Snce the desgned robot conssts of a crocontroller and other low-cost coponents naely RF transtters, the hardware desgn can easly be carred out. The results of the tests shows that a better recognton rate can be acheved nsde the laboratory and especally f the phonees of the selected word for voce coand are qute dfferent. However, a good poston of the crophone and addtonal flterng ay enhance the recognton rate. Several nterestng applcatons of the proposed syste dfferent fro prevous ones are possble as entoned n secton 5. Besde the desgned applcaton, a hybrd approach to the pleentaton of an solated word recognton agent HSR was used. Ths approach can be pleented easly wthn a DSP or a CMOS DSP crocontroller or even FPGA odules. The use of hybrd technque based on classcal recognton ethods akes t easer to separate the class represented by the varous words, thus splfyng the task of the fnal decson block. Tests carred out have shown an proveent n perforance, n ters of sclassfcaton of the words pronounced by the user. Segentaton of the word n three prncpal fraes for the Zero Crossng and Extrees ethod gves better results n recognton rate. Another lne of research wll be nvestgated n the future relates to natural language recognton based on seantc and large data base. References [1] Nguyen L., Belouchran A., Abed-Mera K. and Boa-shash B., Separatng ore sources than sensors usng te- frequency dstrbutons. n Proc. ISSPA, Malysa, 1. [] M Mokhtar., Ghorbal M. and Kadouche R., Moblté et Servces : applcaton aux ades technologques pour les personnes handcapées», n procedngs 7th ISPSAlgers, ay, 5, pp. 3948, 5. [3] P. Arbelle, J Ruz, M. Chevllet, Teleleoperated robotc Ar for Echographc Dagnoss n Obstetrcs and Gynecology, Untrasound In Obstetrcs and Gynecololy Vol 4. No.3 pp. 4-43, August 4.. [4] Gu L. and Rose K., "Perceptual Haronc Cepstral Coeffcents for Speech Recognton n Nosy Envronent," Proc ICASSP 1, Salt Lake Cty, Utah, May 1. [5] C. pasca, P. Payeur, E.M. Petru, A.M. Cretu, 34 Intellgent Haptc Sensor Syste for Robotc
anpulator, n Porc. IMTC/4, IEEE Instru. Meas. Technol. Conf. pp. 79-84, Italy 4. [6] Tony F. Chan and Jacke (Janhong) Shen, Iage Processng and Analyss - Varatonal, PDE, Wavelet, and Stochastc Methods, Socety of Appled Matheatcs, ISBN 89871589X (5) [7] Ingrd Daubeches, Ten Lectures on Wavelets, Socety for Industral and Appled Matheatcs, 199, ISBN -89871-74- [8] Hongyu L. Y. Zhao, Y. Da and Z. Wang,. A secure Voce Councaton Syste Based on DSP, IEEE 8 th Internatonal Conf. on Cont. Atau. Robotc and Vson, Kunng, Chna, 4, pp. 13-137. [9] Ferrer M.A., I. Alonso, C. Traveso, Influence of ntalzaton and Stop Crtera on HMM based recognzers, Electroncs letters of IEE,, Vol. 36, pp.1165-1166 [1] Vtran M., Morel G., and Ortaer T., Autoatc gudance of a surgcal nstruent wth ultrasound based vsual servong, n Proc. IEEE Int. Conf. on Robotcs and Autoaton, Barcelona, Span, Aprl 5. [11] Reynolds D., Autoatc Speaker Recognton, Acoustcs and Beyond, Senor Meber of Techncal Staff, MIT Lncoln Laboratory, JHU CLSP, 1 July. [1]. P.S. Addson. (5). Wavelet transfors and the ECG: a revew. Phys- ol. Meas., Vol. 6, R155 R199. [13] Kwee H., Stanger C.A., The Manus robot ar, Rehabltaton Robotcs News letter, Vol. 5, No, 1993, H. G. Evers, E. Beugels and G. Peters, MANUS toward new decade, Proc. ICORR 1, Vol. 9, pp. 155-161, 1 [14] Evers H. G., Beugels E. and Peters G., MANUS toward new decade, Proc. ICORR 1, Vol. 9, pp. 155-161, 1 [15] Data sheet PIC16F876 fro Mcrochp nc. User s Manual, 3, Avalable at: http://www.crochp.co. [16] Radoetrx coponents, TX-433 and SILRX- 433 Manual, HF Electroncs Copany. Avalable at: http://www.radoetrx.co. [17] K W. J., et al., Developent of A voce reote control syste. Proceedngs of the 1998 Korea Autoatc Control Conference, Pusan, Korea, Oct. 1998, pp. 141-144, 1998. [18] Mahoney RM, Jackson RD, Darge GD. An Interactve Robot Quanttatve Assessent Test. Proceedngs of RESNA 9. 11-11. 199. [19] Bourlard, H., Morgan, N.: Connectonst Speech Recognton: a Hybrd Approach. Boston: Kluwer Acadec, Norwell, MA (USA) (1994) []Ch. Neukrchen, G. Rgoll: "Advanced Tranng Methods and New Network Topologes for Hybrd MMI-Connectonst/HMM Speech Recognton Systes", Proc. IEEE-ICASSP, Munch, 1997, pp. 357-36. 35 [1]I. El-EMARY,. FEZARI a nd H. ATOUI," Hdden Markov odel/gaussan xture odels (HMM/GMM) based voce coand syste", n Scentfc Research and Essays Vol. 6(), pp. 341-35, 18 January, 11