RELIABLE DETECTION OF SOUND S DIRECTION FOR ACTIVE AUDITION

Transcription

1 RELIABLE DETECTION OF SOUND S DIRECTION FOR ACTIVE AUDITION Hyun-Don Kim, Jon-Suk Choi, *Chan-Hoon Lee and Munsan Kim Intellient Robotics Research Center Korea Institute of Science and Technoloy *Department of Information Communication Enineerin, Paichai Uni., Taejon, Korea {reynolds, cjs, munsan}@kist.re.kr, *nairo@pcu..kr Abstrt: In this paper, we propose reliable detection of sound s direction for human robot intertion. Compared with preious researches, this system comprises simpler alorithm and a proposed nonlinear amplifier which has adantaes to increase a detectible distance of sound sinal in spite of simple circuits. Moreoer, we propose the new performance index usin cross-correlation to make reliable detection of sound s direction. Also, since this system include a function to detect a period of oice sinals, the robot can know when to start findin sound s direction and performin speech reconition automatically. In order to erify our system's performances, we install the proposed audition system to the prototype robot, called IROBAA (Intellient ROBot for Actie Audition), and describe the experimental results. Keywords : sound localization, oice tiity detection, human robot intertion, intellient robot 1. INTRODUCTION The speech reconition has been applied into arious systems and its performance has been improed reatly. In addition to the reconition, sound localization becomes a technoloy of much interest in the research field of human-robot intertion. In order to reconize speech with hih confidence, the techniques which separate speech from arious sound and remoe noise from the sinal of speech hae been receied a reat deal of attention. Also, humanoid robots interated with computer ision and arious sensors hae been deeloped for similar behaiors of human [1]-[1]. The objectie of this research is to deelop the techniques which enable the followin scenario. Our robot reconizes its name and the direction of sound as well when a man calls. Then it turns its fe to the direction, and can possibly reconize the person throuh fe reconition technique to communicate with him. We use nonlinear amplification as a preprocessor in order that speech reconition system can reconize speech at a lon distance robustly aainst noises in the enironment. Furthermore, to make reliable detection of sound s direction, we propose a new performance index usin differences of crosscorrelation. Also, since this system includes a function of VAD (Voice Actiity Detection), the robot can know when to start findin the direction and performin speech reconition automatically. To erify our system s feasibility, the proposed audition system is installed in the prototype robot, called IROBAA (Intellient ROBot for Actie Audition), which has been deeloped at the KIST (Korea Institute of Science and Technoloy). Fi. 1 shows the audition system installed in IROBAA. The audition system is composed of preamplifier board, mic-mounted circle pad, commercial AD conerter, and a notebook computer to execute our proram. The AD conerter samples data from three microphones to the rate of 11 khz for eh. Microphone Fiure 1. Actie audition system installed in IROBAA. 2. Audition System of IROBAA Nonlinear amplification which is able to make dynamically ariable amplification cordin to the sinal manitude is required to increase the rane of detectable distance. If the ratio of amplification is fixed to small one, the sinal of speech occurrin at the lon distance can be hardly extrted from its receied sinal whose manitude is small enouh for the contents of speech to be canceled by noise. To the contrary, with lare ratio, the sinal occurrin nearby sometimes may be saturated in the AD conersion. For this reason, the speech reconition system which

2 is less affected by the distance to sound s source is necessary. To resole this problem, we propose the use of SSM2166, made by Analo Deice Corporation, which enables the nonlinear amplification. Our circuit, as shown in Fi. 2, is adjusted to compression ratio of 5:1. detectable distance of sound sinal and to reduce calculation time so as to execute arious filterin alorithms such as low or hih pass filterin. Fiure 4. Compared two data of a distance of 1.5m Fiure 2. Circuit of Nonlinear Pre-Amp In order to erify our nonlinear amplifier's performance, we should perform experiments to compare with normal linear amplifier. In the Fis. 3 and 4, the left hand shows sampled sinals amplified by linear amplifier and the riht hand shows sampled sinals amplified by proposed nonlinear amplifier. Eery source of speech sinals is the same. Besides, main computer performs normalization within a rane of ±.5V so that they are suited to speech reconition. 3. Trkin of Sound s Direction This paper uses DOA (Delay Of Arrial) for trkin the direction of sound [6]-[1]. DOA is the method that uses a time-delay from the source of sound to eh microphone. Een thouh the time delay is short, the difference of arrial time occurs between array-shaped microphones. In Fi. 5, three microphones are arraned such that their distances from the center of trianular rod are the same. Two couples of A s. C and B s. C are selected in the iew point of C. Note that the samplin data has maximum delay of time when a sound enters straihtly throuh both A and C, or B and C. Fiure 3. Compared two data of a distance of.5m Fi. 3 shows speech sinals outputted at.5m distance. This fiure shows that linear sinals are more noise and unclear shape than nonlinear sinals. Fi. 4 shows speech sinals outputted at 1.5m distance. This fiure shows that because the linearly amplified sinals (left) are far smaller than nonlinearly amplified sinals (riht), they can be canceled with noise sinal. Ultimately, as a preamplifier board is made with a proposed nonlinear circuit, we can et adantaes to increase a Fiure 5. Location of three mirophones In this case, the relatie distance correspondin to the maximum delay is defined as l (or l bc ). Also, the distance between sound s source and mic. A (mic. C) is defined as l sa (or l sc ). The elocity of sound and samplin frequency are defined as and F s

3 respectiely. The number of samplin about the maximum delay is defined by (1) and (2) where n is the number of samplin of maximum delay between A s. C microphone and n bc is the other one between B s. C microphone. n n l lbc (2) (1) bc The relation coefficient between mic. C and mic. A is defined by (3). Also, The relation coefficient between mic. C and mic. B is defined by (4). The ariable t is a taret number of delay in the th samplin period. Equation (3) and (4) is considered by samplin data from = to =. Howeer, the real application of infinite period is impossible. Therefore, ariable t is determined by suitable samplin data. We should decide the optimal samplin period consisted of 552 samples throuh experiments. R R bc ( k) = ( k) = { At ( kct ) ( )} = 2 2 A( t k) C( t) = = { Bt ( kct ) ( )} = 2 2 B( t k) C( t) = = (3) (4) The ariable k represents the number of tual delay samples. The number of delay k, in our confiuration, spans to the rane of -n ~ n in this (3) and -n bc ~ n bc in this (4) where its positie/neatie alue means that the sound enters microphone A and B earlier/later than microphone C. Now, sound s direction should be calculated usin relation coefficient R and R bc for all possible k and k bc. Fi. 5 illustrates the number of delay samples and the tual anle of sound s direction. An tual delay of sound s direction is expressed as (5) and (6). k k bc ( lsc lsa ) ( lsc lsb) (5) (6) Howeer, we can t know the location of sound source (θ, d) yet. Therefore, the followin method is proposed to estimate the sound source location. Matrix r presents the cross correlation of R and R bc for all possible k and k bc. All alues of matrix r are calculated by (7). r( θ ) = R[ k( θ)] Rbc[ kbc( θ)] where 1 θ 36 i.e. θ=1,2,...,36 (7) Next, because we want to find the anle of sound s direction, we should first know the maximum alue in the matrix r. After we fix threshold alue in the r by usin (8), we perform normalization to the r by usin (9). rthr =.99 max{ r( θ)} where 1 θ 36 i.e. θ=1,2,...,36 r() θ = if r() θ < rthr (() rθ rthr ) if r() θ r ( r r ) max thr thr where 1 θ 36 i.e. θ=1,2,...,36 (8) (9) And, if we perform a weihted aerae to the r by usin (1), we will find the anle of sound s direction. 36 (() r θ θ) θ = 1 = θ 36 sd (1) r( θ ) θ = 1 4. Reliable Detection of Sound s Direction In a real speech sinal, as there are reerberations, noise sinals and consonants which hae weakly periodic sinals, wron detections of sound s directions are calculated by computer frequently. Therefore, in order to find curate directions of speech sinal, we should detect sound s direction at the frame which has maximum enery within a period of speech sinal. The enery of a frame is expressed as (11). 1 = () (11) k 2 Eframe x i k i = The x(i) is a samplin data of i-th in speech sinals. Howeer, a method usin frame enery has seeral problems. First, if much noise is included in a speech sinal, it will be able to select a frame which is not a period of speech sinal. Second, because the frame hain a maximum enery has not always ood data to find an curate direction of sound, cury related to detectin sound s direction can be reduced.

4 To fix these problems, we propose a new performance index rather than the frame enery. Gien eh frame, the performance index is expressed as (12). P= r r (12) max We e found a notable feature throuh lots of experimental inestiation: it is true that when we spread alues calculated by usin (7) on the rane of all anles, the difference between manitudes of the cross-correlation is ery informatie to find reliable detection of sound s direction. After selectin the reference frame hain the maximum alue of our performance index P in a sample period, we decide direction whose cross-correlation alue is the maximum at the selected frame as the final result. To compare a frame enery method with a crosscorrelation method, we used three commands such as o to a liin room, o to a bi room, and patrol my home. Generatin spots of eh command were total 13 points at a distance of 1 meter. The azimuth which ranes from -9 to 9 was diided by eery 15. Table I is the aerae of experimental results. min Table I. Compare frame enery with proposed index Method Aera e Successful detection of sound s direction Frame Proposed Enery Anle error of sound s direction Frame Enery Proposed 82% 97% As you see Table I, the cross-correlation method is better in the percentae of successful detection and aerae of anle error than the frame enery method. Fiure 6 illustrates a 3 dimension raph which consists of numerical alues calculated by usin (7). At this time, used speech command is patrol my home comin at a distance of 1 meter and 3. Where a frame has the proposed performance index which hae the larest alue throuhout all the frames (See the inside of blue circle in Fi. 6), we can find an curate direction of sound. 5. Voice Actiity Detection For the purpose of effectie intertion between human bein and a robot, it is necessary to extrt the period in which only oice sinals are included: Nonoice or silent periods are unnecessary or harmful. Therefore, we propose a function of VAD (Voice Actiity Detection) usin autocorrelation method to find pitch information. IROBAA executes a reliable detection of sound s direction and speech reconition when the robot is decided to detect sinals of oice by VAD method. Pitch information rather than enery is applied to the VAD since the former has the adantae of a robust feature aainst noises [11][12]. Beside, In order to detect a pitch, we use an autocorrelation method which is composed of simpler alorithm instead of FFT. The frequency of a ocal cord concernin human bein exists in the rane between 5 and 25Hz in case of a male and between 12 and 5Hz in case of a female. Therefore, if we put 552 samples per one frame into the autocorrelation equation, the executed sinal will show pitch hain periodic form of human ocal cord. The equation of the autocorrelation is expressed as (13). R ( k) = cc 6 { Ct ( kct ) ( )} = Ct ( k) Ct ( ) = = (13) Then, after we perform a medium filter which has excellent features in remoin impulse noise, ede sinal preseration and smoothin, we can calculate differential alues about autocorrelation results. In case of real oice sinals, since manitude of a periodic form and differential alues between sampled sinals is lare, the peak alues can be calculated by applyin threshold alue to differential alues. Finally, as we can know the number of samples between two peak sinals, the pitch can be detected by (14). To improe cury of VAD, we should also detect the second pitch in a frame. Samplin Frequency Pitch = (14) A number of samlpes between the two peaks Fiure 6. The 3D raph of cross-correlation Now, after makin weihted sum of the calculated pitches of 2 frames, we can infer extrtin the period of oice sinal. Since the A/D conerter which is installed in IROBAA has the function of double bufferin, the robot can continuously execute the VAD alorithm at a second interals without loss of raw data. Consequently, it can automatically and continuously perform findin direction of sound and speech

5 reconition wheneer speech commands enter to microphones. 6. Speech Reconition System For the purpose of human-robot intertion, it is necessary for performin speech reconition and synthesis as well as detection of sound s direction. IROBAA performs speech reconition usin a commercial enine, howeer, with no additional mic. for the reconition only. To reconize speech reliably, we input pre-processed sinal of speech to L&H enine. Fi. 7 shows the block diaram of the speech reconition system. Fiure 8. Experiment Setup Table II is experimental results at 1m distance. Three commands are mostly used commands in IROBAA: Go to a liin room, Go to a bi room, and Patrol my home. Table II. The results of experiment at 1m distance Fiure 7. Block diaram of the speech reconition system The speech synthesis of IROAA uses commercial speech synthesis enine which outputs the most natural speech throuh a speaker. 7. Experimental Results Fi. 8 shows experiment setup. This experiment was conducted in an ordinary room, where bkround noise of about 55dB is enerated by computer fans, an air-conditioner and motor noise in throuh the robot. Spots for sound s sources are total 24 points (See the red points in Fi. 8). The anle is diided by eery 15 azimuth at a distance of 1 meter. The computer s speaker emits commands which were preiously recorded in about 9dB olume. If the error of detected sound s direction is up to 3, the result will be rearded as failure. Command Anle Go to a liin room reco nition direc tion Go to a bi room reco nition direc tion Patrol my home reco nition direc tion FAIL 352 FAIL 352 FAIL FAIL 15 OK 13 OK 352 FAIL 13 3 OK 21 OK 2 OK OK 36 FAIL 44 FAIL 43 6 FAIL 62 OK 72 OK FAIL 83 OK 76 OK 76 9 OK 9 OK 95 FAIL OK 14 OK 18 FAIL OK FAIL OK FAIL FAIL OK FAIL OK 138 OK FAIL 15 OK FAIL OK 155 FAIL FAIL 165 FAIL 165 FAIL OK 24 OK 194 FAIL OK 199 OK 199 OK OK 25 OK 25 OK OK 221 FAIL 227 OK FAIL 239 FAIL 239 OK FAIL 26 FAIL 256 OK OK 268 FAIL 276 OK OK FAIL OK 28 OK 28 3 OK 34 OK 288 OK FAIL 316 FAIL 331 OK FAIL 331 FAIL 331 FAIL OK 347 OK 342 FAIL 347 Successful Aerae Sound s Direction Anle Error Speech Reconition 9.3% %

6 This results show excellent performance at the short distance (1m): percentae of successful sound s direction is 9.3%. Moreoer, the aerae of errors about the estimated sound s direction is 5.1 correspondin to three commands. Howeer, percentae of successful speech reconition is not so ood. This may result from the limited performance of speech reconition enine and the incompleteness in extrtin ext period of oice from VAD. Therefore, we should analyze and improe the VAD alorithm to increase successful rate of speech reconition in the interated system. 8. Conclusions In this paper, conentional form of array-typed microphones is aoided. Also, simple and reliable alorithm with new pre-processin hardware is deeloped such that we are able to find the direction of sound s source from entire azimuth by usin three microphones. Furthermore, it makes possible to perform speech reconition without another specific microphone (i.e., wireless or unidirectional sensitie microphone). The audition system of IROBAA is desined for the optimized performance in the intertion between a human bein and a robot. Consequently, this system has some distinuished functions. First, usin the proposed pre-amplifier with simple circuits, we can et adantaes to increase the detectible distance of sound s sinal and to reduce noise. Second, trkin of sound source direction is often obstructed by echo sound due to surrounded obstles. Moreoer, as there are noise sinals and consonants which hae weakly periodic sinals in the sampled speech sinals, wron detections of sound s directions may be calculated by computer frequently. In order to sole the problem, we proposed the new method to select the frame which has ood information to find sounds direction (The fame is composed of 552 speech sampled data) based on cross-correlation. Third, as we apply VAD usin autocorrelation, it can automatically and continuously perform findin direction of sound and speech reconition wheneer speech commands enter to microphones. Finally, for the purpose of effectie intertion between a human bein and a robot, we hae interated functions bein able to perform speech reconition and synthesis. For further application to the real life, the system should extrt the desired sinal when oices of seeral people are mixed. Also, it should eliminate the noises een thouh lare ones are mixed. REFERENCES [1] J. Huan, N. Ohnishi, N. Suie, Sound localization in reerberant enironment based on the model of the precedence effect", IEEE Trans., Vol. 46, pp , Au [2] J. Huan, T. Supaonprapa, I. Terakura, N. Ohnishi, N. Suie, "Mobile robot and sound localization" IEEE/RSJ, Vol. 2, pp , Sept [3] J. Huan, N. Ohnishi, N. Suie, "Spatial localization of sound sources: azimuth and eleation estimation" IEEE/IMTC, Vol. 1, pp , May [4] J. Huan, K. Kume, A. Saji, Robotics spatial sound localization and its 3D sound human interfe" Cyber Worlds, pp , No. 22. [5] J. Huan, N. Ohnishi, N. Suie, "A biomimetic system for localization and separation of multiple sound sources", IEEE/IMTC, Vol. 2, pp , May [6] H. D. Kim, J. S. Choi, C. H. Lee, G. T. Park, M. S. Kim, "Sound's direction Detection and Speech Reconition System for Humanoid Actie Audition", ICCAS23 Int. conference, Oct. 23. [7] H. D. Kim, J. S. Choi, C. H. Lee, G. T. Park, M. S. Kim, Humanoid Actie Audition System Usin the Fuzzy Loic System, Journal of Control, Auto., and Sys. En., Vol. 9, No. 5, May, 23. [8] K. Nakadai, T. Matsui, H. G. Okuno, H. Kitano, Actie Audition System and Humanoid Exterior Desin, IEEE/RSJ ol. 2, pp , 2. [9] K. Nakadai, H. G. Okuno, H. Kitano, Epipolar Geometry Based Sound Localization and Extrtion for Humanoid Audition", IEEE/RSJ ol. 3, pp , 21. [1] S. A. Sekmen, M., Wilkes, K. Kawamura, "An Application of Passie Human-Robot Intertion: Human Trkin Based on Attention Distrtion", IEEE Trans. Sys., Man and Cybern., ol. 32, no. 2, pp , 22. [11] R. V. Prasad, A. Sanwan, H. S. Jamadani, Chiranth M. C, Rahul S, Comparison of Voice Actiity Detection Alorithms for VoIP, IEEE/ISCC 2, pp , 22. [12] G. Monti, M. Sandler, Monophonic Transcription with Autocorrelation, GOST G-6 Conference, Dec. 2.