PERFORMANCE OF THE ETSI DISTRIBUTED SPEECH RECOGNITION ALGORITHM OVER GSM AND IP NETWORKS

Transcription

1 PERFORMANCE OF THE ETSI DISTRIBUTED SPEECH RECOGNITION ALGORITHM OVER GSM AND IP NETWORKS Olav Skundberg 1, Jamil Y. Khan 2, Graham Wade 2 1 Sør-Trøndelag University College, Trondheim, Norway 2 School of Electrical Engineering & Computer Science, University of Newcastle, Callaghan, NSW 2308 ABSTRACT: The purpose of Distributed Speech Recognition (DSR) is to improve the performance of speech recognition services across communication networks. The DSR method is to extract feature vectors at a front-end terminal and transmit them as data with error protection to a back-end server. The ETSI STQ-AURORA DSR working group has standardised such a method. In this paper the ETSI DSR algorithm is tested over GSM and IP networks with varying bit error rates and packet loss conditions. A GSM channel with poor conditions (a bit error rate of 6.7e-3) is found to reduce the recognition performance by 6.3% compared to an errorfree signal. An IP network with poor conditions (15% packet loss) reduces the recognition performance by as little as 0.9%. The test results show that the ETSI DSR algorithm achieve good recognition performance over GSM and IP networks. 1. INTRODUCTION With the growth of telecommunication services and mobile and Internet applications, service providers probably will offer a significant number of new services based on speech recognition. These services may include automatic phone directories; call centre services and voice portals for information retrieval from the Internet. Currently used mobile terminals (mobile phones, PDA s etc.) have not got enough capacity to support medium or large-scale speech recognition tasks, therefore speech recognition tasks are performed on a centralised server. The mobile terminals traditionally would use a voice channel to connect to a centralised recognition server, but the performance of the recogniser could be poor because of the low bit rate voice coder and transmission errors [Pearce, 2000]. To overcome this hindrance, a new approach is to share some of the processing tasks between a front-end terminal and a back-end server. The DSR method is to extract speech feature vectors at front-end terminal and transmit them as data with error protection to a back-end speech recognition server. This paper evaluates the performance of the ETSI DSR algorithms over GSM and IP networks. The network conditions are varied to see how it affects recognition performance. GSM radio channel with bit errors IP network with packet loss Speech recognition server DSR front-end feature extraction and coding Gateway DSR backend decoding and error mitigation Figure 1. A mobile terminal communicating with a speech recognition server across GSM radio channel and IP packet network Accepted after abstract review page 468

2 2. DISTRIBUTED SPEECH RECOGNITION 2.1 DSR algorithms The STQ-AURORA DSR Working Group of European Telecommunications Standards Institute (ETSI) has standardised a DSR technique [ETSI 2000]. This paper is using the ETSI DSR V1.1.2 example software implementation. The software is distributed along with the ETSI standard and has three processing steps: Feature extraction, Compression and formatting, and Decoding and recovery. The codebooks used for compression and decompression is supplied with the ETSI DSR code DSR Feature vector extraction: The DSR method extracts speech feature vectors used for the recognition task from the original speech signal at a front-end terminal. The mel-frequency cepstral coefficients (MFCC) are extracted, where the vectors consist of 13 static cepstral coefficients and a log-energy coefficient. A vector is generated every 10 ms, independent of the sampling rate of the voice signal. DSR Compression and formatting: The extracted feature vectors are compressed in order to reduce the number of bits needed to represent them. The MFCC coefficients are grouped into pairs, where C0 and loge is a pair of its own. Each pair is quantised based on a weighted Euclidian distance and using its own codebook. The resulting index values are packed into a multiframe that contains frames from 24 samples along with multiframe header information and CRC codes. The header of a multiframe has 16 CRC code bits for error recovery, and each pair of frame indexes have 4 bits of CRC code to detect errors. The resulting transmission rate (including extra bits for header information and error protection) is 4.8 kbit/s. DSR Decoding and error recovery: At the receiving side (the back-end) the vectors are recovered and presented to the speech recognition system. Frames received with errors are replaced with a copy of the closest vector without errors. 2.2 The GSM channel GSM transmission channels can be characterised by varying bit error rate conditions. These are caused by several factors; signal strength (which normally depends on distance from base station), noise levels, the effect of signal reflections (multipaths) and the velocity of the mobile terminal. These physical conditions influence the Bit Error Rate (BER) of a transmission channel. Typical BER in a mobile radio channel varies from e-5 (good conditions) to e-3 (poor conditions). Several strategies have been advised to minimize the effect on the actual application from bit transmission errors. A method is to add redundancy bits before transmission. Depending on the coding of these bits, the receiver may be able detect if an error has occurred or not and can take action according to this situation. The erroneous data could be re-transmitted, but this would introduce an uncertain element of delay and that is generally not good in voice communication. Rather the receiver can do some interpolation for the erroneous data. The ETSI DSR standard specifies that for every two pair of compressed feature vectors, a 4-bit CRC code be added. This is used to determine if a frame pair has been received correct or not. In case of errors, the last correct frame is used to replace the first bad frame of the frame pair, and the first correct frame after the error is used to replace the last bad frame of the frame pair. This interpolation method is also used if there is a sequence of bad frame pairs. The GSM bit error masks used in this work are obtained from simulations with SystemView software ( The masks are used to introduce bit errors similar to that of a GSM transmission into the test files. 2.3 The IP network IP networks can be characterised by packet loss. One of the reasons for packet loss is congestion where network nodes could drop packets. This network behaviour, with typically bursty packet loss, can be modelled using a 2-state Markov Model [Riskin, 2001]. A packet is discarded with a probability Accepted after abstract review page 469

3 p if the previous packet was received ok, and if the previous packet was lost then the packet is discarded with a probability q. There are a number of methods to packetise a bit stream and send the packets across an IP network. The packet size can be varied. Redundant data can be added to recover from packet loss. IP packetising is not part of the ETSI DSR standard. In this work IP packets are numbered so that a loss can be detected by comparing the sequence numbers. If a packet is lost, it is replaced with a fixed pattern 0x01 (no interpolation) to let the ETSI DSR algorithms do the error mitigation. Packet sizes of 12 and 24 bytes of data (excluding TCP/IP header information) are tested. These are normal packet sizes for VoIP transmissions. A second algorithm that transmits an extra copy of the packet if it contains the multiframe header is tested under high loss conditions. We developed our own program to implement the IP network packet loss algorithms. 2.4 The speech recogniser The HTK (Hidden Markov Model Toolkit, speech recognition system is used to measure the performance of DSR. The HTK system compares the recognition output with the transcription of the voice input and calculates the word accuracy. The word accuracy is reduced with insertion, deletion and substitution errors. The HMM model: A sub-word monophone HMM is used with 3 states left-to-right with no skip transitions and with 1 Gaussian per state. Two pause models are defined according to the commonly used short pause between words and silence at the beginning and end of sentences. The energy measure is based on the loge component. Delta and Acceleration coefficients are used, bringing the total number of coefficients to 39. The MFCCs are extracted from the voice recordings with the ETSI DSR extraction program. The models are trained on unquantised vectors, discarding the C0 coefficient in the process. The corpus: Voice recordings for training the recogniser and testing the DSR performance were collected at Newcastle University. The recogniser is used for testing continuously spoken random numbers from one to nine. Sentences with random sequences of these numbers are recorded using a PC soundcard and microphone in an office environment. 2 female and 3 male persons are reading 50 sentences each, totalling 2879 words. 80% of the recordings (200 sentences) are used for training and 20% (50 sentences) are used for testing the performance. Feature extraction Bit stream coding GSM channel errors IP network packet loss Decoding and recovery Recognition A B C D E F Result Voice recordings Figure 2. A summary of the test bed showing the steps from recorded voice input at the DSR front-end to the final recognition result from the speech recogniser. Accepted after abstract review page 470

4 3. EXPERIMENTS 3.1 Performance of DSR over GSM channel The error masks from the GSM channel are obtained from simulations with a combination of signal loss, noise, number of multipaths and speed to get the various BER values. Table 1. The effect of GSM bit errors on recognition performance GSM Condition GSM BER % Word Accuracy No errors Medium 1.2e Medium/Poor 8.9e Poor 6.67e Performance of DSR over IP network The test files are modified with the bursty packet loss characteristics of IP networks according to Table 3, where Packet Loss Rate are experienced values. For algorithm 1, the packets are numbered and sent, and if a packet is lost it is replaced with a fixed error pattern (no interpolation by the receiver). IP packet sizes of 12 and 24 bytes of data are tested. A second algorithm, which in addition sends the packet twice if it contains the multiframe header, is tested only under very high packet loss rate conditions. Table 2. The effect of IP packet loss on recognition performance IP network characteristics % Word Accuracy Algorithm 1 % Word Accuracy Algorithm 2 p q Packet Loss 12 byte 24 byte 12 byte 24 byte Rate No loss % % % % % % % Performance of DSR over GSM and IP network in tandem The last test is to combine the GSM channel and IP network in tandem. The files modified to a certain BER from the GSM channel are further modified with packet loss in the range from 5-15 %. Table 3. The combined effect of GSM bit errors and IP packet loss on recognition performance GSM BER No IP loss 05% IP loss 10% IP loss 15% IP loss 1.2e e Accepted after abstract review page 471

5 4. CONCLUSIONS The purpose of this paper has been to test the performance of the ETSI DSR algorithm over GSM and IP networks. The training of the speech recogniser and testing of the performance has been done with locally collected data. To get more accurate and optimal results, more data is needed for training and testing. However, the results obtained for the ETSI DSR performance are showing good results even on the background of the small amount of data used. A GSM channel with poor conditions (a bit error rate of 6.7e-3) reduces the recognition performance by 6.3% compared to an errorfree signal. An IP network with normal conditions (packet loss less than 10%) reduces the recognition performance by 0.9%. Even under extremely poor IP network conditions (packet loss of 33%) the recogniser performance is reduced by as little as 5.6%. To get this result it is enough to use a simple packetising method where the packets are numbered (so that the receiver can detect if a packet is lost) and replace lost packets with a fixed error pattern without interpolation or re-transmission. A GSM radio channel and IP network may well be used in tandem. If the IP packet loss is less than 10%, then the effect of IP packet loss is insignificant also in combination with GSM bit errors. 5. FUTURE WORK There is obvious redundancy in the voice signal when a high IP packet loss of 33% is found to reduce the recognition performance by as little as 5.6%. New ways of compressing the feature vectors to obtain even lower bit rates can be designed. Also there are other and more elaborate ways to packetise the bit stream file across IP network and interpolate for lost packets at the receiver. Specifically, a new Frame Pair Alignment method can be tested where the multiframe header and feature vector frame pairs are aligned with the IP packet boundaries. 6. ACKNOWLEDGEMENTS Thank you to colleagues at Newcastle University and Sør-Trøndelag University College that have been very helpful with running the software on combined Windows and Linux platforms. Joshua Wall have written the programs for modifying the files with GSM bit errors and IP packet loss. Trym Holther and Gabriel Brabant have been very helpful with various steps in the process of speech recognition. This work has been supported by TISIP research and development foundation. 7. REFERENCES ETSI (2000), ETSI Standard ETSI ES v.1.1.2, ( Pearce, D. (2000) Enabling New Speech Driven Services for Mobile Devices. AVIOS 2000:The Speech Applications Conference, May 22-24, 2000 San Jose, CA, USA. Riskin, E. A. & Boulis C. & Otterson S. & Ostendorf M. (2001) Graceful Degradation of Speech Recognition Performance Over Lossy Packet Networks. Eurospeech 2001, September 3-7, 2001 Aalborg, Denmark Accepted after abstract review page 472