Assessment of VoIP Quality over Access Networks

Transcription

1 Assessment of VoIP Qualty over Access Networks M. Amr Mehmood, Pakstan Internet Exchange, IT Infrastructure Dvson, PTCL, Lahore, Pakstan Tarq M. Jadoon, Lahore Unversty of Management Scences, Lahore, 54792, Pakstan Noor M. Shekh, Unversty of Engneerng and Technology, Lahore, 54890, Pakstan Abstract Ths paper assesses VoIP qualty over access networks n Pakstan usng a delay tter measurement methodology for evaluatng the perceptual qualty of voce calls usng the ITU-T G.107 speech qualty E-model. Passve measurements for voce calls n the presence of background Internet data traffc for G and G.729a codecs are carred out usng a non-ntrusve parametrc model. The R-factor and resultant Mean Opnon Scores (MOS) were calculated at dfferent lnk loads and congeston hot spots were dentfed. The study hghlghts the nadequacy of access networks for handlng VoIP traffc at current n Pakstan and suggests allevatng congeston by ncreasng capacty n access networks. Index Terms VoIP, Perceptual Qualty Assessment, E-Model. T I. INTRODUCTION he Internet s evolvng nto the ubqutous packet swtched nfrastructure that aspres to provde an Integrated Broadband Network seamlessly ntegratng voce, vdeo, data and multmeda traffc. Convergng telephone and IP networks entals provdng the same tollqualty servce over best-effort IP networks. Ths s a sgnfcant engneerng challenge bearng n mnd that we now consder the hgh voce qualty standards that are a hallmark of the publc swtched telephone network (PSTN) for granted. Voce qualty s ultmately adudged by the lstener and thus, speech qualty s nherently perceptual or subectve n nature. Usng a numerc scale rangng from 1 (unacceptable) to 5 (excellent), the Mean Opnon Score (MOS) test provdes a wdely accepted measure for subectve speech qualty [1]. However, assessng speech qualty through surveys s a tme consumng and expensve process. A vable alternatve s to develop qualty models that smulate human ratng behavour by correlatng perceptual QoS wth quantfable parameters. Ths s not a straghtforward process as obectve metrcs do not necessarly correlate well wth perceptual qualty. A number of qualty models and tests that provde obectve MOS measures by correlatng well wth subectve scores have been developed. Rx classfes Ths work was supported n part by a research grant from the PTCL R&D Fund R&DF/Thematc-01/2004/06. The author s e-mal addresses are (amr.mehmood@ptcl.net.pk adoon@lums.edu.pk and adstec@malcty.com ) these tests as ether ntrusve or non-ntrusve test methods [2]. Intrusve testng methods nvolve comparng a reference acoustc speech sgnal wth a degraded verson of the sgnal receved through the system under test. The ITU-T standardsed the Perceptual Speech Qualty Measure (PSQM) [3] n Problems n algnng the reference and degraded sgnal whch are especally accentuated n VoIP networks necesstated mprovng PSQM and a new model called the Perceptual Evaluaton of Speech Qualty (PESQ) [4] was standardsed by the ITU-T as P.862 n The E-model s a non-ntrusve parametrc model that s well-establshed as a transmsson qualty model. It s defned n ITU-T G. 107 [5] and s based on the prncple that transmsson mparments combne addtvely nto a sngle psycho-acoustc transmsson ratng (R-factor) on a scale of 0 to 100. The R- factor can further be translated nto a MOS through a smple transformaton. Sendng speech as packets over the Internet entals samplng the orgnal voce sgnal at a fxed rate and convertng each sample n to a fxed number of bts. Ths constant bt rate stream s then ether drectly flled n packets of an approprate sze or s processed n frames of 10-30ms duraton and compressed before packetzaton. Packets are subsequently prefxed wth RTP/UDP/IP headers. Thus, a sample must wat for an algorthmc, processng and packetzaton delay before t can be placed on the wre. VoIP packets that traverse the Internet are subect to two prncpal mparments namely, packet loss and packet delay. Loss may ether be due to congeston and may lead to packets beng dscarded at ntermedate nodes or t may result from a falure of network components such as lnks and/or nodes. Packet delay has a fxed component as a consequence of the propagaton and transmsson delay as well as a varable component as a result of varable queueng delays packets encountered along buffers at ntermedate nodes whlst traversng the Internet. Thus, packets receved at the recever do not have the same temporal relatonshp as they dd at the sender resultng n delay tter. An approprately szed fxed or adaptve detter or playout buffer compensates for most of ths at the expense of an added delay. Delay tter manfests tself as packet loss for packets that arrve latter than a maxmum threshold and degrades the qualty perceved by the lstener. The conversatonal qualty of a call s prmarly affected by the end-to-end delay n addton to the packet loss and delay tter. These parameters consttute the network QoS

2 parameters and can be mapped to a perceptual QoS measures such as the MOS through qualty models such as the E- model. Ths paper examnes the perceptual VoIP qualty over access networks usng a smple delay tter methodology smlar to that suggested by Cole and Rosenbluth [6]. The balance of the paper s organzed as follows. Secton II revews the E-model. Secton III descrbes the expermental set-up, measurements and results. Secton IV concludes the paper by summarzng key results. II. E-MODEL The E-model was developed as a standard for measurng the transmsson qualty of narrowband telephony by ETSI [7] and latter standardzed by the ITU-T as recommendaton G.107. The E-model s a computatonal model that determnes a transmsson qualty ratng called the R factor from the transmsson parameters to predct the qualty of the mouth to ear (M2E) speech path. The typcal range for the R factor s for the PSTN and correspondng values of the MOS are 1-5 as shown n Fg 1. The basc prncple of the E-model s that the varous mparments contrbutng to the overall percepton of voce qualty are addtve when converted to the approprate psycho-acoustc scale (R). The basc formula for the E- model s: R = R0 I s I d I e eff + A (1) where R 0 s the sgnal to nose rato of the connecton. R User Satsfacton Very Satsfed MOS I s are mparments smultaneous to voce sgnal transmsson. It consders non-optmum sdetone, quantzng dstorton, overall loudness and other mparments, whch occur more or less smultaneously wth the voce transmsson. I d are mparments caused by delay after voce sgnal transmsson. It s a mathematcal summary of transmsson delay, talker echo and sdetone. I e eff are effectve equpment mparments (e.g. due to codecs). Ths value depends on the qualty of reproduced speech as rated by a user and s usually calculated from the MOS assgned to a partcular compresson algorthm. Fnally, A s the advantage factor gven to a partcular voce sgnal. The user expects some level of degradaton n the speech sgnal and psychologcally antcpates that the qualty wll not be as good as that of the PSTN for dfferent lnks such as moble connectons, etc. The advantage factor compensates for ths. The default values of all the aforementoned parameters (usng the year 2000 revson Annex A of [5]) result n R = The E-model s a wdely used computatonal model and several enhancements have been proposed to the basc model to cater for VoIP traffc [8]. Cole and Rosenbluth [6] outlne a methodology for ncorporatng packet loss and delay varatons as a reducton n base transport metrcs through the equpment mparment factor. Bascally, the end-to-end delay varatons result n loss at the detter buffer due to the arrvng packet stream underflowng or overflowng the decoder s detter buffer and ths s ncorporated by adustng Ie eff approprately. Other components that contrbute to a further reducton n QoS are the end-to-end delay as well as the packet loss Satsfed Some users dssatsfed Many users dssatsfed Nearly all users dssatsfed Not recommended Fg. 1: Voce Qualty classes III. EXPERIMENTAL SET-UP In order to perform an assessment of the perceptual qualty of VoIP traffc on access lnks, an expermental set-up usng a smple delay tter measurement methodology was employed for a number of access lnks as depcted n Fg 2. Three sets of experments were performed: 1. Cable-Modem to Dstrbuton Router-1 through ISP-1 2. Dal-up Modem to Dstrbuton Router-1 through ISP-2 3. Cable Modem to Dal-up Modem between ISP-1 and ISP-2. Voce sessons were establshed between soft phone clents usng the Sesson Intaton Protocol (SIP). Mcrosoft MSN Messenger ver 7.0. was selected as the soft phone clent due to ts low average mouth-to-ear M2E delay n comparson wth other IP phones or soft clent phones [9].

3 Delay - Jtter Montorng Staton INTERNET STM-1 Gateway Router Reverse RTP Packets Forward RTP Packets Fast Ethernet SIP Based Soft Phone Dstrbuton Router -1 Ggabt Ethernet Core Router Ggabt Ethernet Dstrbuton Router 2 E3/E1 E3/E1 ISP-1 ISP-2 Cable Modem Dalup Modem Delay - Jtter Montorng Staton Fg. 2: VoIP delay tter measurement set-up The frst two sets of experments entaled establshng calls from an Intel 1.5 GHz Centrno processor based laptop connected through a fast Ethernet port of a Csco 7500 seres Dstrbuton Router-1 placed at the Pakstan Internet Exchange node at Lahore to other machnes wth soft phone clents connected to access networks of ISPs va 64 kbps cable-modems as well as 56 kbps dal-up modems as shown n Fg. 2. Whle, the thrd set of experments was between soft phone clents connected to two dfferent ISPs. It s mportant to note here that the traffc characterstcs presented to the VoIP soft phone applcatons are a snapshot of the nstantaneous traffc condtons presented to the VoIP streams. Nevertheless, the readngs shown are representatve of the large degree of varaton n the loads and tter present n the access network. The voce streams were captured usng an open source packet capturng applcaton (Ethereal Ver ) nstalled on a delay-tter montorng staton. The VoIP stream from the soft phone s encoded as RTP/UDP/IP packets and s transmtted as the forward RTP stream whle the captured voce streams from the far sde s the reverse RTP stream. Jtter calculatons are carred out n accordance wth RFC 3550 [10] for both drectons. If S and S are the sender s and R and R the recever s RTP tmestamps for packet and, respectvely. The Inter packet spacng D may be expressed as: D(,) = ( R R ) ( S ) = ( R ) ( R ) (2) Ths nter-packet spacng gves a clear llustraton of the end-system vew of packet delay varaton. The nter arrval tter can be calculated successvely for each packet receved usng D accordng to the formula [10]: J ( ) = J ( 1) + ( D( 1, ) J ( 1)) 16 (3) The codec used n the 1 st set of experments was G.729a. Each VoIP frame had a lnk-layer frame sze of 94 bytes. 54 of these bytes make-up the RTP/UDP/IP/Ethernet headers and the remanng 40 bytes were the payload contanng compressed speech. The 2 nd and 3 rd sets of experments were performed usng the G codec. Each VoIP frame was 78 bytes n length and once agan 54 bytes were header overhead, whle the remanng 24 bytes were compressed speech. The tter was calculated usng (3) n the 1 st set of experments (cable modem to dstrbuton router-1 ) for two lnk loads,.e. 47.8% and 97.6%. An ncrease n the lnk load clearly manfests tself n greater tter as shown n Fg. 3. The correspondng emprcal probablty densty functons (pdf) of the nter-packet spacng for the cable modem to dstrbuton router-1 through ISP-1 are shown n Fg. 4. The mean nter-packet spacng s approxmately 20ms whle the standard-devaton of the nter-packet spacng ncreases from 1.455ms to 6.161ms when ISP-1 s lnk utlzaton ncreases from 47.8% to 97.6%. For comparson the pdf of the 2 nd set of experments (dal-up modem to dstrbuton router-1 through ISP-2) s also shown on the same graph. The pdf of the dal-up modem connecton has a varance of 12.33ms and a mean of approxmately 30ms. Fg. 5 shows the one-way nter-packet spacng D calculated usng (2) between a dal-up and cable modem connecton for experment 3.

4 Fg. 3: Jtter between a cable modem and dstrbuton router Fg. 5: One-way nter-packet delay between a dal-up and cable modem user. Fg. 4: Emprcal probablty densty functon of the nter-packet spacng Fg. 6 shows the correspondng emprcal pdf of nter-packet spacng. Ths s a bmodal dstrbuton and clearly ndcates that a sgnfcant number of packets get severely delayed as evdent from the second hump whle a delayed packet followed by a packet on tme results n clumpng of packets (apparent from the frst hump). The mean nter-packet spacng s 30ms. However, the standard devaton of the nterpacket spacng s 49.31ms. Further, 2.57% packets are lost as a consequence of network congeston. The R-factor and resultant MOS of voce calls for each set of experments can be calculated usng the E-model. The R factor can be estmated by calculatng ndvdual terms n the RHS of (1). The man terms are the effectve equpment mparment factor Ie eff and I d delay mparment factor whle the other parameters can be assumed to be a fxed for a gven expermental set-up: Ppl I e eff = I e + ( 95 I e ) (4) Ppl + B pl where I e s the equpment mparment factor. P pl s the packet-loss percentage and Bpl s the packet-loss robustness factor. Fg. 6: Emprcal probablty densty functon of the nter-packet spacng These values are codec dependent and have been suggested n ITU-T G.107 [5] and G.113 Appendx I [10]. Usng a lne of argument smlar to the one presented by Cole and Rosenbluth [6], one can calculate a conservatve bound on the probablty of packet loss at the detter buffer as a result of tter P detter _ buffer by usng Chebyshev s nequalty. The detter buffer sze s assumed to have a capacty of three packets. The total loss P s then gven by: P = P + ( 1 P ) P (5) pl pl detter _ buffer P would replace the packet loss Ppl n (4) to ncorporate the effect loss due to tter. Ths would result n an effectve equpment factor Ie eff ncorporatng the effects of VoIP packet loss as a consequence of tter. The other man parameter affectng R s I d. However, I d s not a sgnfcant component of (1) as long as the end-to-end delay s wthn the 175ms bound [12]. Beyond ths voce qualty degrades rapdly. Factors contrbutng to I d are the algorthmc delay as well as network latency (sum of propagaton and transmsson delays) and the decodng delay. The R values can now be calculated usng (1) and the correspondng MOS scores can be obtaned through expresson (6).

5 TABLE I Perceptual Qualty Assessment for VoIP for Access Networks Experment Codec Lnk I e P pl B pl e eff I µ ms σ ms P % * I R MOS e eff 1 G.729a 48% (ISP-1) % (ISP-1) G dal up (ISP-2) G cable to dal-up (ISP-1 to ISP-2) * ncludes the effects of loss due to tter Table I shows the correspondng values for the dfferent scenaros descrbed n the experments. For R < 0: MOS = 1 For R> 100: MOS = 4.5 For 0 < R < 100: MOS = R + 7.5x10-6 R (R-60)(100-R) (6) For the frst set of experments: Ie = 11, Bpl = 19 for G.729a and consequently I e eff 11 as there s no packet loss n the network. Incorporatng the effects of tter requre that the probablty of packet loss due to tter s calculated. A bound for ths loss can be calculated usng Chebyshev s nequalty assumng a buffer that can hold three packets. Ths results n a modfed effectve equpment mparment factor Ie eff whch s then used to calculate the R-factor and consequently the MOS values. The MOS values calculated of the frst set of experments were 4.1 and 3.75 for a load of 48% and 97% respectvely. By and large ths represents a satsfactory perceptual qualty. For the second and thrd set of experments, the G codec was employed. The correspondng values of I e = 15, Bpl = 16.1 and I e eff = 15 when there s no packet loss n experment 2 whle I e eff = 26 when P pl = 2.57% n experment 3. The algorthmc delay for G s 37.5ms but ths stll results n the end-to-end delay remanng wthn the 175ms bound. Ths ultmately results n a transmsson qualty factor R 62 for experment 2 and consequently an MOS of approxmately 3.2 suggestng that most, to nearly all users are dssatsfed. For experment 3, the resultant value of MOS mples that all users are dssatsfed as can be seen from Table 1. Experments 1 and 2 have been conducted from access networks to the core and typcally do not represent a realstc scenaro for VoIP calls. In realty, most calls would orgnate from end-user equpment on one ISP and termnate at end user equpment n the same or other ISPs. IV. CONCLUSIONS In concluson, the qualty of access networks s not adequate to support VoIP servces based on our measurements of the delay and tter prevalent n access networks n Pakstan. The method employed for carryng out the perceptual qualty assessment was a modfed E-model method caterng for effectve equpment mparments as a consequence delay, loss and tter over packet swtched networks for G.729a and G codecs. In order to provde a reasonable QoS, ISPs wll need to ncrease bandwdth n ther access networks or deploy QoS archtectures such as Dfferentated servces to provde preferental treatment and thus a reasonable QoS for voce traffc. REFERENCES [1] Methods for subectve determnaton of transmsson qualty. ITU-T Rec. P. 800, Aug [2] Rx, A. W., Perceptual speech qualty assessment - a revew. IEEE Internatonal Conference on Acoustcs, Speech and Sgnal Processng (ICASSP), Montreal, (3), pp , May [3] Obectve qualty measurement of telephone-band ( Hz) speech codecs. ITU-T Rec. P. 861, Feb [4] Perceptual Evaluaton of Speech Qualty (PESQ): An Obectve endto-end speech qualty assessment of narrow-band telephone networks and speech codecs. ITU-T Rec. P. 862, Feb [5] The E-Model, a computatonal model for use n transmsson plannng. ITU-T Rec. G.107, March [6] R. G. Cole, J. H. Rosenbluth, Voce over IP performance montorng. ACM SIGCOMM Computer Communcaton Revew, vol.31 no.2, Aprl 2001 [7] Speech Communcaton Qualty from mouth to ear for 3.1 khz Handset Telephony across Networks. ETSI ETR 250, July 1996 [8] L. Dng, R. A. Goubran, "Speech qualty predcton n VoIP usng the extended E-model", GLOBECOM 2003, vol. 22, No. 1, Dec 2003 [9] W. Jang, K. Koguch and H. Schulzrnne, QoS Evaluaton of VoIP End-ponts. IEEE Internatonal Conference on Communcatons (ICC), Anchorage, Alaska, May [10] H. Schulzrnne, et. al., RTP: A Transport Protocol for Real-Tme Applcatons, RFC 3550, July [11] Transmsson mparments due to speech processng: Appendx I: Provsonal plannng values for the equpment mparment factor Ie and packet-loss robustness factor Bpl, ITU-T Rec. G.113 Appendx I, May [12] One-way transmsson tme, ITU-T Rec. G.114, May 2003.