start of a talkspurt as given by (2). (1) (2)

Transcription

1 An Evaluation of the Potential of Synchronized Time to Improve Voice Over IP Quality Hugh Melvin and Liam Murphy Abstract Delivering PSTN-like quality over current besteffort Internet infrastructure presents many technical challenges. Much research in recent years has focused on receiver-based approaches which adapt to varying network conditions in order to optimize playout quality. In this paper, we propose and evaluate a receiver-based approach that implements a hybrid adaptivefixed playout regime by integrating synchronized time into the playout algorithm. Such an approach can deliver significantly better quality than existing adaptive techniques particularly when the underlying network is not heavily congested and end-toend delays are not excessive. We present some initial results from our testbed system using the ITU-T E-model to quantify improvements. I. INTRODUCTION THE degree to which Internet Telephony has replaced the traditional PSTN has to date been very limited [1]. The Internet s best-effort service compares poorly with the deterministic, circuit switched PSTN network. The ITU-T recommendation G.114 specifies that one-way delays should not exceed 1 msec [2]. With the exception of satellite links, the PSTN easily meets these limits, presenting end-to-end delays of which propagation time forms a significant and consistent proportion. The Internet generally presents higher and more variable end-to-end delays where propagation time is often dwarfed by congestion delays. In addition, severe Internet congestion often leads to packet loss. The ITU-T E-model, though limited enables such delay and loss to be quantified on an additive Transmission Rating factor (R) - 1 scale, which can be mapped to the better known Mean Opinion Score (MOS) 1- scale [3]. It is significant that R (& MOS) values are more sensitive to an increase in packet loss than to an increase in delay [4]. Because of the commercial worth of voice services, significant research has been undertaken in attempts to improve quality of service (QoS). Such research can be categorized by whether it involves measures taken at the sender, in the network, or at the receiver []. In this paper we focus on receiver-based buffering approaches. These compensate for network jitter by delaying playout to facilitate the arrival of delayed packets, at the expense of adding to the overall end-toend delay. We summarize such work outlining both the merits and shortfalls of the differing approaches. We also propose, describe and evaluate a new hybrid adaptive-fixed playout strategy. In this approach, synchronized time, provided via the H. Melvin is a Ph.D. student at the Department of Computer Science, University College Dublin, Ireland ( hugh.melvin@nuigalway.ie). Liam Murphy is a lecturer at the Department of Computer Science, UCD and the director of UCD s Performance Engineering Laboratory. Network Time Protocol () is used to determine end-toend delays on a per-packet basis and this precise information is used to select an optimum playout strategy. Results were gathered from a testbed that linked Dublin City University with the National University of Ireland, Galway. Extensive tests were carried out over a number of days under varying network loads. Our results indicate an R-scale improvement of up to 2 when compared to baseline existing adaptive approaches. More generally we propose that a hybrid adaptivefixed strategy can significantly improve quality on Internet links where the network is not heavily congested, and actual end-to-end delays rarely exceed ITU-T G.114 requirements. The remainder of the paper is organized as follows. Section II outlines and evaluates some alternative receiver-based buffer strategies, introduces the issues surrounding accurate delay measurement, and describes the hybrid algorithm developed by the authors. Section III outlines the use and limitations of the ITU-T E-model. Section IV describes the test system in detail. Section V presents detailed results including an analysis of performance. Section VI concludes the paper, outlining further work currently being implemented. II. RECEIVER BUFFER STRATEGIES Jiang and Schulzrinne outline that the total end-to-end delay in a session includes [6]: Operating System delay. Host hardware input/output delay including packetization, encoding and decoding delays. Network delay comprising transmission delay, propagation delay and variable queuing delay within intermediate routers and switches. Application delay, introduced in the receiver to absorb the variation in end-to-end delay principally due to queuing delays. From a packet loss perspective, it is important to distinguish between link loss and late loss. The design of the Application delay involves a trade-off between total end-to-end delay and late packet loss. Different approaches have been taken in the implementation of application delay or buffering within the receiver. Reference [7] compared four playout delay adjustment algorithms, two of which (denoted Alg. 1 and 4 in [7] and known as Alg. A and B respectively in this paper) are based on stochastic gradient algorithms. They differ in that Alg. B additionally employs a spike detection mode. In non-spike mode, both algorithms use a linear filter mechanism that tracks network conditions (see 1) and adjusts playout time accordingly at the

2 start of a talkspurt as given by (2).!#"$ % & In the above, ' refers to packet ', is the estimated endto-end delay, is the filter gain, is the measured delay, is the playout time, is the send time, & is the estimated variation in delay and " is a multiplication factor. Adjusting on a per-talkspurt basis maintains the integrity of speech within talkspurts whilst altering the inter-talkspurt silence periods. Moon, Kurose and Towsley propose a different approach in [8] (denoted Alg. C in this paper) whereby delay percentile information for 1 packets is maintained and updated with each received packet and used to dynamically adjust playout delay, again on a per-talkspurt basis. They also incorporate a spike detection mode and report significant benefits from their approach when compared with Alg. A and B. They conclude that: Alg. A reacts too slowly to sudden network variations and thus is only suitable for slowly changing network conditions. Alg. B tends to underestimate playout delay particularly after exiting spike mode due to its gain values. More recent work by Liang, Farber and Girod [9], denoted Alg. D in this paper, propose yet another mechanism that differs significantly from the above. Similar to Alg. C, it maintains a continuously updated histogram of previous packet delays to predict future playout delay. In contrast to Algorithms A, B and C, the playout adjustment of Alg. D is made on a per-packet basis. This is achieved by compressing or elongating packets (known as scaling) using a technique based on the Waveform Similarity Overlap-Add (WSOLA) algorithm. They report from tests that such scaling of packets results in little degradation of audio quality (.3-. in the DCR (Degradation Category Rating [1])) though qualify this by noting that scaling occurs infrequently during the reported tests. They note also that per-talkspurt algorithms will fail to react to short spikes where such spikes are contained within a single talkspurt. Considering the composition of human speech [11], this limitation can be very significant. Finally in [4], further comparisons are made between fixed and per-talkspurt adaptive playout algorithms (Alg. B above). The main points of this analysis are as follows: Adaptive algorithms that utilize the TCP-like formula such as (1) and (2) tend to overestimate delays. The selection of parameters such as and " is a nontrivial matter and tuning is required to suit network characteristics. Such characteristics may however change with time. Note that Alg. A-D are designed to operate with some late loss although the degree to which they do so varies. Additionally Alg. A-C suffer from distortion of inter-talkspurt silence periods and Alg. D suffers from packet scaling (distortion of both silence periods and talkspurts). Adaptive algorithms are thus most useful when receivers have no knowledge of actual one-way delays which is usually the case. If actual delays are (1) (2) known, and are within G.114 requirements, a fixed playout delay can avoid many of the problems associated with adaptive algorithms. However a fixed playout delay, imposed at the start of a session may quickly become inappropriate resulting in high late losses or unnecessarily high delays. A. DELAY MEASUREMENT Accurate delay measurement within the Internet is a nontrivial issue. Montgomery evaluated a number of solutions in [18], including the use of Round Trip Times (RTT), distributed synchronized time and a novel approach involving a variable delay estimation mechanism within routers, requiring a specific protocol format. More recently, RFC 2679 [19] examines the core issues surrounding accurate delay measurement. RTT remains an unreliable delay estimation mechanism but the availability of distributed synchronized time has greatly increased in recent years. This is due in part to the widespread deployment of the Network Time Protocol () but more importantly, to the availability of cheap yet highly accurate time sources such as GPS receivers. B. HYBRID ADAPTIVE-FIXED ALGORITHM The authors have implemented a hybrid adaptive-fixed algorithm that combines the useful characteristics of both adaptive and fixed buffer strategies. It operates as follows: Session commences implementing an adaptive buffer algorithm. The availability of synchronized time enables one-way packet delays to be precisely determined. Each receiver builds up a histogram of delays and from this extracts a delay estimate value (*) to meet target loss requirements. In the following pseudocode, +-, refers to the packetization delay and./ refers to a weight factor (-1) applied in determining the fixed playout delay. Recall that the G.114 limit is 1 msec. If (est < (1-Pkt)) { playout = est + (1-Pkt-est) * Wf Fixed Playout Mode = TRUE // switch to fixed mode } Maintain adaptive playout mode Maintain rolling histogram and at periodic intervals, recalculate delay estimate (*) : If (est < (1-Pkt)) playout = est + (1-Pkt-est) * Wf If (Fixed Playout Mode) If (conditions improved) Decrease fixed playout delay if (conditions deteriorated) Increase fixed playout delay Fixed Playout Mode = TRUE // switch to fixed mode // high delays If (Fixed Playout Mode) Fixed Playout Mode = FALSE // switch to adaptive mode Maintain adaptive playout mode The hybrid algorithm maintains a profile of network delays before determining a fixed playout delay, and thus avoids the

3 inflexibility usually associated with fixed buffer algorithms. The weight factor./ provides an extra delay margin by positioning the fixed playout delay between the extracted delay estimate (*) and the G.114 limit. Increasing./ will result in greater reductions in late losses at the expense of higher fixed playout delay. By implementing a fixed playout whenever possible, the integrity of speech both within and between talkspurts is maintained. Adaptive mode is thus used only when absolutely necessary. III. ITU E-MODEL The E-model is an ITU-T standardized tool for predicting how the average user rates the voice quality of a phone call with known transmission parameters. Technical details are specified in [3]. The model returns a transmission rating factor R, defined as: The factors of interest are (delay impairment) and (loss impairment). includes the distortion caused by low bit rate codec operation as well as the effect of packet loss (both link and late loss). Both and are intrinsic to the voice signal whereas factor is an advantage factor. All three are not considered in this analysis. In the context of evaluating real voice sessions, the E-model is limited in that it delivers an instantaneous rating based on singular loss and delay figures. Other research has thus examined issues such as bursty versus random loss, recency [12] [4], perceived versus instantaneous quality [12] and rating of entire voice calls rather than segments [4]. For the purposes of this paper, we use a simplified yet conservative E-model analysis to compare the performance of the hybrid algorithm with pertalkspurt adaptive algorithms. As such, our analysis is aimed at extracting approximate relative values rather than detailed absolute values. In considering, we simplify the effect of both talker and listener echo by applying a 1dB loss value which corresponds to reasonable echo cancellation. As such, the impairment can be approximated by 1 units per 1 msec [4] [13]. Note that acoustic echo is a serious problem with PC-based speaker/mic kits [13] leading to much worse impairment values than those assumed here. Regarding the factor, all tests utilized the G.711 codec which introduces almost no distortion, compared to lower bitrate codecs such as GSM or G.723. For the purposes of quantifying per percentage packet loss, we presume firstly that PLC (Packet Loss Concealment) is applied thus reducing the impact of packet loss [see G.113 [14] for details]. Secondly we determine the impairment value on the basis of bursty rather than random loss. From [14], for packet loss in the range -% can therefore be approximated by a dualslope curve set at 1 units per % above 3% and at 3 units per % below 3% (packet loss rarely exceeded % during the tests). Note also that values from [14] are based on 1 msec G.711 packets rather than the 3 msec packets used in our tests. As such we underestimate the impairment as [16] illustrates for G.723. A critical factor for the hybrid algorithm GPS Satellite Local Client NUI,G Testbed Implementation GPS Receiver NUI,G LAN GPS Satellite Server, NUI,G Fig. 1. Internet HEANET Remote Servers Internet Testbed system Remote Client is the lower sensitivity of to increased delay than that of to packet loss. Practically speaking, this means that users are more tolerant of higher delay than higher loss. This justifies the use of the weight factor./ in calculating the fixed playout delay from the delay estimate (*). A simple mapping exists from values to the 1- MOS scale [3]. DCU IV. IMPLEMENTATION DETAILS Fig. 1 describes the test system developed for evaluating the hybrid algorithm. The end-hosts, one at the National University of Ireland, Galway (NUI,G) and the other at Dublin City University (DCU) are 22km apart and connected by the Higher Education Authority network ( HEAnet is Ireland s academic and research network and is linked to similar networks in Europe and the US. HEAnet is a well-provisioned network and considering its limited geographical size, internal network delays are usually well within the G.114 limit. Synchronized time was implemented via. A stratum 1 server was set up at NUI,G to strengthen the local infrastructure. A GPS clock provides its reference source. Six other stratum 1 servers, located in the UK, France, Germany and Switzerland were included in the client configuration files. For more detail on the implementation of within the test-system see []; for greater detail on operation see [17]. An open source implementation of H.323 ( was used to deliver pre-recorded voice streams. This code was modified to extract trace data for simulation and analysis, and indeed more fundamentally, to implement both adaptive and hybrid algorithms in realtime. A Matlab based simulator was also used to verify the correct operation of the modified code. The adaptive algorithms chosen for comparison were Alg. A (with =.98 and " =4) and Alg. B ( varied from.87 to.98, " set to 4). The weight factor./ used within the hybrid algorithm was set to.33, based on network characteristics. Its value represents a trade-off between total end-to-end delay and the acceptable degree of late loss and is influenced by network jitter. Each end-host advertises its implementation of. This is verified by sending an query to the remote end and

4 Offset of NUIG client to candidate servers 23: hrs 4: hrs 9: hrs 14: hrs 19: hrs 24: hrs 1 Offset of DCU client to candidate servers 23: hrs 4: hrs 9: hrs 14: hrs 19: hrs 24: hrs NUIG Server ntp sop.inria.fr ntp1 rz.rzze.uni erlangen.de ntp2.ja.net hora.cs.tu berlin.de swisstime.ee.ethz.ch ntp2.ptb.de Offset (msec) NUIG Server ntp sop.inria.fr ntp1 rz.rzze.uni erlangen.de ntp2.ja.net hora.cs.tu berlin.de swisstime.ee.ethz.ch ntp2.ptb.de Offset (msec) 1 2 Sample Number (taken every 1 min) Sample Number (taken every 1 min) Fig. 2. Performance of the NUI,G client Fig. 3. Performance of the DCU client analyzing the response. As detailed in [17], a robust subnet requires careful consideration and design. The integration of synchronized time within the hybrid algorithm is done through the Real-Time Transport Protocol Control Protocol (RTCP) Sender Report (SR) packets. RTCP is a companion protocol to RTP and packets are generated periodically during a session [2]. On receipt of the 1st RTCP SR packet, each receiver extracts the and RTP timestamps contained within the packet. As system clocks are synchronized through, the receiver can therefore determine all subsequent incoming RTP packet transit times. V. RESULTS Before evaluating the relative performance of the algorithms, it is important to assess the performance of as synchronized time is a critical requirement for the hybrid algorithm. Fig. 2 & 3 outline the performance of the NUI,G and DCU client hosts respectively over a 36 hr period, during which the tests were carried out. The legend outlines which of the seven servers were candidates at each sample instant plus the clock offset of each of the candidates relative to the client clock. The NUI,G client clock offset remained within a +/- 1 msec band for 97% of the time whereas the DCU client clock offset remained within a +/- 7. msec band for 9% of the time. As expected, the NUI,G client remained tightly coupled to the NUI,G server, selecting it as a candidate 93% of the time. The DCU client selected the NUI,G client as a candidate 6% of the time, disregarding it whenever the DCU-NUIG link became congested, relative to the other servers. This is reflected in the range of offset values and confirms the need for robust local infrastructure. Overall within a context, the performance of both the NUI,G and DCU clients was very satisfactory. In all 4 tests were carried out over a 3 day period. Figures 4, and 6 summarize results for one of the 3 days whereas Fig. 7 outlines a single test result during a period of significant congestion. The latter compares the performance of the hybrid algorithm with that of Alg. A. In all but 3 of the tests, the hybrid switched from adaptive to fixed at the 1st opportunity (after 1st RTCP Sender Report is received) and remained in fixed playout mode. As illustrated in Figures 4 and, the hybrid algorithm resulted in a late loss reduction of up to 4 percent whilst incurring an increase in end-to-end delay of 3-4 msec. Note that Alg. B, though reacting well to spikes, performed poorly, resulting Late loss rate % Late loss rate for different algorithms 1: 11: 12: 13: 14: 1: 16: 17: 18: 19: 2: 21: Fig. 4. Late Loss Reduction due to hybrid algorithm Algorithm A Algorithm B Hybrid in high late losses when exiting spike mode. As outlined in section III, a simplified E-model analysis was applied which focuses on the relative performance of the algorithms rather than absolute R or MOS scores. Applying the approximate and impairment formulas from section III to Figures 4 and, Fig. 6 shows that the net effect of the hybrid algorithm was an R-factor improvement of between 2-2. Although Fig. 7 shows a high variance in network delays over a specific test period, the long term variation in delay over the testbed link was much less dramatic. Fig 8 shows the delay seen by the DCU client host whilst querying the NUI,G server over a 36 hr period, during which the tests were carried out. Congestion was not a serious problem on this link and other than very infrequent spikes, the delay remained around 16 msec. In fact, analysis of the delays to the other six servers, none of which is located in Ireland, indicates that the delay pattern was quite similar although the baseline delay ranged from 3-6 msec depending on location. In [21], the Delay Penalty (msec) Additional delay due to hybrid algorithm Hybrid delay penalty relative to Alg. A Hybrid delay penalty relative to Alg. B 1: 11: 12: 13: 14: 1: 16: 17: 18: 19: 2: 21: Fig.. Increased delay due to hybrid algorithm

5 1: 11: 12: 13: 14: 1: 16: 17: 18: 19: 2: 21: R factor improvement Delay (msec) Fig. 6. R factor performance comparison R factor improvement : Hybrid v Alg. A R factor improvement : Hybrid v Alg. B R-factor improvement due to the hybrid algorithm Performance of Hybrid vs Adaptive algorithms Hybrid commences in adaptive mode Hybrid changeover to fixed playout Subsequent c/o to higher fixed playout Adaptive playout 1st RTCP packet received => analysis of actual delays commences Fig. 7. Packet Number Typical test result authors extend the evaluation of the hybrid to more diverse networks and network conditions. Results are consistently encouraging, indicating that the hybrid algorithm is a viable alternative over much of the Internet s infrastructure. VI. CONCLUSIONS In this paper we propose and evaluate a new hybrid adaptive-fixed playout algorithm for. We describe and evaluate existing adaptive algorithms. Developments in and GPS technologies have greatly facilitated the implementation of distributed synchronized time and thus accurate delay measurement. Where network delays are known, and are within G.114 requirements, adaptive playout is largely unnecessary, and results in unnecessary late packet loss and voice distortion. In such cases, a fixed playout delay, though increasing overall end-to-end delay, will significantly reduce Delay (msec) Fig. 8. Delay seen by DCU to NUI,G 23: hrs 4: hrs 9: hrs 14: hrs 19: hrs 24: hrs Sample Number (taken every 1 min) Delay to NUI,G server seen from DCU late packet loss and fully preserve speech integrity. This tradeoff can often result in improved quality as users are more sensitive to increased loss than delay. Our hybrid algorithm combines the useful characteristics of both adaptive and fixed buffer schemes, estimating and implementing an optimum fixed playout delay whenever possible whilst utilizing an adaptive playout scheme only when necessary. Our results indicate an R-factor improvement of up to 2 over a series of tests carried out within the Irish research network. Finally, we show that, though requiring careful subnet design, more than adequately supports the implementation of the hybrid algorithm. Current work is examining the benefit of dynamically varying, both the weight factor and the frequency of playoutmode re-evaluation, according to network conditions. Due to the E-model s limitations, more analysis including subjective testing is also required to accurately quantify relative performance gains. REFERENCES [1] S. Bradner, Internet Telephony-progress along the road, IEEE Internet Computing, vol. 6,no. 3,May-June 22, pp [2] Recommendation G.114, One way transmission time, ITU, May 2. [3] Recommendation G.17, The E-model, a computational model for use in transmission planning, ITU, May 2. [4] A. Markopoulou, F. Tobagi, and M. Karam, Assessment of quality over Internet backbones, IEEE Proc of Infocom 22. [] H. Melvin and L. Murphy, Time synchronization for Quality of Service, IEEE Internet Computing, vol. 6,no. 3,May-June 22, pp [6] W. Jiang and H. Schulzrinne, QoS measurement of Internet real-time multimedia services, tech. report CUCS--99m, Columbia Univ., New York, Dec [7] R. Ramjee, J. Kurose, D. Towsley, and H. Schulzrinne, Adaptive playout mechanisms for packetized audio applications in wide-area networks, Proc. Conf. Comp. Comm. (IEEE Infocom), IEEE CS Press, Los Alamitos, Calif., June 1994, pp [8] S. Moon, J. Kurose, and D. Towsley, Packet audio playout delay adjustment:performance bounds and algorithms, ACM/Springer Multimedia Systems, vol. 6, pp , January [9] Y. Liang, N. Farber, and B. Girod, Adaptive playout scheduling using time-scale modification in packet voice communications, Proc. of ICASSP 21. [1] Recommendation P.8, Methods for Subjective Determination of Transmission Quality, Aug [11] J. Daigle and J. Langford, Models for analysis of packet voice communications systems, IEEE Journal on Selected Areas in Comm., vol. SAC-4, no. 6, pp. 847-, Sept [12] A. Clarke, Modeling the effects of burst packet loss and recency on subjective voice quality, Proc. of IP Telephony Workshop, Mar. 21. [13] J. Janssen, D. De Vleeschauwer, M. Buchli, and G. Petit, Assessing voice quality in packet-based telephony, IEEE Internet Computing, vol. 6,no. 3,May-June 22, pp [14] Recommendation G.113, Transmission impairments due to speech processing, ITU, Feb. 21. [1] W. Jiang and H. Schulzrinne, Modeling of packet loss and delay and their effect on real-time multimedia service quality, NOSSDAV 2, Chapel Hill,NC, Jun. 2. [16] S. Voran, Speech quality of G coding with added temporal discontinuity impairments, Proc. of ICASSP, May 21. [17] D. Mills, Network time protocol: specification, implementation, and analysis, IETF RFC 13, Mar [18] W. Montgomery, Techniques for Packet Voice Synchronization, IEEE Journal on Selected Areas in Comm., vol. SAC-1, no. 6, Dec [19] G. Almes, S. Kalidindi, M. Zekauskas, A One-way Delay Metric for IPPM, IETF RFC 2679, Sept [2] H.Schulzrinne, S.Casner, R.Frederick, and V.Jacobson RTP: A Transport Protocol for Realtime Applications, IETF RFC 1889, Jan [21] H. Melvin and L. Murphy, Implementation of a Hybrid Playout Algorithm for with Clock Skew Compensation, unpublished.