Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme Miroslaw Narbutt, Mark Davis Communications Network Research Institute Dublin Institute of Technology Wireless Cluster Workshop, Maynooth June 21/22 nd 2
Application mechanisms used to improve QoS Network Factors Congestion Network Packet Loss Clock End-to-end Packet Loss Perceived Quality Network Variable delay Jitter buffer Terminal Factors Codec Echo path & control Distortion Echo Topology Network Fixed delay End-to-end Delay Delay responsibility for packet loss, delay and jitter is delegated to the end-points as long as the problems in the network remain below a certain level, actions at the end-systems can greatly mitigate their effect: ow bit-rate codecs Packet loss concealment - PLC echo cancellation de-jitter buffering Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Application mechanisms used to improve QoS Network Factors Congestion Network Packet Loss Clock End-to-end Packet Loss Perceived Quality Network Variable delay Jitter buffer Terminal Factors Codec Echo path & control Distortion Echo Topology Network Fixed delay End-to-end Delay Delay transmission impairments (delay, loss, jitter) affect conversational speech actions at the end-systems can greatly mitigate their effect: low bit-rate codecs Packet loss concealment - PLC echo cancellation de-jitter buffering Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Application mechanisms used to improve QoS Network Factors Congestion Network Packet Loss Clock End-to-end Packet Loss Perceived Quality Network Variable delay Jitter buffer Terminal Factors Codec Echo path & control Distortion Echo Topology Network Fixed delay End-to-end Delay Delay transmission impairments (delay, loss, jitter) affect conversational speech actions at the end-systems can greatly mitigate their effect: Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Application mechanisms used to improve QoS Network Factors Congestion Network Packet Loss Clock End-to-end Packet Loss Perceived Quality Network Variable delay Jitter buffer Terminal Factors Codec Echo path & control Distortion Echo Topology Network Fixed delay End-to-end Delay Delay transmission impairments (delay, loss, jitter) affect conversational speech actions at the end-systems can greatly mitigate their effect: low bit-rate codecs Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Application mechanisms used to improve QoS Network Factors Congestion Network Packet Loss Clock End-to-end Packet Loss Perceived Quality Network Variable delay Jitter buffer Terminal Factors Codec Echo path & control Distortion Echo Topology Network Fixed delay End-to-end Delay Delay transmission impairments (delay, loss, jitter) affect conversational speech actions at the end-systems can greatly mitigate their effect: low bit-rate codecs packet loss concealment - PLC Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Application mechanisms used to improve QoS Network Factors Congestion Network Packet Loss Clock End-to-end Packet Loss Perceived Quality Network Variable delay Jitter buffer Terminal Factors Codec Echo path & control Distortion Echo Topology Network Fixed delay End-to-end Delay Delay transmission impairments (delay, loss, jitter) affect conversational speech actions at the end-systems can greatly mitigate their effect: low bit-rate codecs packet loss concealment - PLC echo cancellation Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Application mechanisms used to improve QoS Network Factors Congestion Network Packet Loss Clock End-to-end Packet Loss Perceived Quality Network Variable delay Jitter buffer Terminal Factors Codec Echo path & control Distortion Echo Topology Network Fixed delay End-to-end Delay Delay transmission impairments (delay, loss, jitter) affect conversational speech actions at the end-systems can greatly mitigate their effect: low bit-rate codecs packet loss concealment - PLC echo cancellation de-jitter buffering Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Application mechanisms used to improve QoS Network Factors Congestion Network Packet Loss Clock End-to-end Packet Loss Perceived Quality Network Variable delay Jitter buffer Terminal Factors Codec Echo path & control Distortion Echo Topology Network Fixed delay End-to-end Delay Delay transmission impairments (delay, loss, jitter) affect conversational speech actions at the end-systems can greatly mitigate their effect: low bit-rate codecs packet loss concealment - PLC echo cancellation de-jitter buffering Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Application mechanisms used to improve QoS Network Factors Congestion Network Packet Loss Clock End-to-end Packet Loss Perceived Quality Network Variable delay Jitter buffer Terminal Factors Codec Echo path & control Distortion Echo Topology Network Fixed delay End-to-end Delay Delay transmission impairments (delay, loss, jitter) affect conversational speech actions at the end-systems can greatly mitigate their effect: low bit-rate codecs packet loss concealment - PLC echo cancellation de-jitter buffering Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Variable delays and de-jitter buffer Size of de-jitter buffer at VoIP receiver is critical! s y Many adaptive playout algorithms: should achieve the best possible trade-off between loss delay should maximize user satisfaction
Variable delays and de-jitter buffer Size of de-jitter buffer at VoIP receiver is critical! Too small -> decreases e2e delay but increases late packet loss Too large -> decreases late packet loss but increases e2e delay Many adaptive playout algorithms: should achieve the best possible trade-off between loss delay should maximize user satisfaction
Variable delays and de-jitter buffer Size of de-jitter buffer at VoIP receiver is critical! Too small -> decreases e2e delay but increases late packet loss Too large -> decreases late packet loss but increases e2e delay Many adaptive playout algorithms: should achieve the best possible trade-off between loss delay should maximize user satisfaction
Variable delays and de-jitter buffer Size of de-jitter buffer at VoIP receiver is critical! Too small -> decreases e2e delay but increases late packet loss Too large -> decreases late packet loss but increases e2e delay Many adaptive playout algorithms! should achieve the best possible trade-off WE NEED A METHOD to evaluate them! should maximize user satisfaction
Playout delay adjustment (when?) 1. fixed throughout the whole session; 2. adjusted during silence periods (per talkspurt) [Ramjee 94, Moon 98, ]; 3. adjusted within talkspurts (per packet) using packet scaling (TSM technique) [Yi Liang 3].
Playout delay adjustment (how?) Reactive algorithm [Ramjee 94] INPUT : delay : network delay of a current packet i d _ est = α d _ est + (1 α) delay i i 1 i v _ est = α v _ est + (1 α) d _ est delay OUTPUT i i 1 i i : p = d _ est + βv _ est : playout delay of a current talkspurt i i i Histogram-based algorithm [Moon 98] d a : Playout delay Past 1 1 delays are stored in a circular array, both the array and the histogram get updated once a new packet is received. Loss rate specified by user. Playout time chosen from the histogram according to the desired loss rate.
Problem to solve management of the playout buffer is not specified by any standard and is vendor specific many fixed and adaptive playout schemes exist, each with a different parameter set information on the implementation of the playout buffer in commercial applications is practically nonexistent (it has a strategic value from vendor perspective) WE NEED A METHOD TO EVALUATE THEM FROM AN END USER PERSPECTIVE
How? Listening-only tests do not take into account delay impairments PESQ do not take into account end-to-end one-way delay in its rating -> NOT RECOMMENDED to asses the effect of CONVERSATIONAL delay E-model relies on static transmission parameters and do not take into account dynamics of changing transmission impairments (delay,loss) PESQ+E-model hybrid solution requires a reference speech signal and does not work in real time old statistical methods focus on late packet loss vs. buffering delay trade-off only and do not provide a direct link to conversational speech quality Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Trade-off: late packet loss/buffering delay Fixed
Trade-off: late packet loss/buffering delay Fixed buffering delay
Trade-off: late packet loss/buffering delay late loss Fixed buffering delay
Trade-off: late packet loss/buffering delay late loss Fixed LOSS-DELAY TRADE-OFF LOSS buffering delay BUFFERING DELAY
Trade-off: late packet loss/buffering delay late loss Fixed LOSS-DELAY TRADE-OFF LOSS buffering delay BUFFERING DELAY
Trade-off: late packet loss/buffering delay late loss Fixed LOSS-DELAY TRADE-OFF LOSS buffering delay Adaptive BUFFERING DELAY
Trade-off: late packet loss/buffering delay late loss Fixed LOSS-DELAY TRADE-OFF LOSS buffering delay Adaptive BUFFERING DELAY buffering delay Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Trade-off: late packet loss/buffering delay late loss Fixed LOSS-DELAY TRADE-OFF LOSS buffering delay Adaptive late loss BUFFERING DELAY buffering delay Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Trade-off: late packet loss/buffering delay late loss Fixed LOSS-DELAY TRADE-OFF LOSS buffering delay Adaptive late loss BUFFERING DELAY buffering delay Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Trade-off: late packet loss/buffering delay late loss Fixed LOSS-DELAY TRADE-OFF LOSS buffering delay Adaptive late loss BUFFERING DELAY buffering delay Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Trade-off: late packet loss/buffering delay late loss Fixed LOSS-DELAY TRADE-OFF LOSS buffering delay Adaptive late loss BUFFERING DELAY good algorithm SHOULD: achieve the best possible loss-delay trade-off buffering delay Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Trade-off: late packet loss/buffering delay late loss Fixed LOSS-DELAY TRADE-OFF LOSS buffering delay late loss buffering delay Adaptive Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme BUFFERING DELAY good algorithm SHOULD: achieve the best possible loss-delay trade-off be well tuned to maximize user satisfaction
E-model, speech transmission categories, user satisfaction Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme Transmission rating factor R = (R Is) Id(echo,delay) Ie(codec, loss) + A Reduced to transmission impairments: R = 94. - Id(echo,delay) 8 7 6 Delay Impairment Id vs. Delay TELR=4dB TELR=dB TELR=dB TELR=6dB TELR=6dB 6 4 Ie(codec, loss) Equipment Impairment Ie vs. Packet Loss Id 4 3 2 1 Ie 3 2 1 G.711 w/o PLC G.723.1 GSM G.729A G.711 Bursty Loss w. PLC G.711 w. PLC Random Loss 1 2 3 4 m2e delay [ms] 1 2 R-value 94. -9 9-8 8-7 7-6 6- Speech transmission quality Best High Medium Low Poor User satisfaction very some very dis many very dis almost all dis
Predicting time varying speech transmission quality s ] a y [m e l u t d p l a y o R ] % s [ lo Id Ie 3 2 2 PLAYOUT DELAY packet delays playout delay average playout delay 8 82 84 86 88 9 2 1 PACKET LOSS late lost 8 82 84 86 88 9 time [s] 2 1 DELAY IMPAIRMENT 8 82 84 86 88 9 EQUIPMENT IMPAIRMENT 8 82 84 86 88 9 1 QUALITY vs. TIME 8 6 8 82 84 86 88 9 time [s] 1. Playout buffer module calculates playout delays (m2e), average playout delays 2. and resulting average packet loss values for fixed time windows 3. E-model simulator calculates corresponding delay impairments Id 4.... and equipment impairments Ie. and resulting rating R Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Predicting time varying speech transmission quality s ] a y [m e l u t d p l a y o R ] % s [ lo Id Ie 3 2 2 PLAYOUT DELAY packet delays playout delay average playout delay 8 82 84 86 88 9 2 1 PACKET LOSS late lost 8 82 84 86 88 9 time [s] 2 1 DELAY IMPAIRMENT 8 82 84 86 88 9 EQUIPMENT IMPAIRMENT 8 82 84 86 88 9 1 QUALITY vs. TIME 8 6 8 82 84 86 88 9 time [s] 1. Playout buffer module calculates playout delays (m2e), average playout delays 2. and resulting average packet loss values for fixed time windows 3. E-model simulator calculates corresponding delay impairments Id 4.... and equipment impairments Ie. and resulting rating R Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Predicting time varying speech transmission quality s ] a y [m e l u t d p l a y o R ] % s [ lo Id Ie 3 2 2 PLAYOUT DELAY packet delays playout delay average playout delay 8 82 84 86 88 9 2 1 PACKET LOSS late lost 8 82 84 86 88 9 time [s] 2 1 DELAY IMPAIRMENT 8 82 84 86 88 9 EQUIPMENT IMPAIRMENT 8 82 84 86 88 9 1 QUALITY vs. TIME 8 6 8 82 84 86 88 9 time [s] 1. Playout buffer module calculates playout delays (m2e), average playout delays 2. and resulting average packet loss values for fixed time windows 3. E-model simulator calculates corresponding delay impairments Id 4.... and equipment impairments Ie. and resulting rating R Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Predicting time varying speech transmission quality s ] a y [m e l u t d p l a y o R ] % s [ lo Id Ie 3 2 2 PLAYOUT DELAY packet delays playout delay average playout delay 8 82 84 86 88 9 2 1 PACKET LOSS late lost 8 82 84 86 88 9 time [s] 2 1 DELAY IMPAIRMENT 8 82 84 86 88 9 EQUIPMENT IMPAIRMENT 8 82 84 86 88 9 1 QUALITY vs. TIME 8 6 8 82 84 86 88 9 time [s] 1. Playout buffer module calculates playout delays (m2e), average playout delays 2. and resulting average packet loss values for fixed time windows 3. E-model simulator calculates corresponding delay impairments Id 4.... and equipment impairments Ie. and resulting rating R Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Predicting time varying speech transmission quality s ] a y [m e l u t d p l a y o R ] % s [ lo Id Ie 3 2 2 PLAYOUT DELAY packet delays playout delay average playout delay 8 82 84 86 88 9 2 1 PACKET LOSS late lost 8 82 84 86 88 9 time [s] 2 1 DELAY IMPAIRMENT 8 82 84 86 88 9 EQUIPMENT IMPAIRMENT 8 82 84 86 88 9 1 QUALITY vs. TIME 8 6 8 82 84 86 88 9 time [s] 1. Playout buffer module calculates playout delays (m2e), average playout delays 2. and resulting average packet loss values for fixed time windows 3. E-model simulator calculates corresponding delay impairments Id 4.... and equipment impairments Ie. and resulting rating R Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
Predicting time varying speech transmission quality s ] a y [m e l u t d p l a y o R ] % s [ lo Id Ie 3 2 2 PLAYOUT DELAY packet delays playout delay average playout delay 8 82 84 86 88 9 2 1 PACKET LOSS late lost 8 82 84 86 88 9 time [s] 2 1 DELAY IMPAIRMENT 8 82 84 86 88 9 EQUIPMENT IMPAIRMENT 8 82 84 86 88 9 1 QUALITY vs. TIME 8 6 8 82 84 86 88 9 time [s] 1. Playout buffer module calculates playout delays (m2e), average playout delays 2. and resulting average packet loss values for fixed time windows 3. E-model simulator calculates corresponding delay impairments Id 4.... and equipment impairments Ie. and resulting rating R Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
E-model, speech transmission categories, user satisfaction Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme Transmission rating factor R = (R Is) Id(echo,delay) Ie(codec, loss) + A Reduced to transmission impairments: R = 94. - Id(echo,delay) 8 7 6 Delay Impairment Id vs. Delay TELR=4dB TELR=dB TELR=dB TELR=6dB TELR=6dB 6 4 Ie(codec, loss) Equipment Impairment Ie vs. Packet Loss Id 4 3 2 1 Ie 3 2 1 G.711 w/o PLC G.723.1 GSM G.729A G.711 Bursty Loss w. PLC G.711 w. PLC Random Loss 1 2 3 4 m2e delay [ms] 1 2 R-value 94. -9 9-8 8-7 7-6 6- Speech transmission quality Best High Medium Low Poor User satisfaction very some very dis many very dis almost all dis
Quality contours for G.711 with PLC (random loss) TELR = 6 db TELR = db 1 2 4 TELR = 6 db 1 2 4 TELR = db 1 2 4 TELR = 4 db 1 2 4 1 2 4 not recommended almost all users dis many users dis some users dis very User Satisfaction Categories
Quality contours for G.711 with PLC (bursty loss) TELR = 6 db TELR = db 1 2 4 TELR = 6 db 1 2 4 TELR = db 1 2 4 TELR = 4 db 1 2 4 1 2 4 not recommended almost all users dis many users dis some users dis very User Satisfaction Categories
Quality contours for G.729A TELR = 6 db TELR = db 1 2 4 TELR = 6 db 1 2 4 TELR = db 1 2 4 TELR = 4 db 1 2 4 1 2 4 not recommended almost all users dis many users dis some users dis very User Satisfaction Categories
Quality contours for GSM TELR = 6 db TELR = db 1 2 4 TELR = 6 db 1 2 4 TELR = db 1 2 4 TELR = 4 db 1 2 4 1 2 4 not recommended almost all users dis many users dis some users dis very User Satisfaction Categories
Quality contours for G.723.1 (6.4kbps) TELR = 6 db TELR = db 1 2 4 TELR = 6 db 1 2 4 TELR = db 1 2 4 TELR = 4 db 1 2 4 1 2 4 not recommended almost all users dis many users dis some users dis very User Satisfaction Categories
Quality contours for G.711 w/o PLC TELR = 6 db TELR = db 1 2 4 TELR = 6 db 1 2 4 TELR = db 1 2 4 TELR = 4 db 1 2 4 1 2 4 not recommended almost all users dis many users dis some users dis very User Satisfaction Categories
Predicting user satisfaction (proposed solution) ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 2 1 ] s [% o t l e pa ck 1 QUALITY CONTOURS not recommended almost all dis many dis some dis very ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 1 not recommended almost all dis many dis some dis very 27% 9% 27% 9% 27% USER SATISFACTION 2 4 2 4 2 4
Predicting user satisfaction (our solution) ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 2 1 ] s [% o t l e pa ck 1 QUALITY CONTOURS not recommended almost all dis many dis some dis very ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 1 not recommended almost all dis many dis some dis very 27% 9% 27% 9% 27% USER SATISFACTION 2 4 2 4 2 4
Predicting user satisfaction (our solution) ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 2 1 ] s [% o t l e pa ck 1 QUALITY CONTOURS not recommended almost all dis many dis some dis very ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 1 not recommended almost all dis many dis some dis very 27% 9% 27% 9% 27% USER SATISFACTION 2 4 2 4 2 4
Predicting user satisfaction (our solution) ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 2 1 ] s [% o t l e pa ck 1 QUALITY CONTOURS not recommended almost all dis many dis some dis very ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 1 not recommended almost all dis many dis some dis very 27% 9% 27% 9% 27% USER SATISFACTION 2 4 2 4 2 4
Predicting user satisfaction (our solution) ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 2 1 ] s [% o t l e pa ck 1 QUALITY CONTOURS not recommended almost all dis many dis some dis very ] s [% o t l e pa ck DELAY/LOSS DISTRIBUTION 1 not recommended almost all dis many dis some dis very 27% 9% 27% 9% 27% USER SATISFACTION 2 4 2 4 2 4
Algorithms evaluation testing environment prerecorded network delays, marker bits 23/7/1 11: 3 3 1 6 3 99 39 12 late packets loss [%] # =1 # =2 # =4 # =1 1 loss/delay trade-off LOSS/DELAY TRADE-OFF: histogram b. algs. concord alg. dynamic α Moon's alg. 1 2 3 4 6 av. buffering delay [ms] time varying quality of the call α=.9982, β=4. R PLAYOUT DELAYS AND PACKETS LOSS DISTRIBUTION 1 9 8 7 6 1 2 2 3 3 time [s] 1 not recommended almost all dis many dis some dis very QUALITY vs. TIME 4% 3% 7% 44% 1 2 3 4 USER SATISFACTION User satisfaction for G.711 w. PLC (random loss) & TELR = 6 18% 24% user satisfaction playout buffer simulator packet loss playout delays E-model simulator parameters parameters: encoder, echo
Experiment in the wireless LAN Measurement setup Influence of the background traffic on delay variation and jitter
Loss/delay trade-off (various algorithms) ] s [% o t l e pa ck te la 14 12 1 8 6 4 LOSS vs BUFFERING DELAY: reactive alg. α=.9 α=.98 α=.99 α=.9982 dynamic α alg. Bolot's alg. Ramjee's alg. ] s [% o t l e pa ck te la 14 12 1 8 6 4 LOSS vs BUFFERING DELAY: histogram b. alg. #= #=1 #=2 #=4 #=1 dynamic α alg. concord alg. Moon's alg. 2 2 4 6 8 1 12 av. buffering delay [ms] LOSS vs PLAYOUT DELAY on QUALITY CONTOURS: reactive alg. 14 dynamic α alg. Bolot's alg. 12 α=.9 Ramjee's alg. α=.98 ] not recommended 1 s [% α=.99 almost all users dis many users dis o t l some users dis 8 e pa ck very 6 te la 4 α=.9982 2 6 8 1 12 14 16 18 2 av. 2 2 4 6 8 1 12 av. buffering delay [ms] LOSS vs PLAYOUT DELAY on QUALITY CONTOURS: hist. b. alg. 14 dynamic α alg. concord alg. 12 #= Moon's alg. #=1 ] not recommended 1 s [% #=2 almost all users dis #=4 #=1 many users dis o t l some users dis 8 e pa ck very 6 te la 4 2 6 8 1 12 14 16 18 2 av.
Transmission quality vs. time and user satisfaction (Ramjee s:.9, Bolot s, dynamic α, Moon s: #2/1% Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme QUALITY vs. TIME QUALITY vs. TIME 1 1 8 8 R R 6 6 ] s [% o t l e pa ck 1 1 2 2 3 3 DELAY/LOSS DISTR. time [s] not recommended almost all dis many dis some dis very 1 2 3 4 8% 6% 6% 12% 16% 2% USER SATISFACTION ] s [% o t l e pa ck 1 1 2 2 3 3 DELAY/LOSS DISTR. time [s] not recommended almost all dis many dis some dis very 1 2 3 4 12% 3% 3% 1% 2% 6% USER SATISFACTION 1 QUALITY vs. TIME 1 QUALITY vs. TIME 8 8 R 6 R 6 ] s [% o t l e pa ck 1 1 2 2 3 3 DELAY/LOSS DISTR. time [s] not recommended almost all dis many dis some dis very 1 2 3 4 9% 1% < 1% 24% 6% USER SATISFACTION ] s [% o t l e pa ck 1 1 2 2 3 3 DELAY/LOSS DISTR. time [s] not recommended almost all dis many dis some dis very 1 2 3 4 9% 3% 3% 2% 1% 32% USER SATISFACTION
USER SATISFACTION CATEGORIES CODEC PLAYOUT MECHANISM not recommended [% time] almost all dis [% time] many dis [% time] some dis [% time] [% time] very [% time] G.711 Ramjee s alg. α=.998 11 1 2 3 42 4 Ramjee s alg. α=.9 1 3 6 11 6 11 Concord alg. 89 1 3 6 1 Moon s alg. 9 3 3 2 1 32 Bolot s alg. 12 3 3 1 2 6 dynamic α alg. 9 2 24 6 G.723.1 Ramjee s alg. α=.998 2 11 72 Ramjee s alg. α=.9 18 1 3 37 Concord alg. 9 2 2 Moon s alg. 1 2 8 8 Bolot s alg. 14 4 8 8 dynamic α 9 2 4 8 G.729A Ramjee s alg. α=.998 12 3 4 4 36 Ramjee s alg. α=.9 6 19 1 Concord alg. 89 1 3 6 1 Moon s alg. 9 1 4 49 37 Bolot s alg. 13 2 4 3 1 dynamic α alg. 9 1 2 2 64
Summary We need a method to evaluate both playout buffer algorithms and encoding scheme in end-to-end VoIP system statistical loss/delay metrics (average delay/average loss) give little information on user satisfaction listening-only tests and PESQ don t take into account interactivity E-model relies on static transmission impairments that do not correspond to dynamics of de-jitter buffering We proposed extended version of the E-model (short-time version) It provides direct link to speech transmission quality by estimating user satisfaction. User satisfaction is estimated from varying transmission impairments (playout delay, resulting packet loss) and takes into account encoding scheme Pictorial representation of playout delays and packet loss gives more detailed view on the performance of a given playout algorithm The proposed method can work in real time and off-line on delay traces
Conclusions We compared the performance of three audio codecs (ITU-T G.711, G.723.1, and G.729A) and various buffering schemes in a WLAN environment under varying load conditions using the extended version of the ITU-T E-model methodology. Results show that the use of the G.711 audio codec in conjunction with dynamic alpha adaptive playout scheme gives the highest user satisfaction of the Voice over WLAN schemes considered.
Thank you.
Performance comparison Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
DELAY/LOSS DISTRIBUTION 18 16 14 12 1 8 6 not recommended almost all dis many dis some dis very 2% 44% 3% 7% 44% 4 2 USER SATISFACTION 1 2 3 4 DELAY/LOSS DISTRIBUTION 18 16 14 12 1 8 6 4 2 not recommended almost all dis many dis some dis 1% very 2% 27% 4% % % USER SATISFACTION 1 2 3 4 Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme
QoS Concerns and Challenges Communication over best-effort networks Delay Impairs interactivity of conversational services Voice over IP: recommended one way delay < ms [ITU-T G.114] Packet loss Impairs perceptual quality Delay jitter Obstructs sequential and continuous media output Assessing the quality of VoIP transmission affected by playout buffer scheme and encoding scheme