VoIP CALL PERFORMANCE OVER DURING HTTP AND BITTORRENT DOWNLOADS R. Yasinovskyy Towson University Towson, MD, 21252 ryasin1@students.towson.edu A. L. Wijesinha Towson University Towson, MD, 21252 awijesinha@towson.edu R. Karne Towson University Towson, MD, 21252 rkarne@towson.edu Abstract We study the performance of a VoIP call in an network during download of a large file using HTTP, BitTorrent, or utp, and simultaneous downloads using HTTP and BitTorrent or HTTP and utp. Performance metrics include maximum delta, maximum and mean jitter, throughput, packet loss, and perceived voice quality. The results indicate that the values of these metrics are stable and that call quality is not affected significantly by the background traffic. Performance over and is similar to performance over. In general, values of metrics indicate that voice quality is least affected by HTTP downloads, but BitTorrent downloads are likely to degrade voice quality more than utp downloads. Transfer times for the file are highest for utp over and BitTorrent over. Bandwidth utilization patterns show that HTTP uses the available bandwidth more efficiently than BitTorrent and utp, although utp is better than BitTorrent. 1. INTRODUCTION is expected to gradually replace in the future Internet even though adoptions by ISPs and organizations to date have been slow. While it is true that layered protocol models insulate applications for the most part from protocol changes at the IP level, is still a new protocol whose performance with newer Internet applications has not been fully tested. For example, VoIP call traffic volumes on the Internet are increasing and this traffic must be delivered without severe loss and in a timely manner if voice quality is to be acceptable. Furthermore, a large proportion of Internet file downloads now involve HTTP and BitTorrent. It is therefore important to study VoIP call quality over, the behavior of HTTP and BitTorrent with, the combined effect of mixing such applications that have differing delay or loss requirements in an network, and the impact of overhead due to interim protocols such as or during the anticipated -v6 transition period, and security protocols such as IPsec. We consider the performance in a test LAN of a single VoIP call over during the download of a large file using HTTP, BitTorrent, or utp (BitTorrent over UDP), or simultaneous downloads using a combination of these protocols. We measure voice quality using the values of maximum delta (packet inter-arrival time), jitter (packet delay variation), and voice throughput (bits delivered per second). In addition, we determine voice quality by listening, packet loss, transfer time for the file, and characterize the typical rise-and-fall bandwidth utilization patterns for the different types of download traffic. Finally, we compare the results for with,, and. All traffic is carried over a virtual private network (VPN) tunnel between IPsec gateways. Although a single VoIP call does not capture the interactions and effects due to multiple calls, it provides insight into the behavior of voice in an network when a bandwidth-intensive file download is taking place. Moreover, this study helps understand how TCP utilizes the available bandwidth over time, whether BitTorrent or utp over UDP is more efficient, and whether quality of the VoIP call is not adversely affected in spite of having to compete for network resources. Finally, our study also determines the performance impact of the overhead due to securing the traffic with IPsec. Our results will be useful for ISPs and system administrators for future capacity planning in and -v6 transition networks, and for making decisions concerning traffic prioritization. The rest of this paper is organized as follows. In Section 2, we briefly discuss related work; in Section 3, we describe the test LAN and the experimental set up, and in Section 4, we present the results of our performance study. Section 5 contains the conclusion. 2. RELATED WORK There are numerous studies on the BitTorrent protocol over. In [1], the authors analyze protocols including BitTorrent focusing on data block distribution. A simulation-based study in [2] shows that BitTorrent is near-optimal with respect to uplink bandwidth utilization, and suggests techniques to improve its fairness and download time. The behavior of BitTorrent over long time periods with respect to popularity, availability, download performance and content lifetime is studied in [3]. In [4], the efficiency of sharing multiple files in BitTorrent is analyzed and a new downloading scheme to improve efficiency is proposed. In [5], it is suggested that network utilization may be improved if ISPs supply information to P2P users about high bandwidth links and proximity to other users. An experimental study of BitTorrent in a large ISP network is presented in [6]. Previous studies of HTTP performance use and most 13
Figure 1. Test LAN are not new. The classical study in [7] uses real-time data to discuss HTTP performance with respect to persistent connections, pipelining and other HTTP-specific features, and the impact of TCP on HTTP performance. In [8], the impact of on data transfers over TCP and UDP is examined supported by results for throughput and latency, and the differences in performance with and are identified. In [9], a comparison of VoIP performance in and networks shows that performance differences between the protocols are insignificant. A study of VoIP call quality over and with IPsec shows that quality is minimally affected by moderate loads on the network [1]. The performance study in this paper differs from previous work in that we consider 1) the effect on VoIP call quality of BitTorrent and HTTP, which are dominant among Internet applications today, as well as the newer utp (BitTorrent over utp) over ; and 2) file transfer times and bandwidth utilization patterns over for HTTP, BitTorrent, and utp. 3. TEST LAN AND EXPERIMENTAL SET UP The test LAN consists of five Ethernet switches connected by four Linux routers as shown in Fig. 1. The VoIP traffic generated by the call between Client #1 and Client #2 competes with background traffic from the HTTP server to HTTP client and/or Torrent seed to Torrent leech connected to the ends of the network. With the exception of the gigabit switch to which the data sources are connected and the gigabit interface of Router #4, all other network interfaces and switches have 1 Mbps capacity, which creates congestion at Router # 4. Routers #1 and #4 serve as IPsec gateways, but are used to perform NAT or conversion when needed. We only used NAT to test ; both clients were assigned routable addresses for the -only tests. All traffic passing through the subnets connected to Router #2 and Router #3 is encrypted and authenticated using IPsec tunnel mode between the IPsec gateways, whereas ordinary (unencrypted) or v6 packets are sent and received on the end subnets connected to Router #1 and Router #4. The Wireshark packet analyzer captures Client #1 s packets via port mirroring at the switch to which it is connected. Each experiment is run for two minutes but only the second minute of data is used to compute results. The details of hardware, operating systems, and other software used for our experiments are as follows: Hardware: Router: Dell Optiplex GX26 (Pentium 4, 2.4 GHz, 512 Mb RAM, Intel PRO/1, 3Com 1/1); Client/Server: Dell Optiplex GX27 (Pentium 4, 2.4 GHz, 248 Mb RAM, Intel PRO/1, 3Com 1/1); Switches: Cisco Catalyst 295, Netgear GS18 (1), Netgear FS38 (1/1), Trendnet TE1-S55E (1/1). Operating Systems: CentOS 5.2 (Kernel 2.6.18-92.1.22) (Routers, SIP Server, NTP Server, Wireshark), Windows Vista Business SP1 (µtorrent), Fedora 1 (Kernel 2.6.27.21-178) (HTTP Server/Client, VoIP Clients). 14
Other Software: Linphone 2.1.1-1 (ITU-T G.711 ulaw codec), Wireshark 1..6, OpenSER 1.3.4-1, Openswan 2.6.14-1, Miredo 1.1.5, httpd-2.2.11-2, curl-7.19.4-3, µtorrent 1.9..14589. The values of delta, jitter, packet loss, and throughput we present are those reported by Wireshark. Voice quality was informally assigned a MOS (Mean Opinion Score) by listening. We determined transfer time for the file downloads from the statistics provided by the applications or by measuring it ourselves. The typical rise-and-fall bandwidth utilization patterns during downloads were plotted by using the observed or reported values of the dynamically varying data rates. We tested quality of a single voice call respectively using,, and (note that we used Miredo, the Linux implementation of ). During the call, download of a 492-MB file was carried out using HTTP, BitTorrent, or utp. We also tested the performance of the voice call during simultaneous downloads of the file using HTTP and BitTorrent, or HTTP and utp. Unfortunately, we were able to use only with HTTP downloads. Also note that our tests with BitTorrent and utp do not reflect behavior that would result if a large number of clients were simultaneously downloading a popular file since clients would upload chunks of files to other clients in this situation. 4. PERFORMANCE RESULTS We first discuss voice quality as measured by the values of delta, jitter, and throughput. During these experiments, we did not see packet loss for voice except in two cases, both of which were insignificant: with utp over and simultaneous downloads using utp and HTTP over, packet losses of 1.12% and.7% respectively were observed. However, voice quality perceived by listening was excellent throughout and assigned a MOS rating of 4 or above. 4.1. Maximum Delta (packet inter-arrival time) In Fig. 2, we compare the values of maximum delta (packet inter-arrival time), which reflects packet delays but does not measure the actual end-to-end delay. Maximum delta is not a reliable measure since a few large delta values do not result in a drop in voice quality. We also computed mean delta for all tests and verified that it is close to the ideal value of 2ms. With no background traffic, maximum delta is around 4 ms; it is slightly but not significantly higher with possibly due to its overhead resulting from extra protocol headers. For HTTP downloads, we observe that maximum delta is largest with (around 83 ms) and smallest (around 48 ms) with both and. For BitTorrent downloads, maximum delta is higher for all protocols (between 73-87 ms). In contrast for utp downloads, maximum delta is lower for all protocols (between 36-46 ms). For simultaneous downloads, we note that for utp with HTTP, maximum delta is lower for (41 ms) and (45 ms), but higher for (73 ms); for BitTorrent with HTTP, maximum delta increases to between 7-73 ms. 4.2. Maximum and Mean Jitter Next we consider values of jitter. Maximum jitter is shown in Fig. 3. For utp, maximum jitter is lower with (7.5 ms) and (7 ms). Maximum jitter is higher for BitTorrent with all protocols (16-18 ms) and (24 ms with no traffic and 19 ms with HTTP). In general, maximum jitter is less for (except with BitTorrent with or without HTTP). In general, simultaneous downloads do not increase the values of maximum jitter. Mean jitter shown in Fig. 4 is highest with BitTorrent (1-12 ms) regardless of the protocol. Excluding BitTorrent, mean jitter for all protocols (except for with HTTP) is about the same varying between 5-9 ms. Note that as with maximum jitter, mean jitter is also not increased by simultaneous file downloads. 4.3. Throughput We observe that voice throughput shown in Fig. 5 is stable and in general close to the expected value that can be computed using the sizes of the headers. This is because delays are not excessive and packets/sec is close to the ideal value of 5 (based on 2 ms of voice data per packet). For example, throughput is lowest with and essentially the same (around 86 kbps) regardless of whether HTTP, BitTorrent, utp or simultaneous downloads are used. Throughput is highest with (around 15 kbps) due to its multiple headers. Throughput with and is the same (around 94 kbps) since packets captured by Wireshark are identical to packets (the header was removed at the gateway) and overhead does not degrade performance. 4.4. File Transfer Time Transfer times for a 492-MB file are shown in Fig. 6. In general, transfer times for BitTorrent and utp are higher than for HTTP due to the characteristics of the BitTorrent protocol. Regardless of the protocol, HTTP transfer times are stable (between 44-46 secs) and close to the expected transfer time of 39 secs for 492 MB at 1 Mbps. utp over and BitTorrent over have the largest transfer times (close to 5 minutes), while utp over is around 3.5 minutes. In contrast, BitTorrent over and and utp over have lower transfer 15
Max delta (ms) Mean jitter (ms) 1 9 8 7 6 5 4 3 2 1 14. 12. 1. 8. 6. 4. 2.. Figure 2. Maximum delta Figure 4. Mean jitter Max jitter (ms) Throughput (Kbps) 3 12. 25 1. 2 15 1 5 8. 6. 4. 2.. Figure 3. Maximum jitter Figure 5. Voice throughput times (89, 94 and 92 seconds respectively). The wide variability of transfer times for BitTorrent and utp with and even in this controlled environment may indicate that more efficient implementations of these applications over are needed. 4.5. Bandwidth Utilization Patterns We finally consider rise-and-fall patterns of bandwidth utilization over time, which are shown in Figs. 7-15. HTTP over all protocols (Figs. 7-9) shows a similar pattern where throughput rises to its peak rate of 88 Mbps in 15 seconds and stays at this level till about 4 seconds when it falls off. BitTorrent over (Fig. 1) rises to 48 Mbps in 2 seconds before falling back to 4 Mbps and then rising to its peak rate of 64 Mbps in a stepwise manner in 6 seconds before falling off gradually. BitTorrent over (Fig. 11) rises to its peak rate of 48 Mbps in 35 seconds and has a sawtooth pattern with very small drops thereafter until 95 seconds when it falls off gradually. BitTorrent over (Fig. 12) has a similar pattern except that it reaches its peak rate of 16 Mbps in about 25 secs, then fluctuates around this rate before falling to 12 Mbps after 1 minute, exhibiting a sawtooth pattern close to this rate thereafter till about 4.5 minutes when it falls off gradually. utp over and (Figs. 13 and 14) are similar in that both rise to their peak rate of 48 Mbps in 35 seconds and stay at this rate before gradually falling off after 85 seconds; however, while utp over rises continually with a slight reduction in the rate of rise around 32 Mbps, utp over falls from 24 Mbps to 16 Mbps around 15 seconds before continuing to rise. utp over (Fig. 15) has a similar pattern to utp over, but it falls from 16 Mbps to 8 Mbps around 15 seconds, then rises till it reaches the peak throughput of 48 Mbps in 45 seconds staying at this rate till it falls gradually at 95 seconds. In the case of HTTP and BitTorrent which use TCP, these utilization patterns may be generally explained by considering TCP slow start behavior and its sliding window mechanism, but a detailed analysis would require examining the nature of the BitTorrent and utp protocols. 5. CONCLUSION The performance of a single voice call over or while downloading a large file using HTTP, BitTorrent, utp, or HTTP with BitTorrent or utp is similar to performance over. Values of maximum delta, maximum and mean jitter, throughput, and packet loss, and perceived voice quality indicate that performance is largely unaffected by the background traffic. Transfer times for the files and bandwidth utilization patterns provide some insight into the behavior of these protocols in an network. In general, call performance is best during HTTP downloads, but less efficient with BitTorrent than with utp possibly due to TCP flow control constraints. 16
6. REFERENCES [1] D. Arthur and R. Panigrahy, "Analyzing bittorrent and related peer-to-peer networks," Proc. the 17 th annual ACM-SIAM symposium on Discrete algorithms (SODA 6), pp. 961-969, 26. [2] R. Bharambe, C. Herley, and V. N. Padmanabhan, "Analyzing and improving a bittorrent networks performance mechanisms," Proc. IEEE INFOCOM, pp. 1-12, Barcelona, Spain, Apr. 26. [3] J. Pouwelse, P. Garbacki, D. Epema, and H. Sips, The Bittorrent P2P File-sharing System: Measurements and Analysis, 4th International Workshop on Peer-to-Peer Systems (IPTPS'5), pp. 25-216, 25. [4] Y. Tian, D. Wu, and K.-W. Ng, "Analyzing multiple file downloading in bittorrent," Int l Conf. Parallel Processing (ICPP 6), pp. 297-36, August 26. [5] V. Aggarwal, A. Feldmann, and C. Scheideler, "Can ISPs and P2P users cooperate for improved performance?" SIGCOMM Comput. Commun. Rev., vol. 37, no. 3, pp. 29-4, July 27. [6] J. Qi, H. Zhang, and Z. Ji, Analyzing BitTorrent Traffic Across Large Network, Int l Conf. 492 MB File Transfer Time Cyberworlds, pp. 759-764, Hangzhou, China, September 28. [7] H. F. Nielsen, J. Gettys, A. Baird-Smith, E. Prud'hommeaux, H.W. Lie, and C. Lilley, Network performance effects of HTTP/1.1, CSS1, and PNG, Proceedings of the ACM SIGCOMM '97 conference on applications, technologies, architectures, and protocols for computer communication, pp.155-166, Cannes, France, September 1997. [8] S. Zeadally and I. Raicu, Impact of on End- User Applications, 1th Int l Conf. Telecommunications (ICT 3), pp. 973-98, Tahiti, Papeete, French Polynesia, February 23. [9] R. Yasinovskyy, A. L. Wijesinha, R. Karne, and G. Khaksari, A Comparison of VoIP Performance on and Networks, 7 th Int l. Conf. on Comp. Syst. and App (AICCSA 9), pp.63-69, Rabat, Morocco, May 29. [1] R. Yasinovskyy, A. L. Wijesinha, and R. Karne, Impact of IPsec and on VoIP Quality over, 1 th Int l Conf. on Comm. (ConTEL 9), pp. 235-242, Zagreb, Croatia, June 29. HTTP over 35 12 Transfer tim e (sec) 3 25 2 15 1 Expected HTTP BitTorrent utp 1 8 6 4 2 5 5 1 15 2 25 3 35 4 45 Figure 6. 492 MB File transfer time Figure 8. Bandwidth utilization: HTTP over HTTP over 12 1 8 6 4 2 5 1 15 2 25 3 35 4 45 Figure 7. Bandwidth utilization: HTTP over Figure 9. Bandwidth utilization: HTTP over 17
BitTorrent over utp over 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 1 11 12 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 1 11 12 Figure 1. Bandwidth utilization: BitTorrent over Figure 13. Bandwidth utilization: utp over BitTorrent over utp over 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 1 11 12 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 1 11 12 Figure 11. Bandwidth utilization: BitTorrent over Figure 14. Bandwidth utilization: utp over BitTorrent over utp over 25 2 15 1 5 15 3 45 6 75 9 15 12 135 15 165 18 195 21 225 24 255 27 285 3 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 1 11 12 Figure 12. Bandwidth utilization: BitTorrent over Figure 15. Bandwidth utilization: utp over 18