BYOO: Build Your Own Overlay

Transcription

1 BYOO: Build Your Own Overlay Chris X. Cai (UIUC) Franck Le (IBM Research) Xin Sun (FIU) Geoffrey G. Xie (NPS) (technical report version) March, 5 ABSTRACT It is widely known that the end-to-end performance of interdomain paths can be fickle due to BGP s indifference to performance metrics and its slow responses to attacks and failure events. Prior research has established overlay networking and ISP-assisted tunneling as attractive solutions to override problematic BGP routes and bypass troublesome ASes. However, ISPs have seemed hesitant in adopting them, possibly because of an unclear business case. As a result, end users cannot take advantage of these solutions. In this paper, we propose and make an empirical case for the novel concept that an enterprise builds its own performance-enhancing overlay network through renting intermediate nodes from a cloud service provider. We conduct a measurement study using seven data centers of a global cloud provider and dozens of PlanetLab and residential end points, across five continents. The results show that more than 6% of the measured Internet paths when given the option of using one of four possible overlay nodes can have consistent throughput improvements, by at least 5% (with a median of.9 times), at a tenth of the cost of leasing private lines of comparable performance. Furthermore, we demonstrate the feasibility of leveraging multi-path TCP to auto-extract performance gain from a set of overlay paths, without having to explicitly choose the best path, which can be a daunting task due to fluctuating network conditions. Finally, we discuss practical deployment issues, such as how to tunnel through NATs and firewalls.. INTRODUCTION In the current interdomain routing system, autonomous systems (ASes) select paths not taking into account any performance metrics (e.g., delay, loss, jitter, available bandwidth), but mainly based on their business agreements with other service providers. This behavior often results in poor performance for the end users. To overcome these problems, researchers have proposed the concept of overlay networks where traffic can be tunneled through intermediate nodes deployed at key access and interchange points to bypass ASes experiencing congestions, failures, or attack [5, ]. Since late 99s, a number of measurements and pilot deployments have confirmed the performance benefits of overlay networks. For example, Andersen et al. showed that overlay networks could reduce packet losses significantly and double the throughput []. More recently, Peter et al. demonstrated that one single tunnel is sufficient to allow user traffic to recover from link failures one to two magnitudes faster than BGP [4]. However, despite these efforts, few commercial overlay networks are available except costly solutions [9] offered by content delivery network (CDN) providers to large web-centric companies. As a result, most end users (including enterprises of all sizes) cannot readily take advantage of the overlay network solutions to improve their network performance. We describe two motivating scenarios where such performance boosts are important. Connectivity between office branches: It is common that enterprises and government agencies consist of multiple branches at different geographic locations. Due to performance considerations, they often need to lease private lines between the branch offices, and each line typically costs thousands of dollars per month. Given the considerably lower prices of Internet access, overlay networks could be an attractive alternative at the tenth or hundredth of the cost [3, 7]. (A more detailed financial analysis is presented in Section 7..) Remote users: VPNs let remote users securely connect to a corporate or home network and maintain access to private applications and data. The quality of such access directly impacts how productive a remote user can be. Therefore, by improving network performance, overlay networks can boost the productivity of the mobile workforce. A growing number of companies authorize employees to work from home. Overlay networks could also help improve the job performance of telecommuters. Conceivably, some Internet Service Providers (ISPs) may be driven by the business opportunity to offer tunneling services [4] in the near future. However, it is far from certain whether and when a large number of ISPs will participate and expand the services beyond a niche market. Tier- ISPs have the desired geographical cov-

2 erage for placing overlay nodes but they probably need to partner with the lower-tier ISPs to make the services widely accessible to end users. In this paper, we propose a new approach for end users to reap the benefits of overlay networking in a proactive fashion, without waiting for ISPs to take actions. We envision them (e.g., an enterprise) to build their own overlay networks by renting virtual nodes from cloud service providers. Cloud service providers can have a large footprint like tier- ISPs, with datacenters at more than 4 geographic locations and connected by a well-provisioned private network [6]. The aggressive peering of cloud providers with all types of ISPs at Internet exchange points (IXPs) leads to path diversity from and to most end points. Moreover, users can rent and control virtual servers with Mbps virtual NIC from any of those locations starting at about $ per month. To the best of our knowledge, a cloud routed overlay network has never been tried before. Little known is about its performance characteristics. Open questions include: Can it bring significant performance gains? Will the performance gains be consistent over time, attainable for a large number of end users, and robust to failures and attacks? How many overlay nodes are needed and how should they be positioned? What is the impact on the stability and security of the overall inter-domain routing? As a first step toward answering these questions, particularly the ones pertinent to performance gains, we have conducted a measurement study in the last four months, using seven data centers of a global cloud provider and dozens of PlanetLab and residential end points, with network paths across five continents (North America, Europe, Asia, South America, and Australia). The main results can be summarized as follows. We conducted a prevalence measurement, consisting of clients in residential settings and PlanetLab nodes. We observed more than,5 paths, and analyzed whether tunneling to virtual overlay nodes deployed on a cloud service provider can improve their performance. Our results show that 74% of the default Internet paths have a lower throughput than at least one overlay path between the same two endpoints, and focusing on those paths with throughput gain, the average and median improvement factors are.8 times, and.38 times, respectively. A cloud-based overlay network can also reduce packet loss and the average RTT for more than half of the observed paths. Our further analysis indicates that a cloud routed overlay path has a high likelihood of increasing the throughput as long as it reduces the packet loss rate and RTT simultaneously, by only.% and.5% respectively. This result is encouraging because such low reduction thresholds are likely to be met for most scenarios given the path diversity on the Internet. We performed a longitidunal measurement, focusing on, and continuously observing a subset of 3 Internet paths over an entire week period. The results show that 9% of the 3 observed Internet paths get significant improvements across the measurement period, with an average improvement ratio of 8.39, and a median improvement ratio of In summary, the performance gains are consistent over time. In addition, the study revealed that only a small number of overlay nodes needs to be deployed, with one to two overlay nodes being able to provide most of the benefits. We propose a new automated path selection algorithm based on the recent Multi-Path TCP (MPTCP) technology [9] for the sender to decide which path(s) to send traffic. Taking advantage of the MPTCP congestion control algorithm design objectives [3], the proposed solution guarantees that the obtained throughput equals that of a single-path TCP connection on the best available path. We conducted experiments both in a controlled testbed, and in the Internet to validate it. The rest of the paper is organized into eight sections. Section describes our experimental methodology and datasets. Sections 3 and 4 quantify the performance gains of the sampled overlay paths and analyze how the gains persisted over time. A variety of analyses aiming to identify the key factors for the performance gains are presented in Section 5, followed by a study of using MP- TCP to auto-extract the performance gains from a set of overlay paths in Section 6. Section 7 discusses other main issues for deploying cloud routed overlay networks, and Section 8 provides a brief summary of related work. Finally, we offer concluding remarks in Section 9.. METHODOLOGY Compared to ISP networks, we know relatively little about the network infrastructures of cloud service providers (i.e., how their data centers are connected to each other and to the external world). Therefore, we have chosen a predominantly blackbox approach to measure potential performance gains from using cloud routed overlay paths. We selected a major commercial cloud provider and rented virtual Linux servers as overlay nodes in several of its 4 data-centers. Each server runs Ubuntu.4, and is provisioned with a single core (. GHz), a Mbps connectivity, and 4GB RAM. To act as an overlay node, a virtual server establishes a tunnel (GRE or IPsec) with the sender and runs a NAT through the Linux IP Masquerade feature. The NAT allows the return traffic from the receiver to the sender to also tra-

3 Figure : Residential setup: blue/white/red pins indicating clients/servers/cloud-hosted overlays Figure : PlanetLab setup: blue and red pins indicating PlanetLab nodes and cloud-hosted overlays, respectively verse the overlay node, without having to establish any tunnel with the receiver. The measurements were carried out in two stages. We first pilot-tested the validity of using a cloud routed overlay with a small-scale experiment involving home or mobile users downloading a large file from a remote web server. After performance gains were observed, we then performed a series of large-scale file download experiments leveraging PlanetLab nodes. For the first smallscale experiment, we rented 7 overlay nodes, located in Dallas, Houston, Amsterdam, London, San Jose, Washington DC, and Hong Kong (Figure.) For the largescale file download experiments, we used 5 virtual Linux servers located in Washington DC, San Jose, Dallas, Amsterdam, and Tokyo (Figure.) We focus on improving the performance of TCP applications as TCP is the dominant transport protocol used on the Internet. We quantify performance gains with respect to three metrics: (i) throughput, (ii) packet loss, and (iii) packet delay.. in Tampa and Paris are home PCs with broadband DSL subscriptions, and the subscribed bandwidth is 3 Mbps and 9 Mbps respectively. The Cardiff client connected to a hotel WiFi network of Mbps bandwidth, which was shared by all guests in that hotel. The test was performed in September 4. At the first step of measurement, we let each of the three client nodes directly download the Eclipse software [] installation file ( MB) from five mirror servers, one at a time. The creator of Eclipse hosts mirror servers in many different locations. On the Eclipse download webpage, one server is automatically recommended based on the user s location, and the other servers can be manually selected to download from as well. We intentionally select five servers from the list to cover a wide geographical range: () Georgia, USA, () Virginia, USA, (3) Waterloo, Canada, (4) Germany, and (5) China. We next repeat the experiment, but this time using the overlay nodes we have set up with the cloud provider. More specifically, for each client and server pair, we perform the download seven times, each time Residential pilot test setup We used three clients located in () Tampa, Florida, US, () Paris, France, and (3) Cardiff, UK. The clients 3 six times for the client in Tampa, and seven times for the

4 using a tunnel that goes through a different overlay node. We then compute the maximum throughput observed among all the overlay paths.. PlanetLab experiment setup PlanetLab is a global research network with 343 nodes at 649 sites. We selected nodes at 5 sites, with 6 in North and South America, 8 in Europe, 5 in Asia, and in Australia. Each PlanetLab node can be representative of a remote user, or a branch office. However, we observe that a limit is frequently enforced on the total outbound bandwidth usage of the Planet- Lab nodes []: after a PlanetLab node sends approximately GB data in one day, its outbound bandwidth burst rate is capped at a limit ranging from kbps to 5 kbps. For this limit not to bias the measurement results, we therefore use one of the virtual servers as the TCP sender, and the PlanetLab nodes as the TCP receivers. As such, given a PlanetLab node (e.g., planetlab-3.cmcl.cs.cmu.edu), we have five candidate TCP senders, i.e., the virtual servers we have rented at Washington DC, San Jose, Dallas, Amsterdam and Tokyo. When one virtual server acts as a TCP sender (e.g., San Jose), the other four virtual servers (e.g., Washington DC, Dallas, Amsterdam and Tokyo) act as overlay nodes. Thus, we sample a total of 5 pairs of Internet end points (each with one direct and 4 overlay paths) in this experiment. Our purpose for this measurement is twofold. By significantly increasing the scale of experiment, we seek to quantify the prevalence of the performance gain. Further, by controlling both sender and receiver of each path, we are able to collect much richer data, that include not only throughput result, but also packet loss rate and round trip time. The latter two metrics can be as important as throughput for many applications such as video conferencing, and online gaming... Data collection For each pair of sender and receiver (A, B), we measure the TCP throughput of a 3-second data transfer, the TCP retransmission rate, and the average packet round trip time (RTT). The throughput is measured through iperf, and the retransmission rate and average RTT are derived using tstat [9]. Additionally, we collect traceroute data for all the paths. We have also examined the possibility of collecting the path capacity and available bandwidth estimates using several of the existing tools [8, 4,,, 8]. However, we found from lab experiments that the capacity and bandwidth estimates are not reliable for paths with high bandwidth links and large RTTs as reported by prior work [, 8]. An additional difficulty stems from clients in Paris and Cardiff, as we added one additional overlay node in Hong Kong the fact that the cloud nodes are virtual machines subject to software-based rate limiting, which may also significantly impact the accuracy of the estimation made by the tools. More specifically, measurements are collected for the following four different path types: Direct: We first measure the characteristics of the direct path A B. This measurement represents the performance of the default path as selected by the current Internet routing system. The other three paths are all through an overlay node (denoted by O) that we have set up in the cloud. Discrete overlay: The discrete measurement analyzes each segment A O, and O B, separately. This measurement does not take into account the overhead of the tunnels, and the processing overhead at the overlay node. The minimum of the two segments throughputs can serve as an upper bound of the achievable throughput through the overlay path. Tunnel overlay: We measure the characteristics of the path A O B, where O decapsulates/encapsulates the packets, modifies the source/destination IP addresses as a NAT, and forwards the packets. This tunnel overlay measurement represents the actual overlay performance over the path A O B. Split-TCP overlay: Some of the paths are transcontinental paths, where the RTT is likely to be large and potentially limit the TCP performance. We thus hypothesize that running a split-tcp proxy [, 3] at the overlay node may further improve the performance. Our insight is based on the following observation of TCP congestion control algorithm: Mathis et al. developed a simple model for the average bandwidth delivered by TCP for sufficiently long network flows [8]: BW MSS RT T p () where p is the probability that a packet is dropped. The equation highlights that the throughput over an overlay path A O B is not the minimum of the throughputs over the segments A O, and O B. For example, when the two segments have similar RTT, the RTT of the overlay path A O B doubles, and the throughput consequently gets halved. To test this hypothesis, we measure the characteristics of the path A O B, where the overlay node acts as a split-tcp, and breaks the TCP connection into two segments. This mode is applicable only when the end points do not enforce IPsec, but the TCP headers are in cleartext. 4

5 3. PERFORMANCE GAINS This section presents the results of the residential and PlanetLab measurements. 3. Residential Measurements The measured throughput for the pilot test is shown in Fig. 3. Each sub-figure (a), (b), (c) represents the performance from one specific client, each with five clusters of bars corresponding to the five remote servers. The first bar in each cluster shows the measured throughput for the directed path while the second shows the maximum throughput observed among the tunnel paths. We make the following observations. First, for a large fraction of the direct paths with severely low throughput, the corresponding overlay tunnel paths substantially improve the throughput, by to 8 times, with some overlay paths approaching the client s subscription upper bound; and this trend is consistent across all three client nodes. This confirms the feasibility of using a cloud-based overlay to improve throughput, and highlight its effectiveness. A particularly interesting and somewhat surprising observation is that, use of overlay can even significantly improve throughput between geographically nearby nodes, e.g., the path between Tampa and Georgia (in fact, the Georgia server is the recommended mirror by the Eclipse downloading webpage for the Tampa client), and between Paris and Germany. We believe this finding highlights the utility of the overlay, and also exposes the complexity in Internet path selection, e.g., the geographical distance may not be a good indicator of the path performance. Second, there are some cases, e.g., between Tampa and Germany, Paris and Virginia, where the use of overlay tunnel does not improve the throughput. However, we argue that for those cases, the throughput of the direct path is already good (e.g., greater than Mbps), some close to the subscription limit, and thus there is no need or room for improvement. Third, we notice that there are also three cases, i.e., between Tampa and China, Cardiff and Georgia, Cardiff and China, where the throughput of the direct paths is low, and the overlay tunnel path only offers moderate (less than times) improvement. These paths would greatly benefit from additional improvement. We observe that all these cases are transcontinental paths, where the RTT is likely to be large and potentially limit the TCP performance. (See discussion in Section...) To confirm, we repeated our overlay experiment for those three cases, but this time we ran split-tcp on the overlay node, in addition to using a tunnel as before. Fig. 4 shows the throughput improvement ratio for the best plain tunnel path and the best tunnel path with split-tcp, compared to the direct path. We observe that in all three cases, using split TCP in conjunction Throughput Improvement Ratio Plain Tunnel Tunnel with split TCP Tempa China Cardiff Georgia Cardiff China Figure 4: Split-TCP improved throughput further for the three transcontinental cases. CDF Tunnel overlay Split-TCP overlay Discrete overlay - 3 Max Overlay TCP Throughput/Direct TCP Throughput Figure 5: CDF on the throughput improvement ratios of plain overlay tunnels, tunnels with split TCP and discrete overlay, over the corresponding direct paths. with overlay further improves the throughput and more than doubled the throughput of the direct path. 3. Planetlab Measurements Next, we present the measurement results from our latest PlanetLab experiment, collected during January -7, Throughput Improvement We first quantify the throughput gain of using an overlay tunnel. For each source and destination node pairs, we compare the maximum throughput achieved by the four overlay tunnels with the throughput of the direct path. The solid blue curve in Fig. 5 shows the cumulative distribution function (CDF) of the ratio of the two throughputs. A point (x, y) indicates that for y% of the observed source and destination pairs, the overlay tunnel offers a throughput improvement ratio no less than x. Note that the X-axis represents the improvement ratio in logarithmic scale. We see that for 45% of the source and destination pairs, the improvement ratio is greater than, meaning that the throughput between those node pairs is improved by using the overlay net- 5

6 Throughput (Mbps) Direct path Tunnel overlay Throughput (Mbps) Direct path Tunnel overlay Throughput (Mbps) Direct path Tunnel overlay 5 Georgia, US Virginia, US Canada Germany China Georgia, US Virginia, US Canada Germany China Georgia, US Virginia, US Canada Germany China (a) client in Tampa, US (b) client in Paris, France (c) client in Cardiff, UK Figure 3: Measured throughput for file download from five web servers by three clients, using direct or overlay paths. work. Focusing on those node pairs with throughput gain, the average and median improvement ratios are 3.5 and.33 respectively. Interestingly, we observe that some paths get as high as over 4 times improvement. We discuss those paths in details in Section 4. We next repeat the experiment but with a split-tcp proxy running at the overlay node for further throughput improvement, as inspired by the residential measurements. We again consider the ratio of the maximum throughput achieved by the four overlay tunnels to the throughput of the direct path, and the result is shown in Fig. 5 by the dashed red line. We observe that, with split-tcp the overlay tunnel improves the throughput between 74% of the source and destination pairs compared to the direct path. Focusing on those node pairs with throughput gain, the average and median improvement factors are.8, and.38, respectively. Compared to the improvement achieved by plain tunnels, the additional gain from using split-tcp is substantial for a large fraction of Internet paths. These results highlight the effectiveness of split-tcp in reducing the perceived RTT of the end-to-end path and subsequently leading to better TCP congestion control behavior. Finally, we seek to understand the optimality of the split-tcp results. While we have demonstrated that split-tcp can effectively cut the perceived RTT, a potential concern is that split-tcp may introduce additional processing delay due to the proxy behavior. We thus compare the split TCP-throughput to that of the discrete overlay. Recall that the throughput of the discrete overlay may be considered as the upper bound of the end-to-end tunnel throughput (Section ). The result of the discrete overlay measurement is shown in Fig. 5 by the dotted green line. We see that, for 76% of the source and destination pairs, the use of overlay can potentially improve the throughput between them, as modeled by the discrete measurement result. For those node pairs, the theoretical average and median improvement factors are.36, and.4, respectively. Overall, the results are very close to those of the split-tcp tun- CDF Tunnel overlay Direct path TCP Retransmission Rate Figure 6: CDF on the TCP retransmission rates of the direct paths and of the overlay paths. The overlay paths reduce the median TCP retransmission rates by an order of magnitude. nels. Thus we can conclude that using split-tcp effectively achieves the optimal throughput that an overlay can offer. 3.. Packet Loss Reduction To infer the packet loss rates, we use tstat [9] and extract the ratio of number of retransmitted bytes, over the total number of bytes sent in the TCP payloads. TCP triggers a retransmission timer for each outbound segment that is handed down to IP. If no acknowledgment is received before the timer expires, the segment is retransmitted. As such, the TCP retransmissions may not exactly correspond to packet losses: a segment may be retransmitted not only when the segment is lost, but also when the segment is excessively delayed. However, both events cause a reduction in the TCP congestion window size, and lead to poor TCP performance. Figure 6 represents the CDF on the number of retransmissions experienced by the TCP transfers over 6

7 ..8 Direct Path Average TCP Throughput Max Split-TCP Average TCP Throughput CDF.6.4 TCP Throughput(Mbits/s) Overlay path average RTT/Direct path average RTT Figure 7: CDF on the ratios of the average RTT of overlay paths over that of the direct paths. Overlay paths reduces the average RTT for 5% of the Internet paths. the direct paths (red dotted line) and over the tunnel overlay paths (solid blue line). Recall that for each source and destination node pair, four overlay nodes may be used to set up tunnels. The numbers reported here represent the lowest TCP retransmission rates across the four tunnels for each node pair. A point (x, y) indicates that y% of the TCP transfers experience a TCP retransmission rate no less than x. Note that the X-axis is in logarithmic scale, and a value of. means that out of TCP segments is retransmitted. The figure reveals that the median TCP retransmission rate experienced over the direct Internet paths is.69 %, whereas that over the tunnel overlay paths is.66 3 %. As such, the overlay network reduces the median TCP retransmission rates by an order of magnitude Packet Round Trip Time Reduction We compute the average packet round trip time (RTT) through tstat, by measuring the time elapsed between the TCP data segments and their corresponding ACK [9]. As such, we capture not only the propagation delay, but also the queuing delay caused by congestion. For each source and destination pairs, we consider the average RTT of the minimum-rtt tunnel of all the four tunnel paths, and also the average RTT of the direct path. Fig. 7 plots the CDF on the ratio of the minimum average RTT of tunnel overlay paths to average RTT of the direct paths. The X-axis is in logarithmic scale. We observe that for 5% of the source and destination pairs, using the overlay tunnel can actually reduce the average RTT between them. Further, the figure also shows that for half of the source and destination pairs, the use of overlay does not increase the average RTT between them. To conclude, these results show that the overlay Path Index Figure 8: Comparison of the direct paths throughput and overlay paths throughputs for a subset of 3 Internet paths over a one-week period. network can be used to improve throughput for the vast majority of the Internet paths without significantly increasing the RTT; in fact, the overlay can actually be used to reduce RTT for many of the paths. 4. PERSISTENCY OF GAINS Section 3 demonstrated that a cloud-based overlay network can bring significant performance gains. In this section, we aim to answer two questions: First, will the performance gains be consistent over time, and attainable for a large number of end users? Second, how many overlay servers are needed? To answer these questions, we conduct a longitudinal study where we focus on the 3 direct Internet paths with the highest throughput improvements with a split-tcp at the overlay nodes, as measured in Section 3.. We sampled the direct paths throughput and corresponding Split-TCP overlay throughputs 5 times, at an interval of 3-hour over a 7-day period. Figure 8 depicts the results of the measurement. The X-axis represents the path index: path index corresponds to the Internet path with the largest improvement as measured in Section 3.; path index corresponds to the Internet path with the second largest improvement, etc. Each path has two bars: the left one represents the average throughput of the direct Internet path over the new measurement period, and the right one represents the average of the maximum observed throughput across the four tunnel paths over the same period. The error bar represents the standard deviation. First, with the exception of path indexes, and 4, all direct Internet paths consistently obtain a larger average throughput through the overlay network, throughput the one-week period. In other words, 9% of the 3 selected Internet paths get significant improvements across the measurement period, with an average 7

8 Min Overlay Number Required Path Index Number of Overlay Nodes Mean of Ave. Improvement Factors Median of Ave. Improvement Factors Table : Overlay Node Number vs Mean and Median of Ave. Improvement Factors..8.6 Figure 9: Minimum number of required overlay nodes for each Internet path to obtain the largest throughput across the measurement periods. CDF.4. improvement ratio of 8.39, and a median improvement ratio of We look more closely at paths indexes,, 3 and 4, and all of them have the same destination. In the previous measurement (Section 3.), those paths experienced the largest improvements. However, we observe that the throughputs of path indexes,, and 4, are now very close to the maximum achievable value of 8 mbps. This explains the reason the overlay network cannot further increase the throughput. We speculate that an intermediate ISP, common to path indexes,, and 4, was experiencing transient events during the time of the previous measurement. This case highlights the benefit of an overlay network to route around AS(es) with congestion or failure. Second, we observe that for majority of the 3 selected paths the standard deviation values are small, which indicates that the performance gains are consistent over time. Next, we seek to understand how many overlay nodes are needed. Figure 9 illustrates the minimum number of required overlay nodes for each Internet path to obtain the largest throughput across all the observed paths throughout the measurement period. We note that 7% of the 3 selected paths requires only one or two overlay nodes: for example, with only two overlay nodes (O, O ), path index is able to obtain the largest throughput across the measurement period. For some parts of the measurement period, the overlay path through O provides the largest throughput, while for the rest time, the overlay path through through O offers the best performance. We then vary the number of considered overlay nodes from to 4 overlay nodes, and compute the mean of the average improvement factors across the observed 3 Internet paths. The results, summarized in Table, highlight that a small number of overlay nodes needs Hop Count Ratio Figure : CDF of ratio (hop count of split-tcp overlay path providing the most improvement):(hop count of direct Internet path). to be deployed; and only one to two overlay nodes can provide most of the benefits. Another key question to answer is what locations should customers of overlay network deploy the overlay nodes at. We discuss more about this issue in Section UNDERSTANDING THE GAINS We have performed a series of analyses to understand the key factors behind the observed performance gains. The results are reported in this section. Our focus is on throughput gains as most TCP applications will benefit from an increased throughput. 5. What types of paths are more likely to see improvement? To answer this question, we first perform a routerlevel hop count analysis using the traceroute data. Surprisingly, 96% of the overlay paths with throughput improved by more than 5% have a longer hop counts than that of the corresponding direct paths (Figure ). We then turn our attention to the possibility of predicting throughput gain from only the packet loss and RTT characteristics of the direct paths. Fig. categorizes all direct paths based on their RTT values, using We have also examined the AS level hop counts for a subset of the paths. The same trend seems to hold. 8

9 Median Throughput Improvement Ratio paths [,7[ [7,4[ [4,[ [,8[ [8, RTT (in ms) Figure : Use of overlay is more likely to improve throughput when the direct paths have higher RTT values, and the level of improvements is also higher for those paths. Median Throughput Improvement Ratio 4 3 paths LR: [] 3 LR: ],.5[ 3 [,7[ [7,4[ [4,[ [,8[ [8, RTT LR: [.5, Figure 3: The throughput improvement of paths categorized by both RTT and loss rate. 5 Median Throughput Improvement Ratio paths [] ],.5[ [.5,.5[ [.5, Loss Rate Throughput Increase Ratio Direct Path Throughput (Mbps) Figure 4: Paths with smaller throughput get more improvement by using overlay. Figure : Use of overlay is more likely to improve throughput when the direct paths have higher packet loss rates, however even direct paths with zero loss can often get improvement as well. the five RTT bins as shown (X axis). The width of a bar represents the number of paths whose RTT values fall into the corresponding bin, and the height of a bar represents the median throughput improvement ratio for all those paths when the overlay network is used. The shade with each bar represents the fraction of paths that get positive improvement, i.e., having an improvement ratio greater than one, and this faction is represented by the fraction of the bar that the shade covers. Fig. is a similar figure but is based on packet loss rate instead. We make the following observations. First, for most RTT bins (except for the one of less than 7ms), and for all loss rate bins, use of the overlay network improves the throughput of more than half of the paths. The faction of improved paths is higher for bins of higher RTT or loss rate values. In particular, use of overlay improves more than 84% of the direct paths with RTT of 4ms or higher, and more than 86% of the direct paths with loss rate.5% or higher. Sec- ond, the throughput improvement ratio is greater for bins with larger RTT values. In particular, the median throughput is more than doubled for paths with RTT of 4ms or higher, and more than tripled for paths with RTT of 8ms or higher. The same trend also applies to loss rate when it is non zero. For paths with zero loss, however, we observed an interesting polarity: paths either do not get improved by using overlay at all, or they improve significantly as evidenced by the high median improvement factor. We hypothesize that the improvement for those zero-loss paths could be due to large RTT values they might have. Fig. 3 categorizes the direct paths based on both RTT and loss, where each row (column) represents a specific range of loss rate (RTT). We observe that, almost all paths with high RTT and high loss rate (i.e., bars at the bottom right of the figure) get significant improvement in throughput. On the other hand, while the majority of paths with low loss and low RTT (top left) get little improvement, a large fraction of paths with low loss but higher RTT (top right) do get significant improvement, which confirms our hypothesis above. Similar improvement also occurs to paths with low RTT but higher loss (bottom left). 9

10 RTT Decrease Ratio Direct path RTT (ms) Figure 5: Paths with higher RTT get more improvement by using overlay. Packet Loss Rate Decrease Ratio e-9 e-8 e-7 e-6 e-5 e-4 Direct path loss rate Figure 6: For vast majority of paths with non-zero packet loss rate, the use of overlay reduces the loss rate to zero. Given these observations, it is clear that paths with higher RTT and/or packet loss rate are more likely to get throughput improvement when the overlay network is used, and the improvement ratio is also typically higher for those paths. We believe this makes the overlay network particularly useful, as paths with high RTT/loss rate typically have small throughput and thus would benefit the most from the improvement. To validate our intuition, we plot Fig. 4. Each point represents a pair of source and destination nodes; the X axis plots the original throughput of the direct path, and the Y axis plots the maximum throughput increase ratio achieved by the best overlay path. The ratio is defined as the following: e-3 e- e- T overlay T direct T direct, if T direct > ;, if both T overlay and T direct are ; infinite, otherwise. where T overlay and T direct are the best throughput achieved by the overlay tunnel path and the throughput of the direct path, respectively. A positive value for this ratio e+ means that the use of overlay improves the throughput. The figure shows that direct paths with lower throughput are more likely to get improvement, and the level of improvement is also higher. In particular, almost all direct paths with throughput less than Mbps would be improved with the overlay, and the majority of them would see an improvement ratio higher than one, meaning that the overlay would more than doubled their throughput. Furthermore, we would like to also quantify the extent of RTT reduction from using the overlay. For this purpose, we define RTT reduction ratio as follows: RT T direct RT T overlay RT T direct, if RT T direct > ;, if both RT T overlay and RT T direct are ; negative infinite, otherwise. where RT T direct and RT T overlay are the RTT of the direct path and the smallest RTT of an overlay tunnel path, respectively. A positive ratio means that the use of overlay improves (i.e., decreases) the RTT, and the maximum reduction ratio is one. Fig. 5 characterizes the RTT improvement by the overlay, in a similar fashion to Fig. 4. It shows a clear trend that the use of overlay is more likely to reduce the RTT for direct paths with higher RTT values; in particular, the overlay would reduce the RTT for the vast majority of direct paths with an RTT of ms or higher. Similarly, we quantify how the use of overlay may also mitigate packet losses. For this purpose, we define the packet loss rate reduction ratio in the same way as the RTT reduction ratio. Fig. 6 characterizes the extent of packet loss reduction by the overlay. We note that, to avoid ambiguity, the figure only plots the sourcedestination pairs for which the direct paths has a loss rate greater than zero. It shows that, for the vast majority of those paths, an overlay path would reduce the loss rate to zero (i.e., achieve the maximum reduction ratio of one) regardless of their original loss rate. 5. How throughput gain is influenced by reductions of packet loss and RTT? We next seek to quantify the correlation of the changes in throughput, packet loss and packet RTT resulted from the use of an overlay tunnel path. For this purpose, we calculate the Pearson correlation coefficients between the throughput increase ratio, the RTT decrease ratio, and the packet loss rate decrease ratio. The coefficient between throughput increase and loss rate decrease ratios is -.5, between throughput increase and RTT decrease ratios is -.6, and between loss rate and RTT decrease ratios is.3. These result show that there is no significant correlation between any two of the three metrics alone. We then use the C4.5 algorithm [] to characterize

11 how the combined changes in packet loss and RTT may impact the change of throughput. For this purpose, we model the decrease ratios of packet loss and RTT as attributes, and consider two classes: throughput increases and throughput decreases. The algorithm produces the following decision tree: RT T <.476 : T hroughput decreases(345/3) RT T >=.476 : Loss <.774 : T hroughput decreases(46/4) Loss >=.774 : RTT >=.458 : T hroughput increases(47/) RT T <.458 : T hroughput decreases(56/7) Note that the first number in brackets following each leaf equals the number of instances which belong to that path in the tree, and the second number equals the number of classification errors encountered out of the total number of classifications made from the instances in that particular path of the decision tree. The results indicate that, when an overlay path decreases both the RTT and loss rate, by at least.5% and.%, respectively, it has a high likelihood to increase the throughput. This result is encouraging because such low reduction thresholds are likely to be met for most scenarios involving a direct path with inadequate performance, given the path diversity on the Internet. 6. ADAPTING TO NETWORK DYNAMICS Previous sections have demonstrated the potential benefits of a cloud-based overlay network. To deploy and benefit from it, end users need to address an additional key question: Given the dynamic nature of Internet paths, how to determine the best path to use? To address this question, researchers have traditionally developed algorithms to verify if a path is alive, and evaluate the quality of potential paths. Those algorithms typically rely on active probing, and therefore introduce overhead. In this section, we instead propose a novel solution, based on the recent MPTCP technology, to automatically select the best path(s). The solution requires the support of MPTCP at the end points, and can therefore be applicable to the connectivity between branch offices, and the connectivity between remote users and their corporate office. Section 6. describes the proposed solution, and Section 6. presents the conducted validation experiments. 6. Path selection through MPTCP MPTCP is a new set of TCP extensions that allow two hosts to use multiple paths concurrently between them, to improve throughput and reliability [9]. MPTCP is supported by major operating systems including Linux, Android, and ios. The basic approach in using MPTCP is to run MPTCP proxies [7], for example, at each branch office, or within the remote users VPN software, and map the end user originated TCP connections into MPTCP connections. Data packets are tunneled over an MPTCP connection between the MPTCP proxies, and then mapped back to the TCP connections at the egress MPTCP proxy. These MPTCP proxies would support robust communications in the presence of network failures, and adapt to network dynamics, transparently to end users and applications, and with minimal network overhead and outage duration. A MPTCP connection set up begins similarly to that of a TCP connection, with a TCP SYN, but the TCP SYN includes a new option, MP CAPABLE, for the end point to advertise its support of MPTCP. Once successfully established, this connection can be used to transfer data. Through TCP options, each endpoint can optionally advertise additional network addresses. Then, additional MPTCP subflows can be created, and combined to the existing MPTCP connection, to appear as a single connection to the user applications. Also, MPTCP includes connection-level sequence numbers to allow the end points to reassemble the segments arriving on the different subflows, and potentially out-of-order or duplicated. MPTCP has its own congestion control algorithm [3], which ensures that the total MPTCP throughput is at least as high as that of a single-path TCP connection on the best available path. At the same time, the MPTCP congestion control algorithm is designed not to take up more capacity on its different paths than if it were a single-path TCP connection using only one of these paths. This is to ensure that MPTCP will not degrade the performance of applications that use single-path TCP at shared bottlenecks. If one of the paths fails, MPTCP detects it and falls backs to using the remaining paths. MPTCP takes advantage of path diversity, and guarantees a throughput equal to that of a single-path TCP connection on the best available path. We exploit these capabilities of MPTCP by creating proxies at the sites to be connected. Each MPTCP proxy has access to N + paths, where N is the number of overlay nodes. One of these paths goes directly to the other proxy, while the other paths are reflected off the overlay nodes. If the default Internet path fails, the two proxies can still continue their connections through the overlay paths. This solution should incur minimal overhead as there is no separate need to probe the different paths and discover the available bandwidth on each of them. Instead, the MPTCP congestion control will infer this information based on the received ACKs for every sent data segment. 6. Validation experiments

12 Throuput(Mbits/s) Direct Path Throughput Overlay Path Throughput MPTCP Throughput Time(seconds) Figure 7: MPTCP Throughput Existing literature on MPTCP [3, 3, 4] has shown that MPTCP can be used to achieve performance of best available path within datacenter and wireless network. In order to further demonstrate that MPTCP can be used for automatic path-selection for cloud-based overlay network, we conduct a validation experiment using the same cloud platform as in Section. Specifically, we launch virtual servers at the Toronto, San Jose and Singapore data-centers. We use the servers at Toronto and Singapore as MPTCP proxies, and set up the server at San Jose as an overlay node for the direct path Toronto Singapore. All servers have a Mbps connectivity. The direct and overlay paths, each when used alone, can reach a throughput close to the maximum capacity. We launch a 8 seconds transmission using MPTCP leveraging both the direct path Toronto Singapore and the overlay path Toronto San Jose Singapore. We rate limit the throughput of the direct path in the middle of experiment to simulate a performance degradation of the direct path, for example, due to network congestion. Fig. 7 shows the results of the experiment. The blue dashed line represents the throughput of the direct path Toronto Singapore, the red dot-dashed line represents the throughput of the overlay path Toronto San Jose Singapore, and the black solid line represents the total MPTCP throughput. During the first 3 seconds of the experiment, the direct path and overlay path each has a throughput of around 5 Mbps. The total MPTCP throughput is close to Mbps. 3 The MPTCP throughput is then equal to that of a single-path TCP connection on the best available path, 3 The total throughput can occasionally exceeds Mbps because the cloud provider uses a software-based bandwidth cap. i.e., max( Mbps, Mbps). Thirty seconds after we start the experiment, we rate limit the throughput of the direct path to Mbps. Fig. 7 shows that MPTCP reacts to the throughput change of the direct path within seconds. Right after the rate limit of the direct path is imposed, MPTCP starts shifting traffic to the overlay path which has a higher throughput than the direct path. As a result, MPTCP is still able to maintain a total throughput close to Mbps, which is still equal to that of a single-path TCP connection on the best available path, i.e., max( Mbps, Mbps). The experiment demonstrates that MPTCP can automatically react to dynamic changes of Internet paths, and can be used to select best available path(s) in cloudbased overlay network. 6.3 Impact on overlay node positioning We observe that with built-in support for optimizing performance over k > paths for each traffic flow, the use of MPTCP greatly simplifies the design of BYOO, eliminating the need to determine a single best-performing overlay node, which can be a daunting task if it must be performed in real time and on a per source-destination pair basis, while at the same time not fundamentally changing the nature of the overlay node positioning problem. Consider the following scenario: A large enterprise has four branches {b, b, b3, b4} and the best overlay node locations for each of the four branches are {l, l, l3, l4} correspondingly. Without using MPTCP, the enterprise needs to pre-determine the best overlay node location for each of the branch and the total number of overlay nodes required will be 4. Leveraging MPTCP, the enterprise can deploy an equal number of overlay nodes at the same locations to achieve best performance without incurring the extra overhead of determining a new best overlay node for each of the branches when the network conditions change. While we expect a small k value ( 4) to be sufficient for most BYOO systems, the question of how MPTCP scales would arise if a scenario requires a large k value. We defer this question to future work. 7. DISCUSSION As with any Internet scale system, deploying a cloud routed overlay network will need to overcome many challenges. We identify and discuss some of them in this section. 7. Financial argument To break down the typical cost of setting up an overlay network vs. that of a private network, we assume an enterprise consisting of two branch offices in the East and West coast, respectively. Suppose the enterprise wishes to have a 6-Mbps (T) connectivity between the offices. Considering a private network cost of $ per

13 mbps per month [7, 3], a private WAN line would cost $/month. In contrast, a cloud overlay solution would cost $4 ( $/month for Internet access 4 for the two branch offices assuming a $ per mbps per month cost [3, 7]) + $9 (3 $3/month for three virtual overlay nodes) = $4/month with a limit of 5 TB transfer limit per overlay node. In other words, an overlay solution (albeit with performance not as predictable) would cost times less than a private WAN. Leasing private lines can be even more costly in other geographic locations [3, 3] making a cloud overlay an even more attractive cost-performance tradeoff point for some global enterprises. This financial argument is corroborated also by the emergence of startup companies, such as VeloCloud [7], which sell cloud based WAN services to home users and small enterprises. 7. Impact on Internet routing stability One natural concern would be that as cloud routed overlays attract more traffic, they may destabilize Internet routing. This can be a difficult problem to address if each end user must constantly switch between paths to find one with the best performance. We observe that MPTCP mitigates this problem naturally, by not requiring the end user to explicitly search for the best path. In essence, multiple MPTCP sessions over a same set of paths would load-balance these paths. Clearly, additional work is required to formally establish that the proposed overlay solution is indeed stable. 7.3 Handling of middleboxes Middleboxes such as NAT and firewall between an endpoint and a cloud hosted overlay can complicate the tunnel establishment between them []: for example, in the presence of a NAT, the IP address of the endpoint is dynamically assigned. As such, the overlay server can not be configured with a static remote tunnel IP address. Also, some firewalls may discard GRE packets. To detect and address the presence of NATs, we rely on the Next Hop Resolution Protocol (NHRP) [7]. This protocol was defined in 998 to optimize routing in non-broadcast multiple access (NBMA) networks such as ATM, and is now commonly used in dynamic multipoint virtual private network solutions. NHRP assumes a hub and spoke topology. By defining a cloud overlay as a hub, the endpoint as a spoke, and configuring the spoke with an IP address of the hub, the endpoint would register its IP address with the overlay server, and the NHRP protocol would detect the presence of NAT(s) and establish an IPsec tunnel through them. Most firewalls would let IPsec traffic pass through as well. 4 ISPs apply different charging models. Some may charge per Mbps connectivity. Others may charge per traffic volume. For simplicity, we assume the first model. 8. RELATED WORK The Detour framework [5, 5] first showed that a large fraction of Internet paths could get improved performance through indirect routing [6]. A number of proposals then ensued. For example, Andersen et al. proposed Resilient Overlay Networks (RON) []: Through a wide-area deployment, RON was shown to improve the loss rate, latency or throughput of data transfers. Miyao proposed an overlay architecture to accelerate content delivery between datacenters []. Akamai s commercial offering SureRoute [9] to route around failures and improve performance is also based on an overlay system. More recently, Peter et al. developed and evaluated a system called ARROW [4] that allows end users to create tunnels to participating ISPs, and stitch together end-to-end paths. ARROW s main focus is to enhance the robustness of end-to-end paths against attacks and failure events. Building upon these efforts, we investigate the feasibility of leveraging the relatively inexpensive and readily available resources from the cloud and the emerging MPTCP technology to significantly lower the deployment barrier. Furthermore, we examine the low-level path characteristics to understand the key factors behind the performance gains. Source control over AS level routing has also been suggested to improve the reliability of Internet paths. For example, Yang et al. presented a solution that gives users the ability to choose the sequence of providers the data packets traverse [33]. Gummadi et al. implemented a one hop source routing solution to recover from Internet path failures [5]. While viable as a long term option, such source routing currently is often blocked by intermediate ISPs due to security concerns [6]. 9. CONCLUSION We have proposed and made an empirical case for end users to build their own overlays, by renting intermediate nodes from cloud providers, to boost their network performance at a reasonable cost and without waiting for ISPs to take actions. For future work, we will focus on developing a systematic methodology for intelligently positioning and relocating the overlay nodes in order to further reduce cost and increase network performance. We are also interested in having a deeper understanding of the security implications of transferring data over cloud overlays and the interactions between the proposed overlay solution and the various security protocols.. REFERENCES [] L. A. Alt, J. P. Rohrer, and G. G. Xie. Demo: Application-transparent deployment of dtn via smartnet. In Proceedings of the 9th ACM MobiCom Workshop on Challenged Networks, CHANTS 4, pages 93 96, New York, NY, USA, 4. ACM. [] D. Andersen, H. Balakrishnan, F. Kaashoek, and M. Robert. Resilient overlay networks. In SOSP,. 3

14 [3] A. Bakre and B. R. Badrinath. I-tcp: Indirect tcp for mobile hosts. In ICDCS, 995. [4] Y. Chen, Y. Lim, R. J. Gibbens, E. M. Nahum, R. Khalili, and D. Towsley. A measurement-based study of multipath tcp performance over wireless networks. In Proceedings of the 3 Conference on Internet Measurement Conference, IMC 3, pages , New York, NY, USA, 3. ACM. [5] A. Collins. The detour framework for packet rerouting Master s thesis, University of Washington. [6] L. Crosby. What s next? $. billion investment. 5 new data centers. January 4. Available at whats-next---billion-investment-5-new-data-centers/. [7] L. Deng, D. Liu, T. Sun, M. Boucadair, and G. Cauchie. Use-cases and requirements for mptcp proxy in isp networks, October 4. [8] C. Dovrolis, P. Ramanathan, and D. Moore. Packet-dispersion techniques and a capacity-estimation methodology. IEEE/ACM Trans. Netw, 4. [9] A. Ford, C. Raiciu, M. Handley, and O. Bonaventure. Tcp extensions for multipath operation with multiple addresses, January 3. RFC684. [] The Eclipse Foundation. Eclipse. [] E. Goldoni, G. Rossi, and A. Torelli. Assolo, a new method for available bandwidth estimation. Internet Monitoring and Protection, International Conference on, 9. [] E. Goldoni and M. Schivi. End-to-end available bandwidth estimation tools, an experimental comparison. In International Conference on Traffic Monitoring and Analysis,. [3] Andy Gottlie. Why does MPLS cost so much more than internet connectivity? April. Network World. [4] M. Jain and C. Dovrolis. Pathload: A measurement tool for end-to-end available bandwidth. In Passive and Active Measurements (PAM) Workshop,. [5] S. D. Gribble H. M. Levy K. P. Gummadi, H. V. Madhyastha and D. Wetherall. Improving the reliability of internet paths with one-hop source routing. In OSDI, 4. [6] E. Katz-Bassett, H. V. Madhyastha, V. K. Adhikari, C. Scott, J. Sherry, P. van Wesep, T. Anderson, and A. Krishnamurthy. Reverse traceroute. In NSDI,. [7] J. Luciani, D. Katz, D. Piscitello, B. Cole, and N. Doraswamy. Nbma next hop resolution protocol (nhrp), April 998. RFC33. [8] M. Mathis, J. Semke, J. Mahdavi, and T. Ott. The macroscopic behavior of the tcp congestion avoidance algorithm ACM CCR. [9] M. Mellia. Tcp statistic and analysis tool. Available at [] Y. Miyao. An overlay architecture of global inter-data center networking for fast content delivery. In IEEE ICC,. [] PlanetLab. Bandwidth limits. Available at [] J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 993. [3] M. Popovic R. Khalili, N. Gast and J. L. Boudec. Mptcp is not pareto-optimal: Performance issues and a possible solution. Networking, IEEE/ACM Transactions on, (5):65 665, Oct 3. [4] Q. Zhang D. Woos T. Anderson A. Krishnamurthy S. Peter, U. Javed. One tunnel is (often) enough. In ACM SIGCOMM, 4. [5] S. Savage, T. Anderson, A. Aggarwal, D. Becker, N. Cardwell, A. Collins, E. Hoffman, J. Snell, A. Vahdat, G. Voelker, and J. Zahorjan. Detour: a case for informed internet routing and transport. IEEE Micro, 999. [6] S. Savage, A. Collins, E. Hoffman, J. Snell, and T. E. Anderson. The end-to-end effects of internet path selection. In ACM SIGCOMM, 999. [7] Jonathan Shieber. Virtualizing wide area networks, velocloud rings up $ million. June 4. [8] J. Sommers, P. Barford, and W. Willinger. A proposed framework for calibration of available bandwidth estimation tools. IEEE Symposium on Computers and Communications (ISCC), 6. [9] Akamai Technologies. Sureroute. Available at com/dl/feature_sheets/fs_edgesuite_sureroute.pdf. [3] TeleGeography. IP transit price declines steepen. August. Available at products/commsupdate/articles//8// ip-transit-price-declines-steepen/. [3] D. Wischik, C. Raiciu, A. Greenhalgh, and M. Handley. Design, implementation and evaluation of congestion control for multipath tcp. In NSDI,. [3] D. Wischik, C. Raiciu, A. Greenhalgh, and M. Handley. Design, implementation and evaluation of congestion control for multipath tcp. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, NSDI, pages 99, Berkeley, CA, USA,. USENIX Association. [33] X. Yang, D.Clark, and A. Berger. Nira: A new inter-domain routing architecture. IEEE/ACM Trans. Netw., 7. 4