Inbound Traffic Load Balancing in BGP Multi-homed Stub Networks

The 28th International Conference on Distributed Computing Systems Inbound Traffic Load Balancing in BGP Multi-homed Stub Networks Xiaomei Liu and Li Xiao Department of Computer Science and Engineering Michigan State University East Lasing, MI 48823 {liuxiaom, lxiao}@cse.msu.edu Abstract Multihoming load balancing improves network performance by leveraging the traffic among the access links in a multi-homed network. Currently, no effective load balancing system is available to handle the inbound traffic in a BGP multi-homed stub network, where the traffic volume is unknown to the network and the route of the traffic is hard to control. In this paper, we propose ILBO, an inbound traffic load balancing mechanism to effectively balance the inbound traffic in a BGP multi-homed stub network. ILBO predicts and schedules the inbound traffic based on outbound traffic. It also provides an inbound traffic control scheme that can guarantee the successful execution of the traffic scheduling. 1 Introduction BGP multi-homed stub networks are stub networks that implement multihoming with Border Gateway Protocol (BGP) routers to provide reliability and redundancy [1]. In order to further improve network performance and lower the user cost of link financial charges, multi-homing load balancing is introduced to actively leverage the traffic among multiple access links of the multi-homed network [2]. A load balancing mechanism generally assigns the traffic based on link status and the traffic flow size [2 5]. It should also be capable of controlling the traffic to make sure that the traffic uses the assigned link. The traffic being balanced can be either the inbound traffic, which comes into the network, or the outbound traffic, which goes out of the network. Since the outbound traffic is generated by stub networks, stub networks have the information about outbound traffic and can directly control to which access link the outbound traffic is sent. This makes it easy to implement outbound traffic load balancing. However, stub networks have no information about the inbound traffic and can not directly control the inbound traffic, which makes it difficult to deploy inbound traffic load balancing for the BGP multi-homed network. In order to successfully balance the inbound traffic, a load balancing system must be able to predict the flow size of the inbound traffic so that the incoming links can be assigned before the traffic enters the network. The system should also include an effective inbound traffic control scheme to guarantee that the inbound traffic comes into the network via the link that is assigned to it. However, most inbound traffic route control systems either not include traffic prediction schemes or the prediction schemes are not accurate enough to predict inbound traffic flows. Existing inbound traffic control schemes change the inbound traffic route by changing the BGP advertisement [6,7]. This type of methods are not capable to achieve dynamic load balance due to the propagation time of the advertisement. The result of these method are also limited due to the path asymmetry and the localized routing decisions of each ISP in the Internet. In this paper, we propose a multihoming load balancing mechanism: ILBO (Inbound traffic Load Balancing based on the Outbound traffic) to tackle the aforementioned challenges and implement dynamic load balancing. ILBO uses outbound traffic to control, predict, and schedule the inbound traffic. ILBO is composed of three modules: twostep traffic predictor, hybrid inbound traffic controller, and the traffic scheduler based on the above two modules. The two-step traffic predictor provides the inbound traffic prediction with an accuracy of above 80% by combining a Support Vector Machine (SVM) classifier and the traditional traffic prediction mechanism. The scheduler schedules the inbound traffic based on the outbound traffic and the prediction results of the two-step traffic predictor. The hybrid inbound traffic controller controls the inbound traffic by changing the source IP index of outbound traffic packets via encapsulation. It only updates the route advertisements when a long term change in the traffic volume is detected or there is a permanent change in the upstream ISPs. The remainder of the paper is organized as follows: Sec- 1063-6927/08 $25.00 2008 IEEE DOI 10.1109/ICDCS.2008.112 367 369

tion 2 reviews the related work. Our solution for the inbound traffic load balancing is presented in Section 3. The setting of the simulation and the performance evaluation are presented in Section 4. Section 5 concludes the entire work. 2 Related work The benefits of multi-homed networks have been investigated in works [8,9]. These studies quantitatively evaluate the reliability and network performance improvement that multi-homed networks achieve. They also show that multihoming load balancing can significantly contribute to the improvement of the network performance, which is the motivation for our work. Products to implement multihoming load balancing in BGP multi-homed networks are available by extending BGP protocol [10, 11]. However, the technical details of these products are not exposed to the public. In addition, most of these products do not implement real inbound traffic load balancing in a BGP multi-homed network. Akella et al. [3] investigate the influence of the different link monitoring schemes in the multihoming load balancing system. Guo [2] investigates the influence of the link monitoring mechanism, the traffic assignment granularity, and link features on the multihoming load balancing with a commercial product from Rether Networks [12]. New schemes were proposed recently to reduce financial charges of multihoming links and improve the network performance in multihoming load balancing systems [4, 5, 13]. Goldenberg [5] and Dhamdhere [4] address the traffic scheduling issues. However, the work of Dhamdhere [4] only schedules the outbound traffic, while the work of Goldenberg [5] treats the inbound traffic and outbound traffic in the same way. Since the inbound traffic is not generated by the stub network and the traffic information is unknown to the stub network, processing the inbound traffic the same way as outbound traffic may restrict the effectiveness of scheduling scheme. In addition, their approaches are based on the assumption that the traffic can be assigned to any access link and none of the studies provide inbound traffic control mechanism to guarantee the execution of the scheduling results. This restricts the use of these approaches in balancing the inbound traffic in the BGP multi-homed networks. Dhamdhere [4] proposes a solution for the ISP subscription problem to reduce the link financial charge and improve the network reliability. They also propose a mechanism to select high performance and congestion free links (and thus paths) for the outbound traffic. The existing inbound traffic control mechanisms generally control the inbound traffic in a BGP multi-homed network by modifying the BGP route. They update the BGP route advertisements on the request of load balancing and propagate the advertisements across the Internet [6, 7]. These methods require a considerable time to propagate route changing information across the Internet. This propagation induces heavy traffic overhead. In addition, the stub network has little control on how the other ISPs propagate and use the route in the advertisement. Virtual peering also controls inbound traffic using packet encapsulation [14]. Compared with our method, virtual peering is aimed at traffic control in general stub networks. Virtual peering requires the establishment of IP tunnels between the virtual peer controllers for the packet encapsulation. In other words, the remote stub network encapsulates the inbound traffic packets upon receiving the Virtual Peering Establishment messages. The tunnel establishment generally takes at least a few hours. Virtual Peering Removal messages are sent upon the closer of tunnels. The cost on tunnel establishment and maintenance may hinder its usage in dynamic traffic allocation. Our scheme is a stateless method. No specific messages are sent out for establishing IP tunnels. The remote stub networks encapsulate packets upon receiving the encapsulated outbound traffic packets from the local stub networks. In other words, the encapsulation of the inbound traffic is revoked by the outbound traffic packets that encapsulated with the specific load balance address. This saves the cost of tunnel establishment/maintenence and can be used to implement the real dynamic inbound traffic load balance. Network address translator (NAT) can help implement the inbound traffic control. The inbound traffic in a NAT multi-homed network can be controlled via a domain name server (DNS)-redirect mechanism, which binds a host inside the network with different IP addresses [2, 15]. NAT does not preserve the transport-layer survivability, i.e., the re-homing transparency for transport layer sessions [1,16]. NAT multi-homed networks remove the uniqueness of the host IP address since NAT maps multiple hosts to one IP address. This hinders the network in its capacity to support the applications that require the unique IP address of the participants for the communication, for instance, the peerto-peer file sharing systems and endpoint authentication in voice over IP (VOIP) systems [17, 18]. Our research focuses on the BGP multi-homed networks. 3 Load Balance Scheme Design The following subsections describe in detail the three components of the ILBO: the two-step traffic predictor, the hybrid inbound traffic controller, and the traffic scheduler. When the outbound traffic is coming, it is first sent to the two-step predictor. The predictor predicts the volume of the potential inbound traffic that will come to the stub network using the outbound traffic and the historical data of the inbound traffic flows of the network. The outbound traffic is then sent to the traffic scheduler, which assigns links 368 370

to both the outbound traffic and the predicted inbound traffic. Finally, the hybrid traffic controller sends the outbound traffic to its assigned link and guarantees the inbound traffic caused by the outbound traffic comes back in the assigned link. 3.1 Two-step Traffic Predictor The two-step traffic predictor predicts the traffic flow rate, which is the traffic volume of a flow in a given time interval. The traffic with the same pair of source IP address and the destination IP address is considered as the same flow. The flow rate of the traffic extracted from the Abilene traces [19] shows an obvious bursty characteristic, ranging from 10 1 to 10 5 bytes per second. In order to better predict the bursty feature of the traffic, we propose to predict the flow rate in two steps. The predictor makes a coarse prediction about the magnitude order of the traffic flow rate in the first step. Specifically, the predictor detects whether the magnitude of the flow rate is in the order of 10 1, 10 2, 10 3, etc. Based on the coarse prediction result achieved in the first step, the predictor predicts the exact flow rate in the second step. We adopt a SVM classifier in the first step to predict the magnitude order of the traffic flow rate [20]. SVM is an effective classification tool used in the machine learning field. The essential issue of using SVM is to construct a SVM training model, which decides the parameters and algorithms that are used in the classification process of SVM. In our case, we need to correctly map our problem into input/output variables and also provide proper data sets for the training process of the SVM. Our first attempt is to use the magnitude order of the outbound traffic as the input variable and the magnitude order of the predicted inbound traffic as the output variable. However, the result of the classification is fairly bad (around 63%). We then include the time gap between the time tick when the current outbound traffic flow appears and the last time tick when this flow appeared in the network. The experiments shows the inclusion of the time gap in the input variable greatly improves the accuracy of the prediction of the magnitude order of the inbound traffic (with an average of 93.1%). The detail of SVM model construction is shown as follows. Assume the flow rate of an inbound traffic flow is v i and the magnitude order of the flow rate is log(v i ). The volume of the outbound traffic is v o and its magnitude order is log(v o ). The time gap between the outbound traffic flow and its previous one is t. Let y = log(v i ) ; x 1 = log(v o ) ; x 2 = t. Let Y =[y]; X =[x 1,x 2 ]. Then we can construct the SVM training model of X and Y: N Minimize (1/2w T w)+c ξ i=1 Table 1: Flow rate prediction Flow rate orders 10 10 2 10 3 10 4 10 5 1 81.3% 47.5% 57.1% 68.7% 80.7% 5 84.3% 71.8% 63.6% 77.7% 78.8% Mean 10 85.4% 60.9% 64.4% 80.9% 74.1% based 20 86.1% 64.2% 66.4% 75.6% 75% 30 86.6% 64.9% 68.6% 75.5% 62.8% 50 87% 53.7% 58.2% 69.7% 65% 200 86.5% 53.7% 58.2% 69.7% 65% EWMA 54% 83.2% 81.4% 63% 52.7% History data size subject to the constraint Y i (w T X i + b) 1 ξ i ξ i 0, i =1, 2,..., N The goal of the training model is to find out the value of w by a given data set X and Y. w is the vector of coefficients; b is a constant; and ξ is a very small non-negative value that is close to 0. i labels the values of N training cases (X i,y i ), and C is the capacity constant. C can be decided beforehand via a cross-validation method. In our case, we found that a value of C around 0.03 can result in the best prediction accuracy in SVM model for the collected trace data. After the training process, we can get Y via the trained model: Y = w T X i + b w T w is the kernel function of the SVM. We found that the performance of the different models are similar in our case. We thus adopt the linear kernel function, which is the simplest one and can thereby reduce the computation time. As for the training data size, we have tested on 1000 five minute time interval data files (the data file in Abilene traces are generated every five minutes) from the traces of 5 different ASes. The number of flow samples in each file ranges from 1288 to 8937. We find the training data set size required for these sample sets is fairly consistent, falling in the range with a lower bound of 71 to a upper bound of 90 to achieve at least 90% accuracy. One issue that needs to be discussed here is the computation overhead of SVM. As only two elements are introduced in the X variable in the training model, the training time is fairly small. In a PC running SUSE Linux 9.3 with 500MB memory and a 1.7GHZ CPU, the time to train the model of a training set of 71 to 90 ranges from 57 to 83 ms, and time to classify 1288 to 8937 flow samples ranges from 5ms to 23ms. The model training time can be further improved using an online training model such as the one introduced in paper [21]. The exact flow rate is then predicted in the second step. For the inbound traffic flows associated with the current outbound traffic, the flow rates that have the similar magnitude order in the latest n time 369 371

intervals constructs a flow rate set V h of historical data. V h = {v i (t) :log(v i (t)) = log(v i (t 1))} The historical data here are not necessary the historical data from the predicted flow. The flow rate of the most recent time interval is noted as v curr. We investigated the prediction solutions based on the mean of historical data, the median of historical data, and exponentially weighted moving average (EWMA) model. The prediction value of flow rate in EWMA model is β v curr +(1 β) v 2, where β is a value between 0 to 1. v 2 is the flow rate in time 2. We have tested 8000 flows from the traces of 5 different ASes. The results show that the best prediction solution for the flows of different size is different. For the flow whose flow rate is in the order of 10 2 and 10 3, the EWMA achieves most accurate result. For the flows of other flow rate, mean based solutions achieve better result than other methods. At the same time, the prediction accuracy for flows with the similar size but from different ASes is consistent. Table 1 shows the prediction accuracy of the flows from AS3 as an illustration. 3.2 Hybrid Inbound Traffic Controller The design of hybrid inbound traffic controller includes two parts. If a long term performance change has been observed or the stub network wants to change the basic traffic routing policy, the system will update the route advertisements to reflect the change. Otherwise, the traffic control is provided by packet encapsulation of outbound traffic (e- Control mechanism). The e-control mechanism can help implement the dynamic load balancing and we give a detailed introduction here. The e-control mechanism is proposed based on the following observations: 1) it is not necessary to control the entire route of the inbound traffic, but only the access link through which the inbound traffic enters the stub network in the multihoming load balancing,. That is, the last hop of the inbound traffic. 2) BGP is a destination based forwarding paradigm, forwarding a packet only based on the destination address of the packet. As a result, the access link that a packet of the inbound traffic is routed to is decided by the destination address of this packet. Therefore, it is possible to control the incoming access link of the inbound traffic by the outbound traffic that causes the inbound traffic. For a multi-homed network with the e-control mechanism, the e-control mechanism reserves one IP address for each IP address block that is inherited from an upstream ISP. This reserved IP address is called load balance address (LB address) and used for the outbound traffic encapsulation. Note, e-control mechanism only deals with the inbound traffic that is originated inside the stub network. That is, the inbound traffic that is induced by the request of a host inside the stub network. This type of the inbound traffic contains the data returned by the remote servers upon the user s request and is much heavier than the corresponding outbound traffic. On the other hand, the inbound traffic originated outside the stub network is generally the user request, which itself is a light flow but incurs heavy outbound traffic, which can be controlled by the stub network directly. This type of traffic can be affected, although not dynamically, by the static routing policy change. The e-control mechanism is illustrated in Figure 1. The stub network X is multi-homed to the Internet via ISP-A and ISP-B. X inherits the IP prefix of 65.3.10.0/19 from ISP-A and 198.18.32.0/24 from ISP-B. Router BGP-R A is connected to ISP-A via link l a, and router BGP-R B is connected to ISP-B via l b. By default, if a host with 65.3.10.7/19 sends a request to its remote destination located in AS-Y with the IP address 10.10.1.35/19, host 10.10.1.35 will send back the inbound traffic with the destination IP address 65.3.10.7. As BGP is a destination based routing protocol, the inbound traffic eventually will be routed to ISP-A, which has a sub-ip prefix 65.3.10.0/19, and be sent back to AS-X via link l a. In Figure 1(a), X wants the inbound traffic from host 10.10.1.35 to be sent to it via link l b due to the load balancing requirement. In this case, the packets of request from host 65.3.10.7 will be encapsulated with the LB address 198.18.32.1 by router BGP-R B. In the destination network of AS-Y, the request is decapsulated in the router. The edge router keeps an encapsulation list of the LB addresses of each source network. Each entry of the list includes an LB address and the real source address of the packet that encapsulated by this LB IP address. If the source IP address of the packet received is an LB IP address, the packet will be decapsulated and sent to the upper host. In Figure 1(b), the responses from host 10.10.1.35 are sent back to AS-X. In this case, the destination IP address will be looked up in the encapsulation list. If this address is in the list, the packet will be encapsulated with the related LB IP address, here, it is 198.18.32.1, and sent to AS-X. Due to the BGP protocol, the packets will be routed to ISP- B and eventually come in AS-X via link l b. The received packets are then be decapsulated by BGP-R B and sent to relative hosts. The deployment of e-control mechanism does not require any support from upstream ISPs or third parties. However, both source and destination stub networks should run the encapsulation mechanism in order to implement the e- Control mechanism. In practice, this may not happen at one click but can be accomplished gradually (Note a stub network can put e-control into effect as long as any of the its destination stub networks also support the e-control mechanism ). One method is to construct a public board that publishes the IP prefixes of the stub networks that have already adopted this mechanism. The stub network can then label 370 372

(a) Outgoing traffic encapsulated by LB address (b) Incoming traffic is sent into l b link Figure 1: e-control mechanism all destination IP indices that implement this encapsulation in its routing table and thus only encapsulates the packets with the destination IP index that supports the decapsulation function. Another way is to encode the information of encapsulation in the BGP local community attributes and propagate the information across the Internet using BGP advertisement. The stub network can label all destination IP indices upon receiving the advertisement. The e-control mechanism avoids the long propagation time and thus enables a BGP multi-homed stub network to implement real dynamic load balancing. It also helps reduce the route update messages and cut the BGP routing table size. This provides the motivation for stub networks to adopt the e-control mechanism. As for the implementation issues, the encapsulation/decapsulation techniques have already been used in other network layer protocol designs such as IPv6 tunnelling and the failure handling process in multi-homed networks and implemented in existing networks [22, 23]. The easiness of encapsulation/decapsulation technique implementation will also encourage the deployment of the e-control mechanism. 3.3 Traffic Scheduler Given a moment t, the scheduling process should schedule both the inbound traffic flows f(t) that come to the stub network at the current moment and the outbound traffic flows g(t) that go out of the stub network at the current moment. In addition, we use ˆf(t) to denote the traffic coming at t moment and caused by g(t 1). Assume there are K access links a stub network connects, and N outbound and inbound traffic flows. In a perfect world, the goal of traffic scheduler is to assign the N flows to the the K links in a way that can minimize the financial cost and maximize the performance. Suppose each flow is not dividable, and each link has a limited capacity E l. Then, the traffic distribution problem can be presented as the follows. { ( ) argmin cost l (f(t)+g(t))+( perf l (f(t)+g(t))) ( kli klj i=1 j=1 (fi(t)+gj(t)) ) E l cost l (f(t)+g(t)) is the link charge function for link l. perf l (f(t) +g(t)) is the performance function of link l. However, inbound flows f(t) cannot be scheduled at t moment: even if they are scheduled, their routes and incoming links could not be changed. The above formulas thus can be changed to: { argmin (cost l (g(t))+( perf l (g(t)))) ( klj j=1 gj(t) ) E l k li i=1 fi(t) Since the incoming links being used by f(t) are already fixed at t, k li i=1 f i(t) is a constant. This is an integer programming problem, which is already NP hard. It will take K N f(t) steps to get the optimized solution using a brute force method. Therefore, instead of obtaining an optimal solution, a sub-optimal solution can be achieved by a greedy algorithm. Here we propose to sort the outbound flows according to their flow rate at t, and then assign each flow in an descending order (based on flow rate) such that an overall maximum benefit based on cost and performance can be achieved. However, the above mechanism only provides benefit upon outbound flows. We can schedule ˆf(t) at moment t 1 with the help of two-step predictor and e-control system to achieve better overall benefit upon inbound flows. Accordingly, we can schedule ˆf(t +1)at moment t. We modify the proposed greedy algorithm and show the pseudo code of the modified algorithm in Algorithm 1. 4 Performance Evaluation In order to evaluate the performance of ILBO, we have deployed a series of simulations based on the real world Internet traces: Abilene traces [19]. Abilene network is an IP backbone network of the Internet 2 and connects to over 220 Internet2 universities, corporations, and affiliate member institutions across the United States. We have extracted from Abilene traces the traffic data of 5 multi-homed stub ASes. Table 2 presents the total traffic data size of each AS and its basic information obtained from the the Cooperative Association for Internet Data Analysis (CAIDA) [24]. The 371 373

Algorithm 1 Sub-optimal solution of traffic scheduler Require: inlink {link candidate for inbound traffic} Require: outlink {link candidate for outbound traffic} sort outbound flows in descending order based on flow rates for each outbound flow g i do for each link l =1...K do if (l achieves larger benefit than selected outlink) AND (g i <available capacity of l) then select l as outlink end if end for end for sort predicted inbound flows in descending order based on flow rates for each inbound flow f i do for each link l =1...K do if (l achieves larger benefit than selected inlink) AND (f i <capacity of l) then select l as inlink end if if l is different than the related outlink then encapsulate the related outbound flow end if end for end for Table 2: Abilene trace AS Upstream AS Trace Organization number ISPs class size 3 MIT 4 edu 2.1G 59 Univ. of Wisconsin, Madison 2 edu 1.2G 70 National Library of Medicine 3 comp 2.3G 1701 DoD(network information center 3 nic 1.7G 22753 Redhat inc. 3 comp 360M AS class edu denotes a school, comp means business companies, nic stands for network information center. The network topology is simulated as follows. We assume the stub network connects to the Internet via 3 ISPs. Most stub networks are multi-homed to 3 ISPs and it is reported that multihoming to more than 3 ISPs only achieves marginal network performance improvement [8,9]. The capacity of the 3 access links are 45 Mbps, 25 Mbps, and 100 Mbps, with the unit prices computed based on the ISP price data in the previous reports [5, 13]. In the following subsections, we first present the results of the user cost of link financial charge, following by the network performance and the link utilization ratio. We adopt the total volume charge model to calculate the user cost. The end-to-end path delay is the time period from when a packet is injected to the access link until when it arrives at the receiver. The link utilization ratio is defined as the ratio of the link load over its bandwidth. As the results of the different ASes are consistent and due to the page limit, we representatively present the results of two ASes of different organization classes: AS1701, a network service organization, and AS3, an education organization. 4.1 Link Financial Charge The user cost of the link financial charge of AS1701 and AS3 in a one day period is shown in Figure 2 and Figure 5 respectively. The cost was computed every two hours. The curve of No inbound traffic control represents the load balancing mechanisms without considering inbound traffic such as those proposed in papers [4, 5]. In these mechanisms, the outbound traffic is assigned to a link such that the assignment can minimize the overall link charge cost. The access link where the inbound traffic comes into the network is the same link being used by the outbound traffic that induces the inbound traffic because no specific inbound traffic control mechanism is employed in this scenario. The curve of Round Robin represents the mechanism that assigns the traffic flows into each link in turn. We used round robin algorithm as the base line and normalized the cost values according to the cost of Round Robin mechanism to make the results more universal. It can be observed in Figure 2 and Figure 5 that both load balancing mechanisms can reduce the user cost of link financial charges compared to the round robin mechanism. In addition, ILBO can reduce more user cost than the load balancing system without balancing inbound traffic. In AS1701, only balancing the outbound traffic can reduce the user cost by 22.6% on average, while ILBO can further reduce the user cost by 21.3%. In AS3, only balancing the outbound traffic reduces the user cost by about 23.3%, while ILBO further reduces the user cost by 11%. 4.2 End-to-end Path Delay The end-to-end path delays of AS1701 and AS3 under different traffic assignment schemes are illustrated in Figure 3 and Figure 6. We present the end-to-end path delays of a half hour period to better show the patterns of the endto-end delays. Each point in the figure represents an average end-to-end delay of a 10 second period. Like the link financial charge, the end-to-end path delay is smaller in the network adopted load balancing mechanisms than that in the network that assigns the traffic in the round robin way. However, compared with the ILBO mechanism, balancing only the outbound traffic induces limited reduction. In AS1701, balancing only the outbound traffic reduces endto-end path delays by 17.6% on average, while ILBO fur- 372 374

Figure 2: Cost of link financial charge (AS1701) Figure 3: (AS1701) End-to-end path delay Figure 4: Link utilization ratio (AS1701) Figure 5: Cost of link financial charge (AS3) Figure 6: (AS3) End-to-end path delay Figure 7: Link utilization ratio (AS3) ther reduces end-to-end path delays by about 19%. In AS3, balancing only the outbound traffic reduces end-to-end path delays by 11.4% on average, while ILBO further reduces end-to-end path delays by about 12.1%. 4.3 Link Utilization Ratio Given the same link-load, the link utilization ratio (linkload/link-utilization) should be reverse proportional to the link-bandwidth. However, since the ultimate goal of our load balancing mechanism is to optimize the user cost and network performance, the link with better network performance and lower link financial charge may have a heavier load than the other links. The results are presented in Figure 4 and Figure 7. We can observe that the link utilization ratio is reverse proportional to the link bandwidth in the round robin mechanism. This suggests that similar amount of traffic load is assigned to each links in the round robin mechanism. In both load balancing mechanisms, there are not obvious relations between the link utilization ratio and the link bandwidth. This may be because the load balancing systems schedule the traffic based on both the end-to-end path delay and the link financial charge. Link 1 is always assigned fairly heavy traffic load in all cases, while the utilization ratio of link 2 and link 3 varies a great deal. This suggests the bandwidth of link 1 is much higher than link 2 while its price is much lower than link 3. 5 Conclusion and Future Work In this paper, we propose an inbound traffic load balancing system, ILBO, to balance the inbound traffic in BGP multi-homed networks. ILBO schedules the inbound traffic separately from the outbound traffic. With the combination of the SVM classifier and the statistical prediction model, the two-step predictor can predict the inbound traffic flow rate with the accuracy of above 80%. ILBO provides a hybrid inbound traffic controller to enable the dynamic assignment of the inbound traffic by encapsulating the packets of the outbound traffic. This method avoids the route table update process, and thus requires no fail-over time. We have evaluated the performance of ILBO using the real world Abilene Internet traces. The simulation results demonstrate that ILBO can achieve better network performance as well as lower user cost of link financial charge. Future work will be deployed in two directions. First, we will investigate more prediction methodologies and look for a prediction method that can predict the traffic in one step 373 375

to further reduce the computation time. Second, we plan to implement our proposed inbound traffic control scheduler and evaluate the ILBO system in an emulation environment. 6 Acknowledgement This work was supported in part by the US National Science Foundation under grants CCF-0514078, CNS- 0551464, and CNS-0721441. References [1] J. Abley, Y. Rekhter, K. Lindqvist, E. Davies, B. Black, and J. Gill, RFC 4116: IPv4 Multihoming Practices and Limitations, 2005. [2] F. Guo, J. Chen, W. Li, and T.-C. Chiueh, Experiences in Building A Multihoming Load Balancing System, in IEEE INFOCOM, vol. 2, 2004, pp. 1241 1251. [3] A. Akella, S. Seshan, and A. Shaikh, Multihoming Performance Benefits: An Experimental Evaluation of Practical Enterprise Strategies, in USENIX Annual Technical Conference, Boston, MA, 2004, pp. 113 126. [4] A. Dhamdhere and C. Dovrolis, ISP and Egress Path Selection for Multihomed Networks, in INFOCOM, 2006. [5] D. K. Goldenberg, L. Qiu, H. Xie, Y. Richard, and Y. Y. Zhang, Optimizing Cost and Performance for Multihoming, in SIGCOMM, Portland, OR, 2004, pp. 79 92. [6] R. K. C. Chang and M. Lo, Inbound Traffic Engineering for Multihomed ASs Using AS Path Prepending, IEEE Network, vol. 19, no. 2, 2005. [7] B. Quoitin, S. Uhlig, and O. Bonaventure, Using Redistribution Communities for Interdomain Traffic Engineering, Lecture Notes in Computer Science, vol. 2511, pp. 125 134, 2002. [8] A. Akella, J. Pang, B. Maggs, S. Seshan, and A.Shaikh, A Comparison of Overlay Routing and Multihoming Route Control, in SIGCOMM, Portland, OR, 2004, pp. 93 106. [10] Cisco, Sample Configurations for Load Sharing with BGP in Single and Multihomed Environments. [Online]. Available: http://www.cisco.com/wrarp/public/459/40.html [11] RouteScience. [Online]. Available: http://www.routescience.com [12] Rether Networks. [Online]. Available: http://www.rether.com [13] H. Wang, L. Qiu, A. Silberschatz, and Y. Yang, Optimal ISP Subscription for Internet Multihoming: Algorithm Design and Implication Analysis, in INFO- COM, vol. 4, Miami, FL, 2005, pp. 2360 2371. [14] B. Quoitin and O. Bonaventure, A Cooperative Approach to Interdomain Traffic Engineering, in EuroNGI, 2005. [15] X. Liu and L. Xiao, A Survey of Multihoming Technology in Stub Networks: Current Research and Open Issues, IEEE Network, vol. 21, no. 3, pp. 19 30, 2007. [16] J. Abley, B. Black, and V. Gill, RFC 3582: Goals for IPv6 Site-Multihoming Architectures, 2003. [17] B. Ford, P. Srisuresh, and D. Kegel, Peer-to-Peer Communication Across Network Address Translators, in USENIX Annual Technical Conference, Anaheim, CA, 2003, pp. 179 192. [18] M. Holdrege and P. Srisuresh, RFC3027: Protocol Complications with the IP Network Address Translator, 2001. [19] Abilene Trace. [Online]. Available: http://abilene.internet2.edu [20] T. Mitchell, Machine Learning. McGraw Hill, 1997. [21] A. Bordes and L. Bottou, The Huller: A Simple and Efficient Online SVM, Lecture Notes in Computer Science, 2005. [22] A. Conta and S. Deering, RFC2473: Generic Packet Tunneling in IPv6 Specification, 1998. [23] T. Bates and Y. Rekhter, RFC2260: Scalable Support for Multi-homed Multi-provider Connectivity, 1998. [24] CAIDA AS Taxonomy and Attributes. [Online]. Available: http://www.caida.org [9] A. Akella and A. R. Sitaraman, A Measurement- Based Analysis of Multihoming, in SIGCOMM, Karlsruhe, Germany, 2003, pp. 353 364. 374 376