Parallel LSPs for constraint-based routing and load balancing in MPLS networks

Parallel LSPs for constraint-based routing and load balancing in MPLS networks J. Tang, C.K. Siew and G. Feng Abstract: Two features of multiprotocol label switching are very useful in network traffic engineering: the path-oriented nature, and the capability to support multiple paths between an ingress egress node pair. The first feature makes it easy to adaptively route traffic through the network based on the load condition in different parts of the network, while the second feature is often used for load balancing. The role of parallel label-switched paths (LSP) in load balancing and constraint-based routing is investigated. An algorithm named parallel-path-based bandwidth scheme (PPBS) is proposed to make use of parallel LSPs in choosing a bandwidth constraint path. The improvement on flow blocking probability by using PPBS is given quantitatively with respect to the average traffic load on the link, the hops along the path, and the possible number of parallel paths. In conjunction with the PPBS scheme, a feedback-based load-balancing algorithm (FBLB) is proposed to properly distribute traffic onto the parallel LSPs determined by the PPBS. This FBLB algorithm relies on the signalling packets to convey network status information back to the source. Consequently the sources can adjust the traffic distributionintoeachlsp accurately andpromptly. Simulation results show that the FBLB algorithm is simple and effective. 1 Introduction Network traffic engineering (TE) has received extensive attention in recent years [1 7]. With the objective to optimise network performance by strategically mapping the traffic to the network resources, it has attracted the interest of both network operators and researchers. Multiprotocol label switching (MPLS), with its ability to support explicit routing and parallel paths, has been well accepted as an appropriate technology for network traffic engineering. The research work on traffic engineering in MPLS networks mainly focuses on two areas: constraint-based routing in Internet protocol (IP) networks using MPLS [8, 9] and load balancing using multiple label-switched paths (LSP) [2 4]. Some notions of constraint-based routing have been supported in current network devices (e.g. some asynchronous transfer mode switches), but MPLS has made it feasible and attractive in public IP networks. To extend the current constraint-based routing implementations to accommodate the peculiar requirements of MPLS, we have at least one of two ways: by extending the current interior gateway protocols (IGP) such as open shortest path first (OSPF) and intermediate system to intermediate system (IS IS) to support constraint-based routing [1, 11], or by adding a constraint-based routing process to each router that can coexist with current IGPs [8]. In either way, the basic routing procedure is the same: first prune resources that do not satisfy the requirements of the traffic trunk attributes, then run a shortest-path algorithm on the residual graph. If no path can be found on the residual graph the flow is blocked. r IEE, 25 IEE Proceedings online no. 24968 doi:1.149/ip-com:24968 Paper first received 28th October 24 and in revised form 13th July 24 The authors are with the School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Avenue, 639798 Singapore In the context of MPLS network another approach to constraint-based routing is to precompute a set of LSPs (called feasible paths) from an ingress node to the egress node. When a new flow arrives a path is selected for that flow from the set of feasible paths based on the requirements of the flow and the available resources on the paths [9, 11 13]. In the light of this, much research work is devoted to compute the set of LSPs and to choose an appropriate path from the precomputed set in an effort to reduce the blocking probability of the flows [9]. Generally, path selection can be based on topology, network resource or dynamic adjustment [14]. In a topology-based algorithm, the shortest path in the set is examined to see if it meets the quality of service (QoS) requirement. If yes, that path is selected, otherwise that path is deleted from the feasible set. This process continues until a path is selected or the set of feasible paths is empty, in which case the flow is blocked [14]. The capacity-based algorithm precomputes the feasible set of paths the same way, but it examines the paths from the one with the lowest available bandwidth (lowest bandwidth first). The dynamic algorithm reroutes the existing traffic flows to find a path for the new flow if the capacity-based algorithm fails to find a path for the new flow [14]. The first-improved dynamic load-balancing algorithm proposed in [15] is an example of the dynamic adjustment scheme, whereby paths crossing the critical links are rerouted when congestion is detected in the network. From this discussion, one can see that all these efforts attempt to locate a single path for a traffic flow that meets the bandwidth requirement. However, one of the advantages of the MPLS networks is the ability to offer connectivity to their clients in the form of multiple end-toend LSPs with bandwidth and/or delay guarantees. By creating aggregate, bandwidth-reserved flows, they offer flexible routing, predictable bandwidth usage, and QoS provisioning. This flexibility in routing allows the implementation of traffic engineering and makes dynamic 6 IEE Proc.-Commun., Vol. 152, No. 1, February 25

multipath routing feasible. Obviously, the constraint-based routing schemes discussed do not suffice in MPLS networks with traffic engineering functions. We propose a parallelpath-based bandwidth scheme (PPBS) to resolve the bandwidth constraint for multipath routing in MPLS networks. We show that in an MPLS network with dense connectivity the proposed PPBS algorithm can greatly reduce the blocking probability of an aggregated traffic flow. A consequence of the proposed PPBS algorithm is that the bandwidth requirement of an aggregate traffic flow may only be met by a number of LSPs. Therefore a necessary task of the ingress router is to distribute the traffic of the flow aggregate to all the supporting paths appropriately. We consider the traffic between an ingress egress node pair of an MPLS network as an aggregate of a large number of individual flows, and the out-of-sequence packets (if any) caused by the delay difference on LSPs will be handled by the end user. In [2] the MPLS adaptive traffic engineering (MATE) is proposed to avoid network congestion by adaptively balancing the load among multiple paths based on measurement and analysis of a cost function. In [3] an algorithm based on measured delay is proposed to achieve similar goals. And in [4], a switch architecture is proposed to distribute flows inside the switch and among the parallel links. However, the MATE algorithm is very difficult to implement due to its complexity, and delay-based load balancing in [3] implies that the available bandwidth in a path is inversely proportional to the packet delay obtained by a probing scheme, which does not apply to a complex network. In [4], load balancing issues are considered but the focus is on the switch architectures. We notice that accurate information of the network status, specifically the available bandwidth along each possible path, is the key to effective load balancing. The feedback-based load-balancing (FBLB) algorithm proposed in this paper obtains the bandwidth information based on explicit feedback of probing packets. Probing in the proposed algorithm can be implemented easily using existing signalling protocols (e.g. resource reservation protocol (RSVP), label distribution protocol (LDP)) with slight modification or extension. Therefore the ingress router can have an up-to-date database of the explicit available bandwidth along each path to an egress router, and can allocate the traffic accordingly. We show that the FBLB algorithm is stable and very easy to implement. 2 Parallel-path-based bandwidth scheme 2.1 Model We model an MPLS network by a set L of unidirectional links. The network is shared by a set S of ingress egress node pairs, indexed 1; 2;:::;S. Each of these pairs s ðs 2 SÞ has a set P s of LSPs available to it. By this definition P s s (s 2 S) are disjoint sets, which means no two (distinct) pairs use the same LSP, even though some of their LSPs may share links. An ingress egress pair s has a total input traffic of rate r s and may send x sp of it on LSP p 2 P s such that X p2p s x sp ¼ r s ; 8s 2 S ð1þ rates on all LSPs that traverse link l x l ¼ X X x sp ð2þ s2s l2p;p2p s Define y sl as the available bandwidth to pair s on link l,that is y sl ¼ C X X x ip ð3þ i2s;i6¼s l2p;p2p i where C is the total bandwidth on link l. The available bandwidth to a pair s on its path p is then defined as y sp ¼ min fy slg ð4þ l2p The constraint-based routing is to find a path p for pair s, that satisfies y sp r s ð5þ The problem is that if r s is big, and no single path p meets the requirement y sp r s, the flow requiring r s amount of bandwidth will be blocked even though r s can be possibly supported by two or more paths in P s. We solve the problem in two steps: expand y sp in (5) to the summation of all y sp, p 2 P s using the proposed PPBS algorithm; and then find appropriate x sp in (1) based on knowledge of y sp using the proposed FBLB algorithm. 2.2 PPBS algorithm The parallel-path-based bandwidth scheme can be described as follows: Precompute the paths for each edge pair s (a) Find the shortest path in the graph. If the path exists, then put it in set P s andgoto(b) else end the precomputation process. (b) Delete the links and routers along the path found in (a) from the network graph, except the ingress and egress router,thengoto(a). For a flow aggregate with bandwidth requirement of r s if r s P y sp, p2p s then accept the flow else block the flow. To precompute the paths for an edge pair s we employ a node-disjoint path method where the first route is the minimum link metric path (e.g. shortest path), and the kth route is the minimum link metric path on the reduced network, where the routers and links along the first route, second route,...,(k)th route are pruned except source and destination routers. Examples of this computation is illustrated in Fig. 1. In this example, the first feasible path source second path (4 hops) first path (3 hops) destination Let x s ¼fx sp ; p 2 P s g betheratevectorofs, andletx ¼ ðx sp ; p 2 P s ; s 2 SÞ be the vector of all rates. The traffic load on a link l 2 L, denoted by x l, is the sum of source Fig. 1 third path (4 hops) Example of node disjoint paths IEE Proc.-Commun., Vol. 152, No. 1, February 25 7

between the source and destination node is the shortest path of three hops, and the second (third) path is the shortest one with the routers and links along the first (first and second) path pruned from the graph. As a result the three paths computed for the source destination pair do not share any link, that is, they are disjoint. The paths computed in this algorithm are parallel paths in the sense that no two paths share a same link. One advantage of these parallel paths lies in the fact that the correlations or traffic interference among the paths between the same source destination pair is removed, and the parallel paths can be considered independent of each other, which greatly facilitates bandwidth provisioning and performance analysis. The downside of this method is that the number of parallel paths between a pair depends on the connectivity of the network, or more specifically, on the ports of the label switching routers including the ingress node. For example, if the ingress node has only three output ports, then at most three parallel paths can be found between the source and destination. Since the parallel feature of these paths is very useful in bandwidth allocation and load balancing, we use this node-disjoint path method forpathprecomputationinthispaper. Consider an ingress egress pair s with a set of nodedisjoint paths P s and the available bandwidth y sp on path p. When a new flow aggregate arrives with bandwidth requirement r s, the aggregate is accepted by the network if the following holds: r s X y sp ð6þ p2p s Compared with traditional constraint-based routing, (6) expands the bandwidth of an acceptable flow aggregate to the summation of available bandwidth on all the parallel paths instead of that of a single path, which greatly reduce the blocking probability of new flows, especially when the overall traffic load in the network is high. 2.3 Performance analysis of PPBS algorithm Consider a general network model in Fig. 2. The available bandwidth on the links along the path i is denoted by o i1 ; o i2 ; :::; o imi, respectively, where m i is the number of links along path i. Similarly, the available bandwidth for the source destination pair along path i is denoted by y i y i ¼ min fo ij g ð7þ j¼1;:::;m i The available bandwidth on a link changes dynamically, so o ij is a random variable. Suppose the probability density function (PDF) of o ij is f oij, and links in the network are independent of each other, then we have the following lemma: Lemma 1: If the available bandwidth on all the links have the same pdf, that is, f oi;j ¼ f o, the PDF of the available bandwidth y i along path i, denoted by f yi,isgivenby f yi ðyþ¼m i f o ðyþr m i o ðyþ ð9þ where m i is the number of links along path i, R o is the reliability function of o, which is defined as R o ðoþ ¼Pfo4og ¼1 F o ðoþ, F o is the probability distribution function of variable o. Proof: Suppose m i ¼ 2, then y i ¼ minfo i1 ; o i2 g,from[16] we have, f yi ðyþ¼f oi1 ðyþr oi2 ðyþþf oi2 ðyþr oi1 ðyþ ð1þ R yi ðyþ ¼R oi1 ðyþr oi2 ðyþ ð11þ Since f oi1 ðoþ ¼f oi2 ðoþ ¼f o ðoþ and R oi1 ðoþ¼ R oi2 ðoþ¼r o ðoþ, f yi ðyþ¼2f o ðyþr o ðyþ ð12þ R yi ðyþ¼r 2 o ðyþ ð13þ Without loss of generality, suppose (9) holds when m i ¼ k; k 2, and f yi ðyþ ¼m i f o ðyþr m i o ðyþ ¼ kf o ðyþro k ðyþ; k 2 ð14þ R yi ðyþ¼r m i o ðyþ¼rk oðyþ; k 2 ð15þ then for m i ¼ k þ 1, y i ¼ minfo i1 ; o i2 ;:::;o ik ; o iðkþ1þ g ¼ minfy i; o iðkþ1þ g where y i ¼ minfo i1 ; o i2 ;:::;o ik g From [16] f yi ðyþ ¼f y i ðyþr oiðkþ1þ ðyþþf oiðkþ1þ ðyþr y i ðyþ ð16þ ð17þ ð18þ source ω 12 ω 11 ω 21 ω n1 ω n2 ω 2m2 ω nmn ω 1m1 destination From the assumptions in (14) and (15), and f oiðkþ1þ ðoþ¼f o ðoþ, (18) becomes f yi ðyþ ¼kf o ðyþro k ðyþr oðyþþf o ðyþr k o ðyþ ¼ðk þ 1Þf o ðyþr k o ðyþ ¼ m i f o ðyþr m i o ðyþ ð19þ Fig. 2 Parallel paths between source and destination Since we focus on only one ingress egress pair in this Section we have omitted the subscript s of the variable y in (7) and continue to use y i to represent y si for simplicity in the rest of this Section. If there are n paths between the pair, the total available bandwidth Y between the pair is Y ¼ Xn i¼1 y i ð8þ That is (9) holds when m i ¼ k þ 1, and this completes the proof. & Lemma 2: The probability density function of the total available bandwidth Y between an ingress egress pair is given by f Y ðyþ¼f y1 ðyþ,f y2 ðyþ,...,f yn ðyþ ð2þ where n is the number of parallel paths between the pair, y i ði ¼ 1;:::;nÞ is the available bandwidth on a contributing path i, f yi is obtained in lemma 1, and, is the convolution operation defined as f x1 ðyþ,f x2 ðyþ ¼ R 1 f x1ðy xþ f x2 ðxþdx. 8 IEE Proc.-Commun., Vol. 152, No. 1, February 25

Proof: Suppose y ð2þ ðyþ ¼y 1 ðyþþy 2 ðyþ, thenfrom[16] f y ð2þðyþ ¼f y1 ðyþ,f y2 ðyþ ð21þ For y ð3þ ðyþ¼y 1 ðyþþy 2 ðyþþy 3 ðyþ¼y ð2þ ðyþþy 3 ðyþ, we have f y ð3þðyþ ¼f y ð2þðyþ,f y3 ðyþ ¼ f y1 ðyþ,f y2 ðyþ,f y3 ðyþ ð22þ Similarly for Y ðyþ¼y 1 ðyþþy 2 ðyþþ::: þ y n ðyþ¼y ðnþ, f Y ðyþ¼f y ðnþðyþ¼f y ðnþðyþ,f yn ðyþ ¼f y ðn 2ÞðyÞ,f yn ðyþ,f yn ðyþ... ¼f y1 ðyþ,f y2 ðyþ,...,f yn ðyþ ð23þ Theorem 2.1: For a single-path constraint-based routing network the blocking probability of a flow requiring bandwidth r is p b ¼Pfy 1 or; y 2 or;...; y n org ¼ ::: f y1 ðoþdo f yn ðoþdo f y2 ðoþdo ð24þ where n is the number of feasible paths between the edge node pair and f yi ðoþ is defined in lemma 1. For a PPBS network with disjoint-node parallel paths, the blocking probability of a flow requiring a bandwidth of r is obtained by p b ¼ PfYorg ¼ f Y ðoþdo ð25þ where f Y ðoþ is defined in lemma 2. Equations (24) and (25) follow directly from the assumption that the parallel paths between an edge node pair are independent of each other and the definition of the blocking probability: the probability that a flow (aggregate) with bandwidth requirement r cannot be accepted by the network because the network cannot meet the bandwidth requirement r. Theorem 1 gives a quantitative blocking probability of a flow with certain amount of bandwidth requirement, both in a single-path traditional constraintbased routing and in a parallel-path network using PPBS. Fromlemmas1and2itiseasytoseethatwiththesame bandwidth requirement r, the blocking probability in PPBS (25) is much lower than that in a single-path network (24). For illustration purpose only, Figs. 3 to 7 give an numerical example of evaluating the flow-blocking probability using theorem 2.1. In this example, each link has a capacity of 15 Mbit/s. The available bandwidth on a link is assumed to be a random variable that follows a normal distribution Nðm ¼ 8, s 2 ¼ 4Þ. This is a viable assumption in a network with relatively large bandwidth and a large number of individual flows [16]. The parameter m ¼ 8 sets the expected available bandwidth to 8 Mbit/s, and the shape parameter s 2 ¼ 4 makes sure that the available bandwidth is distributed in the range of 15 Mbit/s. Figure 3 shows the blocking probability of a flow with bandwidth requirement r, in a single-path scheme. It can be seen that the blocking probability increases when the number of hops along the path grows. When the bandwidth requirement is 5 Mbit/s, the blocking probability is about 12% in a two-hop path, but it becomes as high as 5% on a ten-hop path. Figure 4 shows the probability density function of the available bandwidth along a path with block probability of request 1.8.6.4.2 2 hops 4 hops 6 hops 8 hops 1 hops 5 1 15 bandwidth requirement, Mbit/s Fig. 3 Blocking probability of single path scheme (with one possible path) probability density function.4.3.2.1 Fig. 4 1 link 2 links 4 links 6 links 8 links 1 links 5 1 15 available bandwidth, Mbit/s PDF of available bandwidth (single path) different hops. The shape of the PDF graph becomes narrower and shifts to the left hand when the number of hops along a path grows, which explains the phenomenon in Fig. 3. Figures 5 and 6 illustrate the relationship between the blocking probability and the bandwidth requirement in a single-path scheme and PPBS scheme, respectively, with a block probability 1.8.6.4.2 5 1 15 bandwidth requirement, Mbit/s 1 path 2 paths 3 paths 4 paths Fig. 5 Blocking probability of single-path scheme (multiple possible path) IEE Proc.-Commun., Vol. 152, No. 1, February 25 9

block probability 1.8.6.4.2 Fig. 6 different number of feasible paths between the ingress egress nodes. We consider the range of the number of parallel paths from 1 to 4 for illustration. Hops along each path is assumed to be the same (eight hops in Figs 5 and 6). Figure 5 shows that when the number of feasible paths increases the blocking probability is reduced to some extent in single-path constraint-based routing. However, the reduction effect becomes unnoticeable when the number of feasible path is more than three or four. When the bandwidth requirement is greater than about 6 Mbit/s in this example, increasing the number of feasible paths does not reduce the blocking probability tangibly. Figure 6 shows a completely different behaviour in a PPBS scheme. In this Figure every additional feasible path not only decreases the blocking probability significantly, but also greatly expands the range of acceptable bandwidth requirement. For example, when there is one feasible path, a flow requiring 5 Mbit/s bandwidth will experience a blocking probability of about 3%. But if the number of feasible paths becomes four, a flow will experience a blocking probability of 3% only when its bandwidth requirement reaches 185 Mbit/s. Intuitively, this is because in the PPBS scheme, more parallel paths implies that each path has a smaller share of the traffic from a flow, thus the network as a whole can support higher traffic load. Figure 7 shows the probability density function of the available bandwidth in a parallel-path scheme, which clearly explains the phenomenon in Fig. 6. probability density function Fig. 7 5 1 15 2 25 3 bandwidth requirement, Mbit/s.3.2.1 Blocking probability of PPBS scheme (eight hops) 5 1 15 2 25 3 available bandwidth, Mbit/s 1 path 2 paths 3 paths 4 paths 1 path 2 paths 3 paths 4 paths PDF of available bandwidth in PPBS scheme (eight hops) 3 Feedback-based load balancing algorithm As a result of the PPBS algorithm the traffic between an ingress egress pair is shared by a number of parallel LSPs between them. If the traffic is not properly distributed to the LSPs, load unbalancing may occur; congestion may occur along some paths while other paths are lightly loaded. Therefore load-balancing becomes an integral part of the functions of an ingress router that employs PPBS scheme. In fact, proper load balancing can minimise the utilisation of the most heavily loaded links throughout the network so that no bottleneck link exists, and proper traffic distribution along parallel LSPs enables the efficient utilization of network resources and prevents any link from being overly congested due to heavy demand of user s traffic. 3.1 FBLB algorithm It is easy to understand that if accurate information of the load condition in the network is available, more traffic can be transmitted through the lightly loaded links of the network while avoiding the heavily loaded links. If such information is accurate and the load adjustment is prompt, congestion will be well controlled. However, it is not a trivial task to obtain the network status information accurately. In [2], a cost function is evaluated based on measurement, which is used for traffic adjustment with the objective to minimise the cost in the network. In [3], the available bandwidth is implicitly assumed to be inversely proportional to the measured packet delay via a probing scheme. But since measurement is difficult to implement, and some measured parameters (e.g. delay in [3]) cannot reflect the load condition accurately, we propose a new feedback-based scheme for load balancing. FBLB algorithm Ingress router (a) Maintain a database for each egress router s (or more accurately an ingress egress pair s from this ingress router). The database may contain the set of parallel paths P s, and available bandwidth y sp ; p 2 P s etc. (b) Send probing packets periodically along all parallel paths to egress routers (to collect the information on available bandwidth along each path). (c) On return of the feedback packet, update the available bandwidth y sp in the database for the corresponding path p and egress router s. (d) After time interval T, adjust the traffic load on each path: compute the traffic share x si on path i (i ¼ 1; :::; m) to egress node s (s 2 S), according to the following x si ¼ y i P m j¼1 y j r s ð26þ where m is the number of parallel paths for egress router s, r s is the total traffic between the ingress and egress routers. Intermediate router: When an intermediate router receives a probing packet, it updates the available bandwidth (ABW) field in the probing packet according to its load condition, and passes the packet to the downstream routers. Egress router: The egress router returns the probing packets to the ingress router after updating the ABW field. 1 IEE Proc.-Commun., Vol. 152, No. 1, February 25

In this algorithm we suppose each router in the network keeps its own state information. In other words, a router knows how much bandwidth is in use for which LSP on each output port. This feature is already commercially available in some routers (e.g. RSVP-capable routers). The source router then sends probe packets periodically to the destination, each probe packet containing an ABW field. Routers along the path will modify the ABW field based on their status and the destination will revert the probe packet to the source. And every time the ingress router receives a feedback packet it will update its database of the available bandwidth accordingly. After a predefined time interval, the ingress router will adjust the load distribution based on the information in its database. Since the source can send probe packets along all the parallel paths it knows the exact available bandwidth along each parallel path, and thus can allocate the traffic accordingly. The load distribution algorithm in (26) implies that the traffic share along a path is proportional to the available bandwidth on that path. Since the available bandwidth is provided by the feedback mechanism, the load distribution algorithm directly works towards minimising the utilisation of the heavily loaded links to avoid network congestion. As stated earlier, the FBLB algorithm does not impose complex computation but it does require some collaboration between the routers involved. First, the ingress routers need to maintain a database to store the information of the LSPs to each egress router, and the available bandwidth along each path. Secondly, all the routers involved need to support the probing packet. Thirdly, all the intermediate routers need to keep track of the bandwidth allocated to each LSP to accurately update the probing packets. However, considering the fact that a signaling protocol is already in use in MPLS networks, implementation of the FBLB would require slight or moderate extention of the existing protocol. Therefore the implementation cost would not be high compared with the benefit gained. 3.2 Simulation results Simulation is conducted to evaluate the effectiveness of the algorithm based on the network model in Fig. 8. In the network model ða 1 ; b 1 Þ, ða 2 ; b 2 Þ and ða 3 ; b 3 Þ are three pairs of ingress egress nodes. Each pair has two parallel paths, pairs ða 1 ; b 1 Þ and ða 2 ; b 2 Þ share link l 1,pairsða 1 ; b 1 Þ and ða 3 ; b 3 Þ share link l 2 and pairs ða 2 ; b 2 Þ and ða 3 ; b 3 Þ share link l 3. We call the traffic from these three pairs the engineered traffic, and background traffic on the links the cross traffic. Here we suppose cross traffic passes through links l 1, l 2 and l 3, making them the bottleneck link for all the three pairs, respectively. Assume the total bandwidth of each link is 15 Mbit/s, and the pattern of cross traffic on links l 1, l 2 and l 3 is shown in Fig. 9. All the nodes in the network employ FBLB scheme, where intermediate routers convey their available bandwidth information to the sources (a 1, a 2 and a 3 )that source nodes a 1 destination nodes x 11, x 21 b 1 l 1 cross traffic Fig. 9 15 1 5 2 4 6 8 1 Pattern of cross traffic will adjust the traffic distribution according to the algorithm (see (26)). Figures 1 and 11 show the traffic load on the bottleneck links when the bandwidth requirement from all three source is 5 Mbit/s, respectively. Figure 1 shows the aggregated traffic from the engineered flows, and Fig. 11 shows the total traffic load (engineered traffic+cross traffic) on the links. In these Figures, when the cross traffic on the bottleneck link increases, the engineered traffic decreases 15 1 5 2 4 6 8 1 Fig. 1 Aggregate traffic on bottleneck links (bandwidth requirement ¼ 5 Mbit/s) 15 1 5 x a 12, x 31 2 l 2 x a 22, x 32 3 l 3 Fig. 8 Network model b 2 b 3 2 4 6 8 1 Fig. 11 Total traffic on bottleneck links (bandwidth requirement ¼ 5 Mbit/s) IEE Proc.-Commun., Vol. 152, No. 1, February 25 11

accordingly. Similarly the engineered traffic increases on a link if the cross traffic is reduced. When the cross traffic remains stable for a long time the engineered traffic load converges to a stable state accordingly. In Fig. 11 the pattern of the total traffic follows the background traffic. But since the engineered traffic is adjusted dynamically, the fluctuation of the total traffic is much smoother than the cross traffic. However, since the engineered traffic is relatively light compared with the cross traffic in Fig. 11, the pattern of the total traffic is dominated by the cross traffic. Figures 12 and 13 shows the traffic load on the bottleneck links when the bandwidth requirement from the engineered flows is higher (8 Mbit/s). In Fig. 12 the aggregated engineered traffic shows a similar pattern of change as in Fig. 1 with a larger value of the absolute load. In Fig. 13 the traffic load fluctuates more than in Fig. 11, and the engineered flows may experience burstiness or even packet loss, since the instant total traffic load is greater than the link capacity for a very short period of time. If there is enough buffer to hold the packets, only a slight delay is incurred, otherwise packet loss will result. It is well understood that the overall traffic load, including the engineered traffic and the cross traffic, has a great influence on the performance of load balancing algorithms. If the overall load is too high the network will 15 1 5 2 4 6 8 1 Fig. 12 Aggregate traffic on bottleneck links (bandwidth requirement ¼ 8 Mbit/s) 16 12 8 4 2 4 6 8 1 Fig. 13 Total traffic on bottleneck links (bandwidth requirement ¼ 8 Mbit/s) experience delay or packet loss no matter which loadbalancing algorithm is used. This is also observed in Fig. 13. However, this issue is generally handled by admission control mechanisms which is beyond the scope of this paper. 4 Conclusions In this paper we have investigated the possibility of using parallel paths in two areas of traffic engineering in MPLS networks: constraint-based routing and load balancing. The parallel LSPs considered do not share links if they are between a same ingress egress node pair. As a result, if a flow between the edge nodes can be routed on the bundle of all the parallel paths, the network can accept more flows with QoS guarantees than when a flow can be routed on only one of the parallel paths, which is a common practice at the moment. The PPBS algorithm provides a method to extend the existing constraint-based routing algorithm to make use of the parallel LSPs when choosing a path with sufficient bandwidth. The improvement on flow-blocking probability by using PPBS is given quantitatively with respect to the average traffic load on the link, the hops along the path, and the possible number of parallel LSPs. Furthermore, a FBLB algorithm is proposed to distribute the traffic to the parallel paths adaptively based on the dynamic load condition of the paths which is fed-back by probing packets. Taking into account the signaling facility in the MPLS networks (e.g. RSVP, LDP), this feedback mechanism is easy to implement and provides exact network status information for load balancing. Simulation results show that FBLB is a stable and effective algorithm. 5 References 1 Awduche, D., Chiu, A., Elwalid, A., and Xiao, X.: Overview and principles of internet traffic engineering. IETF RFC 3372, 22 2 Elwalid, A., Jin. C., Low, S., and Widjaja, I.: MATE: MPLS adaptive traffic engineering. Proc. IEEE Infocom 21, pp. 13 139 3 Gao, D., Shu, Y., Liu, S., and Yang, O.W.: Delay-based adaptive load balancing in MPLS networks. Proc. IEEE ICC 22, pp. 1184 1188 4 Widjaja, I., and Elwalid, A.: Exploiting parallelism to boost data-path rate in high-speed IP/MPLS networking. Proc. IEEE Infocom 23 5 Mitra, D., and Wang, Q.: Stochastic traffic engineering, with applications to network revenue management. Proc. IEEE Infocom 23 6 Fortz B., and Thorup, M.: Internet traffic engineering by optimizing OSPF weights. Proc. IEEE Infocom, 2 7 Feldmann, A. et al.: Netscope traffic engineering for IP network, IEEE Netw., 21, 14, pp. 11 19 8 Awduche, D., Malcolm, J., Agogbua, J., O Dell, M., and McManus, J.: Requirements for traffic engineering over MPLS. IETF RFC272, 1999 9 Iwata, A., and Fujita, N.: A hierarchical multilayer QoS routing system with dynamic SLA management, IEEE J. Sel. Areas Commun., 2, 18, (12), pp. 263 2616 1 Apostolopoulos, G., Williams, D., Kamat, S., Guerin, R., Orda, A., and Przygienda, T.: QoS routing mechanisms and OSPF extensions. IETF RFC2676, 1999 11 Katz, D., et al. Traffic engineering extension to OSPF. IETF Internet Draft., draft-karz-yeung-osfp-traffic-.ext, 1999 12 Chen, S., and Nahrstedt, K.: An overview of quality of service routing of next-generation high-speed networks: problems and solutions, IEEE Netw., 1998, pp. 64 79 13 Guerin, R., and Orda, A.: QoS routinginnetworks withinaccurate information: theory and algorithms, IEEE Trans. Netw., 1999, 7, (3), pp. 35 364 14 Long, K., Zhang, Z., and Cheng, S.: Load balancing algorithms in MPLS traffic engineering. Proc. IEEE Workshop on High Performance Switching and Routing, Dallas, TX, USA, pp. 175 179 15 Salvadori, E., Battiti, R., Sabel, M.: A reactive scheme for traffic engineering in MPLS networks. Technical Report DIT-2-98, University of Trento, http://rtm.science.unitn.it/bbattiti/archive/ 98.pdf, 22 16 Papoulis, A.: Probability, random variables, and stochastic processes (McGraw-Hill International Editions, 1991, 3rd edn.) 12 IEE Proc.-Commun., Vol. 152, No. 1, February 25