Traffic protection in MPLS networks using an off-line flow optimization model

Traffic protection in MPLS networks using an off-line flow optimization model A.E. Krzesinski and K.E. Müller Department of Computer Science University of Stellenbosch, 76 Stellenbosch, South Africa Phone: +27 21 884232 Fax: +27 21 884416 Email:{aek1,kmuller}@cs.sun.ac.za P.G. Taylor Department of Mathematics and Statistics University of Melbourne, Melbourne, Australia Email: p.taylor@ms.unimelb.edu.au Abstract MPLS-based recovery is intended to effect rapid and complete restoration of traffic affected by a fault in an MPLS network. We present a model of protection switching in MPLS networks. A variant of the flow deviation method is used to find and capacitate a set of optimal label switched paths. The traffic is routed over a set of working LSPs. Protection switching is implemented by reserving a set of pre-established recovery LSPs. A simulation model is used to evaluate the MPLS recovery cycle in terms of the time needed to restore the traffic after a unidirectional link failure. The model is applied to evaluate the effectiveness of protection switching in a 2-node network. Keywords Label switched paths, multi-protocol label switching, traffic engineered routes, traffic recovery, protection switching. I. INTRODUCTION The Internet is becoming the ideal platform to support all forms of modern communications including voice, data and multimedia transmissions. However, the standard IP routing protocols were developed on the basis of a connectionless model where routing decisions are based on simple metrics such as delay or hop count which leads to the selection of shortest path routes. Despite its ability to scale to very large networks, this approach provides support for only rudimentary Quality of Service (QoS) capabilities which cannot be used to provide scalable service level agreements for bandwidth intensive applications in modern networks. Multi-protocol label switching (MPLS) [1] extends the IP destination-based routing protocols to provide new and scalable routing capabilities. MPLS routing/switching is achieved by forwarding IP packets along virtual connections called label switched paths (LSPs). LSPs are set up by a label distribution protocol which uses the information contained in layer 3 routing tables. The LSPs form a logical network that is layered on top of the physical network to provide connection oriented processing above the connectionless This work is supported by grants from the Australian Research Council, the South African National Research Foundation, Siemens Telecommunications and Telkom SA Limited. IP network. In a previous paper [2] we developed a method for finding an optimal set of LSPs. We formulated the problem of finding an optimal set of LSPs and optimally allocating bandwidths to these LSPs as a constrained non-linear programming problem (NLP) which minimizes an objective function that affords an appropriate representation of the quality of the network. We adapted the flow deviation method [3], [4], [5] to solve the NLP. In this paper we investigate some aspects of MPLSbased recovery [6] which is intended to effect rapid and complete restoration of traffic affected by a fault in an MPLS network. Two recovery models have been proposed for MPLS networks: IP re-routing which establishes recovery paths on demand, and protection switching which works with pre-established recovery paths. This paper presents a model of protection switching in MPLS networks. A more detailed study of MPLS recovery including analytic and simulation models can be found in [7]. The NLP solver is used to design and capacitate an optimal set of LSPs which defines the working (active, primary) LSPs: the traffic is routed over the working LSPs. We next present a method for finding one or more alternate (recovery, back-up, protection) paths for each working path: the recovery path is used when its working counterpart fails. Global repair is implemented by reserving a set of LSPs for use as pre-established recovery paths. The rest of the paper is organized as follows. Sect. II presents an overview of MPLS recovery. The objectives of MPLS recovery are presented and various recovery models are compared. Sect. III presents a model of an MPLS network, definitions of feasible and optimal LSP bandwidth assignments and a description of the LSP design problem whose solution yields an optimal set of LSPs and optimal LSP bandwidth assignments. Sect. IV presents an efficient method for computing an optimal set of recovery paths. Sect. V presents a simulation model which is used to evaluate

the dynamics of protection recovery in a models of 2- and 5-node networks. Our conclusions are presented in Sect. VII. II. AN OVERVIEW OF MPLS BASED RECOVERY A. The Objectives of MPLS Based Recovery MPLS based recovery mechanisms and techniques should: (1) be subject to the traffic engineering (TE) goal of optimal use of resources, (2) facilitate restoration times that are sufficiently fast for the end-user applications, (3) maximize network reliability and availability and minimize the number of single points of failure in the MPLS protected domain, (4) enhance the reliability of the protected traffic while minimally or predictably degrading the traffic carried by the diverted resources, (5) protect the traffic at various granularities 1 (6) be applicable to an entire end-to-end path or to segments of an end-to-end path, (7) take into consideration the recovery actions of the lower layers and should not trigger lower layer protection switching, (8) minimize the loss of data and packet reordering during recovery operations, (9) minimize the state overhead incurred for each recovery path maintained and, (1) preserve the constraints on traffic after switchover, if desired, so that the recovery path meets the resource requirements of the working path and achieves the same performance characteristics as the working path. Some of the above goals are in conflict with each other and real deployment will involve compromises based on a variety of factors such as cost, end-user application requirements, network efficiency, and revenue considerations. B. Recovery Models for MPLS Networks Two recovery models have been proposed for MPLS networks: IP re-routing which establishes recovery paths on demand, and protection switching which works with pre-established recovery paths. IP re-routing is robust and frugal since no resources are pre-committed but is inherently slower than protection switching which is intended to offer high reliability to premium services where fault recovery takes place at the 1 ms time scale. This paper does not address IP re-routing. Protection switching works with pre-established recovery paths. A recovery path may support the same traffic contract as the working path, or it may not. An equivalent recovery path can replace a working path without degrading service. A limited recovery path lacks the resources (or the resource reservations) to replace the working path without degrading service. 1 Three levels of traffic granularity are proposed: part of the recovery traffic can be allocated to an individual path, all of the recovery traffic can be allocated to an individual path, or all of the recovery traffic can be allocated to a group of paths. There are two options for the initiation of resource allocation: pre-reserved allocation which only applies to protection switching and reserved-on-demand allocation which may apply either to IP re-routing or to protection switching. A pre-reserved recovery path reserves required resources on all hops along its route during its establishment before any failure has occurred. A reserved-on-demand recovery path reserves required resources after a failure on the working paths has been detected and before the traffic on the working path is switched over to the recovery path(s). C. Comparison Criteria Several criteria have been suggested for comparing various MPLS-based recovery schemes. The recovery time is the time between a failure of a node or link and the time before a recovery path is installed and the traffic starts flowing on it. The full restoration time is the time required for traffic to be routed onto links which are capable of (or have been engineered to) handle traffic in recovery scenarios. The setup vulnerability time is the time that a working path or a set of working paths is left unprotected during such tasks as recovery path computation. Recovery schemes may require differing amounts of back-up capacity in the event of a fault. This capacity will depend on the traffic characteristics of the network. However, it may also depend on the protection plan selection algorithms as well as the signaling and re-routing methods. Recovery schemes may introduce additive latency to traffic. For example, a recovery path may take many more hops than the working path. This may be dependent on the recovery path selection algorithms. The quality of protection: recovery schemes can offer a spectrum of packet survivability options which may range from relative to absolute. Relative survivability may mean that the packet is on an equal footing with other traffic for the surviving network resources. Absolute survivability may mean that the survivability of the protected traffic has explicit guarantees. Recovery schemes may introduce re-ordering of packets. Also the action of putting traffic back on preferred paths might cause packet reordering. As the number of recovery paths in a protection plan grows, the state overhead required to maintain them also grows. Recovery schemes may require differing numbers of paths to maintain certain levels of coverage. The state overhead may depend on the recovery scheme. In many cases the state overhead will be in proportion to the number of recovery paths. Recovery schemes may introduce a certain amount of packet loss during switchover to a recovery path. In the case of link or node failure a certain packet loss is inevitable. Recovery schemes may offer various types of failure coverage. A recovery scheme may account for only certain types of faults such as link faults or both node and link faults. The recovery scheme may also respond

to service degradation. For example, a scheme may require more recovery paths to take node faults into account. A recovery scheme may be able to handle concurrent faults : depending on the layout of the recovery paths in the protection plan, multiple-fault scenarios may be able to be restored. A recovery scheme can offer multiple recovery paths : for a given fault, there may be one or more recovery paths. A recovery scheme may offer a varying degree of coverage : depending on the recovery scheme and its implementation, a certain percentage of link and node faults may be covered. Finally, a recovery scheme has a reaction time: the number of protected paths may effect how fast the total set of paths affected by a fault can be recovered. III. THE MODEL Consider a communications network with N nodes and L links. The nodes represent the routers in the MPLS-capable core of a network. Some nodes are connected by a link. The links are directed: each link has a starting node and an ending node. Let b i denote the capacity (bandwidth) of link i. Let d (m,n) denote the offered traffic load that wants to enter the MPLS network at node m and exit at node n. We consider only a single class of service. A. Feasibility and Optimality A path P is a sequence of links L 1, L 2,..., L HP where H P 1 is the hop count of the path P. In our terminology a route and a path and an LSP (label switched path) are synonymous. No path traverses the same link or the same node more than once. The flow deviation algorithm ensures that no paths contain cycles. Let P denote the set of all such non-cycling paths. Let P (i) denote the set of paths that utilize link i. Let P (m,n) denote the set of paths from node m to node n with m n. Each path P will be assigned a bandwidth B P. The goal is to select these bandwidths in an optimal way. Let B = (B P ) P P denote a set of bandwidths. B is said to be feasible if the following two constraints hold: 1. For each pair of nodes (m, n): P P (m,n) B P = d (m,n) so that all of the offered traffic is carried. 2. For each link i: P P B (i) P b i so that no link has an offered load greater than its capacity. We next choose a definition of optimality. Let f i = P P B (i) P denote the flow on link i. Let f i /b i denote the utilization of link i and let s i = b i f i denote the slack on link i. Let F i (f i ) denote a penalty function for link i when the link carries a flow f i. The LSP design problem is specified in terms of the following constrained nonlinear optimization problem: Find a set of feasible bandwidths B opt that minimizes the objective function F (B) = i F i (f i ) (1) subject to the two constraints above where the sum in equation (1) is over links i with b i >. B opt is said to provide an optimal solution to Eq. (1). Note that the optimal link flows f i are almost certainly unique although the optimal bandwidths B are usually not [2]. B. The Penalty Function Previous studies of the flow deviation algorithm [3], [4], [5] modelled each link as an M/M/1 queue with F i (x) = m x b i x (2) where m is the average length of a packet and b i is the bandwidth of link i. The link penalty function (2) is the product of the link flow x and the average link delay (waiting and service both included, but propagation delay excluded). However, in the modern Internet with TCP, and RED and all its variations, it is possible to have very highly utilized links (utilization practically one) and still low delay and low loss in the buffer: all delay is moved to the edge of the network. Equation (2) is probably no longer a suitable penalty function. Given these concerns, we use a link penalty function [2] with properties which make it suitable for use in an objective function whose minimization will yield routes and bandwidths that correspond closely to the optimal operation of a modern internet. Our choice of penalty function is ( ) ν σi F i (x) = c i x + ησ i (3) b i x where link i has a bandwidth b i, a weight factor σ i > with η >, ν > 1 and c i = τ i ην (σ i /b i ) ν+1 where τ i denotes the propagation delay on link i. We choose η positive but small so that if a feasible solution exists for which all flows f i are small and all link utilizations f i /b i are low in which case the system is said to be uniformly lightly loaded then the penalty function (3) will yield routes that are in agreement with OSPF routing where the propagation delays are the OSPF metrics of the links. If the system is not uniformly lightly loaded then the penalty function (3) enforces a distance from the barrier b i. We showed [2] that a suitable choice of parameter values enables the penalty function (3) to perform a lexicographic maximization of all weighted slacks: first it maximizes the smallest weighted slack, then the next smallest, and so on. C. The Flow Deviation Algorithm The operation of the flow deviation algorithm [3], [4], [5] is simple. The algorithm executes in a loop where each iteration of the loop implements one step

of the algorithm. During each step the algorithm computes the current set of shortest (least cost) paths from all sources to all destinations. An optimal amount of flow is diverted from the current set of LSPs to the shortest paths. Those shortest paths that are not already in the LSP set are added to the LSP set, the link costs are updated (the link costs have changed because the link flows have changed) and the next step of the algorithm is executed. The loop continues until flow re-distribution achieves no further reduction in the objective function. IV. A MODEL OF PROTECTION RECOVERY Our protection model is subject to the TE goal of optimal use of resources (see goal (1) of section II) in the sense that when the link loads are low, the flow deviation solution yields the OSPF routes, but when the link loads are high, the flow deviation solution selects relatively short routes so that the slack on each link is maximized. This means that goal (3) is also satisfied, since the network reliability and availability are maximized by not overloading any one specific link. Our protection recovery model aims to facilitate restoration times that are sufficiently fast (2) and may be applicable for an entire end-to-end path or for segments of an end-to-end path (6). The protection model does not trigger lower layer protection switching (7) and aims to minimize the loss of data (8). Our protection recovery model works with pre-established reserved-on-demand recovery paths. Some of these paths are limited recovery paths only when the network is not well designed and capacitated, otherwise all of the recovery paths are equivalent recovery paths. If the network is sufficiently capacitated then the recovery paths can meet the resource requirements of, and achieve the same performance characteristics as the working paths (1). Our recovery model also makes use of split path protection where multiple recovery paths are allowed to carry the traffic of a working counterpart. This is useful and sometimes necessary when no single recovery path can be found that can carry the entire traffic of the working path in case of a fault although this may be detrimental to the performance of the restored traffic. A. Computing a Set of Recovery Paths The following method was used to compute a set of traffic engineered recovery paths. Let F denote a set of network failure scenarios. The flow deviation method was first used to find a set A of optimal LSPs for the network in the absence of failures. For each failure scenario i F the flow deviation method was then used to find a set A i of optimal LSPs for the network with failure i. The set B i = A i \ A defines the set of optimal recovery paths for failure i, and the set B = i F B i defines the optimal set of recovery paths for all the failure scenarios. V. THE SIMULATION MODEL This section presents a simulation model which recognizes both connection-level and packet-level events. Unlike for example the NS simulator [1] which presents a faithful simulation of the network protocols, our simulator accounts for only those protocol features which influence MPLS based recovery. A. Connection-Level Arrivals and Departures Let L denote the set of O-D pairs in the network. Let P i denote pathset i L which is the set of LSPs that connect the O-D pair i. Let P = i L P i denote the set of LSPs in the network. Connections arrive to pathset i L individually at the instants of a Poisson stream with rate ν i. The connection holding times are exponentially distributed with parameter 1/τ i. This is not essential the simulation can be modified to relax this assumption. There are two connection-level network state descriptors: n = (n 1,..., n L ) where n i denotes the number of connections on pathset i and m = (m 1,..., m P ) where m i denotes the number of connections on LSP i. Clearly m = n. Since no confusion can arise we use the notation n i and m i in the place of n i (t) and m i (t) where t is the current simulation time. Connection arrivals and departures are generated as follows. Let t denote the current simulation time. Generate an exponentially distributed random time T conn with parameter Γ = i L (ν i + n i τ i ). A single connection-level event with due time T = t + T conn is inserted into the event scheduler which is used at the packet level (see below). When the simulation gets to time T a random variable U is chosen uniformly distributed on [, 1]. If there exists an I such that I 1 ν i < ΓU < I then a connection arrives to pathset I. The arrival is modelled as follows. Set n I := n I + 1 so that the number of connections on pathset I is increased by one. Any type of connection admission control can be employed at this stage. Let J P I be the LSP in pathset I with the least delay. Then m J := m J + 1 so that the number of connections on LSP J is increased by one. Clearly j P I m j = n I. Alternatively, if there exists an I such that i L I 1 ν i + n i τ i < ΓU < i L ν i ν i + I n i τ i

then a connection completes on pathset I. The completion is modelled as follows. Set n I := n I 1 so that the number of connections on pathset I is decreased by one. Assume that all connections on pathset I have the same average holding time so that, with an abuse of notation, τ i = τ I for all i P I. Let Ω = i P I m i τ i. Choose a random variable U uniformly distributed on [, 1]. If there exists a J such that J 1 J m i τ i < ΩU < m i τ i then a connection completes on LSP J. Set m J := m J 1 so that the number of connections on LSP J is decreased by one. Clearly j P I m j = n I. B. Packet-Level Arrivals Packets for any connection on LSP i arrive at rate λ i = L/τ i where L is the average number of packets that each connection offers to the network. Consider a time instant t when a packet arrival has just occurred, or when a connection arrival or departure has occurred. The next packet arrival is generated as follows. Generate an exponentially distributed random time T pack with parameter Λ = i P m i λ i. When the simulation gets to time t a random variable U is chosen uniformly distributed on [, 1]. If there exists an I such that I 1 I m i λ i < ΛU < m i λ i then a packet is generated on LSP I and assigned a label j. Let J denote the set of packet labels in the network at time t. The arrival time a j for packet j is a j = t + T pack and packet j is placed in the queue at the first link k j of LSP I. If packet j is the only packet in this queue then a completion time for packet j is generated as follows. Generate an exponentially distributed random time T serv with parameter b kj where b kj is the bandwidth (packets/sec) of link k j. Packet j is scheduled to complete at time a j + T serv and the packet is inserted into the scheduler. C. Progression of Packets At any time t there is a schedule of times with at most L + 1 entries: one entry for the next connection event and one entry for each packet completion event. The schedule is organized in chronological order. The simulator also knows (at least) the order of packets in the queues at each of the links, the delay on each of the LSPs, the initial arrival time a j and the current link k j of every packet j J. Suppose the first event in the schedule is a packet completion event with due time T prog. Thus at time T prog say packet j at link k j completes: the departure time for the next packet (if any) in the queue k j is generated and this packet is inserted into the scheduler. If the packet j was at the final link of its route, the endto-end delay T prog a j is recorded together with other relevant statistics and the label j is released. If packet j was not at the final link of its route, the packet selects the next link of its route (either deterministically or according to some routing probabilities) and packet j is inserted at the end of the queue at the next link. If packet j is the only packet at this queue, a new departure time is generated and packet j is inserted into the scheduler. D. Link Failure The failure of a uni-directional link i is modelled by disabling all LSPs that use the failed link and by activating the recovery LSPs. Currently the simulator does not represent the delay in communicating the fault indication signal to the path-switch and path-merge LSRs. Packets which complete service on the failed link, and packets which are forwarded to the failed link are destroyed and all packets queued for service at the failed link are destroyed. For each j L let P j (i) = {P P j i P } (4) denote set of paths in pathset j which use the failed link i. The number of connections on pathset j is decreased n j := n j m P. P P j (i) The number of connections on each failed path is set to zero so that m P = for all paths P P j (i) and all j L. VI. EXPERIMENTAL RESULTS This section presents the results of MPLS recovery in a model of a 2-node network. The results were produced using an extended version [7] of the simulator presented in Sect. V. The computation was performed on a 1.8GHz AMD6 processor. The set of recovery paths was designed to provide protection against all uni-directional single link failures. We examine single link failure scenarios which are the most probable failure events since (see [8] and the references therein) a complete fibre cut is the most common and frequently reported failure event. The 2-node network [9] has 12 bi-directional links and carries 1 traffic class. The links are capacitated with 5,289,78 units of bandwidth. A total of 1,344,75 units of flow are offered to the 38 origindestination pairs. The flow deviation method finds an

2e+6 network failed pathset affected pathsets 1 3 failed LSP failed pathset network 3 1.5e+6 8 packet throughput 1e+6 6 4 delay 2 1 2 1 5 2 1e+7 2e+7 3e+7 packets sent 1e+7 2e+7 3e+7 packets sent Fig. 1. The 2-node network: (a) packet throughput (b) packet delay. optimal LSP set containing 54 paths. The worst-case uni-directional single link failure will disable 4 LSPs which carry 8% of the network flow: 144 recovery paths are deployed. The simulator requires some 1 minutes to process 1 million events which include 32 million packet transfers. An average of 45,75 connections are simultaneously active and 47,62 packets are on average in the network, either in transmission or queued in link queues awaiting transmission. The link failure caused 3,643 connections to fail and 6,496 packets were lost. From equation (4) pathset j is affected by the failure of link i if P j (i) in which case pathset j contains an LSP that used the failed link i. Fig. 1(a) shows the packet throughput in packets per second for the entire network (top curve, left hand scale), for the failed pathset (middle curve, right hand scale) and for the affected pathsets (bottom curve, left hand scale). The throughput is a moving average computed over the most recent 4 connection completions. A bezier curve is fitted to the throughput of the failed pathset (middle curve) to smooth the fluctuations. The impact of the link failure on the throughputs can clearly be seen. The recovery LSPs quickly restore the throughputs to their pre-failure values. The failed link connects an O-D pair that prior to the link failure was served by a single LSP a direct path. After the link failed, the O-D pair is served by 4 recovery LSPs each of which is 4 links long. Fig. 1(b) shows the average end-to-end packet delay on the failed LSP (top curve, left hand scale), the average end-to-end packet delay of the failed pathset (middle curve, right hand scale) and the average end-to-end packet delay for the entire network (bottom curve, left hand scale). The moving averages are computed over the most recent 4 completing packets. The link failure has a negligible impact on the network average delay. However the failure gives rise to a 25-fold increase in the average packet delay on the four recovery LSPs. In this case the recovery paths do not have the same delay performance characteristics as the working path. VII. CONCLUSION This paper considers the problem of protection switching in MPLS networks. A variant of the flow deviation method was used to find and capacitate a set of optimal working LSPs. Global repair is implemented by reserving a set of pre-established recovery LSPs. A simulation model was used to evaluate the dynamics of the MPLS recovery cycle in terms of the time needed to effect the restoration of the traffic affected by a fault. The model was applied to evaluate the effectiveness of protection switching in a 2-node network. REFERENCES [1] Consult http://www.ietf.org for MPLS RFC and Draft Documents such as RFC 331 (MPLS Architecture), RFC 336 (LDP Specifications), RFC 272 (Requirements for Traffic Engineering over MPLS). [2] J.E. Burns, J.M. de Kock, A.E. Krzesinski and T.J. Ott, Path selection and bandwidth allocation in MPLS networks: a nonlinear programming approach, ITCom-21 Conference on Internet Performance and Control of Network Systems, pp. 15 26, 21. [3] D. Bertsekas and R. Gallager, Data Networks Second Edition, Prentice-Hall International Inc., 1992. [4] A. Kershenbaum, Telecommunication Design Algorithms, McGraw-Hill, 1993. [5] L. Kleinrock, Queueing System Vol. 2: Computer Applications, John Wiley & Sons, New York, 1976. [6] V. Sharma et al., Framework for MPLS-based Recovery, Internet Draft, draft-ietf-mpls-recovery-frmwrk-3.txt, Work in Progress, July 21. Obtainable from http://www.ietf.org/1idabstracts.txt. [7] A.E. Krzesinski and K.E. Müller, Traffic protection in MPLS networks using an off-line flow optimization model, to appear in ITCom-22 Conference on Internet Performance and Control of Network Systems, Boston, USA, July 22. [8] K. Murakami and H.S. Kim, Virtual Path Routing for Survivable ATM Networks, IEEE/ACM Transactions on Networking, Vol.4, No.1, pp. 22 39, February 1996. [9] C. Villamizar, http://brookfield.ans.net/omp/random-testcases.html. [1] S. McCanne and S. Floyd, The LBNL network simulator, http://www-mash.cs.berkeley.ed/ns, (17) Lawrence Berkeley Laboratory.