Internet routing diversity for stub networks with a Map-and-Encap scheme

Internet routing diversity for stub networks with a Map-and-Encap scheme Xavier Misseri, Jean-Louis Rougier TELECOM ParisTech Paris France Email: {misseri,rougier}@telecom-paristech.fr Damien Saucez INRIA Sophia Antipolis France Email: damien.saucez@inria.fr Abstract Routing diversity has been identified as essential for network robustness and traffic engineering. The Internet possesses by its very nature a large path diversity. However this diversity cannot be fully exploited due to BGP limitations, which only keeps one single route for each available prefix. Despite some previous works in the area, no operational and non-disruptive architecture have been proposed yet to allow the networks to better exploit Internet path diversity. This paper proposes one step in this direction, focusing on the interconnection between an Autonomous System (AS) and its Internet Service Provider (ISP). We propose the use of a so-called Map-and-Encap scheme to bypass current BGP limitations in order to use arbitrary paths. With this scheme, an ISP may propose its rich path diversity (at least partially) to its customers, in order to perform advanced traffic engineering (e.g. fast recovery, load balancing...) based on richer and more flexible path selection policies (e.g., considering price, performance or stability of routes). To assess the potential benefits of the proposed architecture, we evaluate the potential route diversity that a Tier 1 may offer to its stub clients, based on different possible route selection policies (i.e. which routes are offered to its customers). We also analyze the overhead that is created at the control-plane (routing updates received by the mapping database) and that may impact the data-plane (path changes that may be caused by some route withdrawals/updates). Our evaluation shows that the increase in diversity has a controllable and acceptable overhead. It also gives some insights into efficient deployment strategies. I. INTRODUCTION Internet is composed of thousands of interconnected Autonomous Systems (AS) exchanging route information with the Border Gateway Protocol (BGP) [1]. Each AS is most likely connected to several neighbors and may receive several routes for each available prefix. This path diversity has been identified as an essential Traffic Engineering (TE) and robustness feature [2]. Unfortunately, end clients (i.e. stub networks) cannot benefit from this diversity as their providers only announce them one route per prefix. Extensive research efforts have in particular been conducted to load balance traffic on the different available paths. However, most ASes hardly use the inherent route diversity of the Internet. For instance, [3] shows that 86% of the load balancing is intra-domain load balancing, only benefiting from intra-as routing diversity (traffic is split and merged in the same domain). This limitation is the result of the BGP decision process that determines one single best route per prefix (potentially using arbitrary tie-break rules) to ensure scalability and to improve convergence. Bypassing this BGP limitation and thus offering several routes would increase the routing flexibility, make the Internet more robust to failures or improve performance [4] [5]. It would also be possible to offer a richer and more flexible set of high-level policies than currently available with BGP (e.g. based on stability or performance). Furthermore, a range of applications could benefit from multiple paths in conjunction with Multipath TCP [6]. MPTCP is a major extension to TCP that allows a TCP flow to be decomposed in several TCP sub-flows running in parallel. MPTCP can help to achieve better performance by leveraging the use of multiple paths in parallel [7]. Hence, increasing available path diversity can result in increasing data transfer performance with MPTCP. Of course, the number of available paths can be extremely large, so the global distribution of all the diversity may cause scalability issues. One may even argue that BGP announces a single best path for this very reason. However, we focus on the local relation between a stub AS and its provider(s). In this context, solutions to benefit from path diversity exist. To our knowledge, however, none of them propose a comprehensive and non-disruptive architecture to enable route diversity, on both control and data planes. Deflection has been identified as a way to enforce the path of packets [8] and benefit from Internet multipath routing. In this paper, we propose an architecture based on Map-and-Encap paradigm to take advantage of the path diversity currently truncated by the BGP decision process. On the one hand, encapsulation in Map-and-Encap enables traffic deflection. On the other hand, mappings are used in the control-plane to administer the deflection. We use the available ebgp paths to construct the mappings, where each BGP prefix is assigned a list of potential egress points (exit ASBRs). This diversity is exploited by tunneling packets toward the chosen exit points, by the mean of encapsulation. In this paper, we leverage the LISP protocol to construct our architecture (cf. section II-B) but this choice does not preclude the use of any other Mapand-Encap solution (e.g., MPLS based). The choice of LISP is motivated by the fact that it provides a well advanced Mapand-Encap protocol that supports incremental deployment as well as a flexible control-plane. We show how the proposed scheme can be implemented in an AS without any interaction with neighboring domains. However, its most promising applications arise when mappings

are provided by the ISP, thereby offering a certain amount of route diversity to its customers. The ISP may also help its customers to choose the best paths (e.g., potentially using path quality and stability). The ISP can benefit from this service for differentiation and to design added-value services (cf. Section IV). To assess the potential benefits of our approach, we propose an evaluation of the potential route diversity that a Tier 1 may offer, based on the different possible routing policies (i.e. which routes are offered to its customers). We also analyze the overhead that is created at the control-plane (i.e. routing updates received by the mapping system) and may impact the data-plane (i.e. path changes that may be caused by some route withdraws/updates). The paper is organized as follow. In Sec. II, we present the technological background. We then propose in Sec. III a Map-and-Encap architecture relying on the LISP protocol that allows a network to use the path diversity of its ISP. In Sec. IV we present two distinct use-cases that benefit from our architecture. Following this, we evaluate different functions to build the mappings in Sec. V and determine the diversity they offer and their dynamics. Our evaluation shows that the overhead caused by the diversity increase is acceptable and controllable. II. BACKGROUND A. Border Gateway Protocol (BGP) The BGP [1] propagation truncates diversity at every step [9] of the propagation in order to ensure the scalability of the Internet. Every BGP speaker computes the decision process to extract the best route and only propagate this route to the neighbors. The decision process consists of successive static rules based on comparisons of global or local path attributes, such as the LOCAL PREF, AS Path length or MED. The network administrators have some limited means to influence this selection process to reflect local policies (for instance by configuration of the local preferences). The decision process ends with a tie-break (selecting the route with lowest next hop IP address) in order to ensure that a single best route is selected. The choice of selecting a unique route is useful for transit networks as the propagation of the diversity to neighbors would face scalability issues. Nevertheless, in the case of a stub AS 1 connected to its ISP, there is no such scalability issue as the diversity will not be propagated further. Some BGP based techniques have been proposed to perform traffic engineering. For instance, [10] proposes Local- Preference tweaking mechanisms in order for multihomed AS to control their outbound traffic sent towards the different providers. In Sec. III, we go further and allow ISPs to offer more diversity to their customers. In order to use route diversity, we face two challenges: first several routes must be announced to the routers. Then, 1 Path diversity propagation may be extended to inter-isp route propagation. This case may cause some stability issues and will be covered in a future work. extended route selection must be allowed to choose the most appropriate paths. BGP add-path [11] is a useful extension, as it allows to announce several paths for the same prefix. However, it does not change the BGP decision process this extension was mostly intended to alleviate some BGP convergence issues. Multipath BGP extension [12], [13] slightly changes the BGP decision process in order to use simultaneously multiple routes. However, the routes must be similar (i.e., same Local-Preference, AS path length and MED) and traffic is balanced equally on these paths. We aim at offering a richer and more flexible range of policies in the choice of routes and load sharing strategies. Besides the already existing BGP extensions, Neighbor Specific BGP (NS-BGP) [4] proposes to adapt advertisements made to neighbors to enable a wider range of policies. From this point of view, our objectives are very close. While the authors of NS-BGP propose some modifications to BGP, we propose to bypass BGP within an AS and to use LISP instead. We also provide a complete architecture encompassing both data and control planes, while NS-BGP mainly focus on control-plane (in particular routing stability and robustness issues). B. Locator/ID Separation Protocol (LISP) LISP has been initially designed to make the Internet more scalable by separating core prefixes from end site prefixes [14], [15]. LISP has today a wider scope of applications and is considered in several domains such as Traffic Engineering or migration issues (see, e.g., [16]). LISP is a Map-and-Encap mechanism whose data-plane consists of an encapsulation protocol and the control-plane consists of a mapping system. Several mapping systems have been proposed and are considered at the IETF [14]. This encapsulating scheme can be used to forward a packet through a deflection point and thus enforcing its path. The mapping system stores and distributes (push or pull) the mappings to the LISP encapsulating routers. A mapping links IP prefixes with a list of IP addresses that can be used as deflection points. When a LISP router receives a packet, it encapsulates it with a LISP header whose outer destination corresponds to the mapping value (the deflection point). The encapsulated packet is then forwarded until it reaches the LISP router identified by the outer LISP header destination address. This router decapsulates the LISP packet and forwards the inner packet to the end host identified by the inner packet destination address. Each locator is assigned a priority and a weight. On the one hand, the priority determines the eligible mappings (i.e. the deflection points). On the other hand, the weight determines how the load must be balanced among the deflection points with the highest priority. III. ARCHITECTURE This paper proposes a solution to reveal path diversity in the control-plane and to ensure the use of this diversity in the data-plane.

Fig. 1. Intra-AS use of LISP encapsulation and mapping system A. Description of the architecture Our architecture offers the stub the flexible capability of controlling the network exit point for its own traffic. Depending on the use-case (cf. Section IV), the stub network can either choose its own exit point or choose the one of its provider. The architecture takes into account some design constraints: (C1) local route diversity management, (C2) path enforcement until the chosen exit point of the domain (independently from BGP best routes), (C3) simple, incrementally deployable and local adoption (e.g., not global deployment required). Therefore, we do not change BGP behavior. To this aim, we rely on a Map-and-Encap architecture. On the one hand, the map part of our solution ensures that every BGP prefix is mapped to a list of exit ASBRs. On the other hand, the encap part (i.e. tunneling) is used to enforce the path through the chosen exit ASBR. Such an exit choice may not be that of the BGP decision process. Fig. 1 summarizes our architecture. Several solutions could be found for building the mapping system (at the control-plane) or for the encapsulation (at the data-plane), separately. However, the LISP architecture provides a unified architecture which offers mapping and encapsulation, and that meets all our requirements. LISP offers a complete mapping system which allows to manage and push routing diversity to encapsulating routers (C1). It also provides an encapsulation scheme (C2) on top of IP (C3) for ease of deployment. Finally, LISP requires no changes to end-systems or to most routers, fulfilling the aim of an incrementally deployable protocol. LISP is still under development at the IETF but it is already a mature architecture and is available on the latest Cisco IOS releases. B. Control-plane At the control-plane level, our architecture relies on a centralized and extended mapping system named the Local Mapping Distributor (LMD). In addition to storing and distributing mappings, the LMD firstly generates them from the diverse Internet routes, filters them and ranks them. The mapping associates the BGP prefix to the list of addresses of the exit routers that can be used to send packet to that particular prefix. They are built from all the ebgp advertisements received by the AS. More precisely, each ASBR maintains an ibgp session with the LMD and propagates the routes it receives from its neighboring ASes. In order to benefit from all the potential path diversity and bypass current BGP policies, it would be very useful for ASBRs to activate BGP ADD-Path (see section II-A). As the chosen routes may be no longer congruent with BGP routing, some difficulties may arise at the level of the exit ASBR. When packets are decapsulated, a (BGP) routing decision may be taken at this point with the potential risks of: (i) forwarding the packet to another exit ASBR (the one providing the default BGP best route). (ii) sending the packet to another neighboring domain that peers with the selected exit ASBR. One possible solution is that an ASBR carries several IP addresses, one for each neighboring domain it is connected to. The next-hop field in every received ebgp update is replaced by the IP address of the local ASBR which is associated with the neighbor the update is coming from. All packets sent towards this address must then be routed towards the appropriate neighbor. For each received BGP route, the LMD remembers the BGP prefix and its mapped ASBR exit point (BGP next hop). With this information, the LMD determines for each BGP prefix the subset of ASBRs that can be used according to its local policies and ranks them. Each mapping contains a priority, calculated according to the local policy ranking, which indicates the ASBRs to use. The mappings are then distributed to the routers that may use the diversity. The way the subset of ASBRs is computed (i.e., the mapping function) and ranked is open. For example, one can choose to insert a fixed number of routes (exit ASBRs) as mapping entries, based on price whereas others can perform advanced selection process to insert only disjoint routes that have proven reliability or stability. The use of performance evaluation tools may be used to rank paths based on real measurements [17]. Simple mapping functions are evaluated in Sec. V for the sake of demonstration and give some insights into possible strategies. C. Traffic forwarding The components that encapsulate the packets can be deployed in any part of the network. For instance, the encapsulation can be performed by a load-balancer deployed at the exit of a stub network could it be a multi-homed network or a single-homed one (e.g. small office, home network...). Once the encapsulating router receives a packet, it analyses the destination IP address and deduces, thanks to the mapping records, the IP addresses of the ISP ASBRs to use to deflect the route. Traffic can be balanced between ASBRs of same priority. Nevertheless, the different packets of a stream will go through the same ASBR thanks to a hash algorithm described in the LISP specification. The ASBR that receives the packet decapsulates it and forwards it to the neighbor ASBR associated with the LISP destination IP address.

IV. POSSIBLE USE-CASES A. Making stubs benefit from their own diversity A straightforward application of the proposed architecture is for a multi-homed AS which benefits from a high degree of connectivity, i.e., connections with a large number of ISPs. The diversity offered by the different Internet connections can thus be exploited, as shown in Figure 1. The use of the Mapand-Encap architecture allows for a fine tuning of outbound traffic, without any coordination with its ISPs. As compared to existing techniques [18], the solution may actually offer a wider choice of route selection policies, such as differentiated route selection for different ingress routers or more flexible use of load balancing on multiple paths. B. Making stubs benefit from their ISP s diversity Fig. 2. Path diversity provided by the ISP and transmitted to the stub network The most novel and promising applications arise when the solution is implemented within an ISP, as depicted in Figure 2. In this scenario, the ISP offers its path diversity (at least partially, based on economic considerations for instance) to its customers. Once a specific binding agreement has been negotiated within an AS and its ISP, the packets are encapsulated inside the stub network and decapsulated at the exit point of the ISP. The ISP is also responsible for collecting ebgp routes and building the path mapping database. The ISP can manage the mappings for its customers, based on some pre-negotiated objectives (e.g., performance, stability). Some large stub ASes with complex TE requirements may, on the contrary, prefer to implement their own route selection policies by managing their own Local Mapping Distributor (as represented in Fig. 2). A specific protocol for inter-lmd communication must be designed in this case. It will not be discussed herein due to space constraints. We expect that some paths are more expensive than others, based on the ISP connection policies, hence the full diversity is not revealed to the customers. However, in the evaluation section (section V), we show that the diversity is of interest for large providers, even after application of several typical filtering policies. The client AS will thus benefit from this diversity for improving their robustness and traffic engineering (e.g. [2], [4], [5]). For the ISP, this service is easy to be deployed as it relies on existing building blocks (essentially LISP). We also show in the next section that the overhead (in terms of route changes or churn) is limited even while using the diversity of routes. We believe that our architecture can be a real differentiation factor for ISPs and also an opportunity to develop added-value services, such as the route selection and prioritization service or the LMD interconnection service. Note also that the LMD framework is flexible and can support a large set of services. For instance, the mapping system allows different levels of route diversity to be offered to different customers, based on their subscription. In particular, special peerings, shortcuts (e.g., with performance and/or protection guarantees) may be reserved for specific customers or traffic. A. Process V. EVALUATION Taking into account a large amount of routes is a tradeoff between path diversity and routing overhead. In particular, instability is one of the biggest issues of the Internet [19]. and churn (i.e. route instability) may increase with the added instability of each new route. We thus evaluate the evolution of churn and path diversity with the redistribution of the BGP updates into the LMD. We applied this evaluation on a Tier 1 (Level 3: AS number 3356). Even if Level 3 is a tier 1 network, it can use our architecture to improve path diversity for its stub clients. Here is a description on the evaluation process. More details can be found in [20]. We define the path diversity as the number of routes that can be used for a given prefix and the instability as the number of updates into the LMD entries that have a direct impact on the data-plane. In our case, where several routes are used, each BGP update that adds or removes a mapping entry is considered as instability as it may lead to packet loss or de-sequencing. We used one week of ebgp data provided by Route Views (from march 7th to march 13th, 2011) to simulate the BGP updates received by a Tier 1 AS. We choose to concentrate on the provider with the largest number of neighbors peering with Route Views [21], for the sake of precision. Once the maximum potential diversity is available thanks to the ebgp feeds, we filter it with several simple selection processes to only select routes that are considered equivalent. An AS can select its routes by taking into account a large set of criteria (i.e. price, delay, stability...). In this evaluation, we only used the BGP metrics such as Local Preference (LP) or the AS Path Length (ASPL) for the sake of illustration. These attributes are relevant as they are related to practical commercial relationships (Local Preference) and to the Internet topology (AS path length). The AS path length is provided directly from Route Views Data. The Local Preference are inferred using the Caida Database [22]. We did not use other BGP path attributes. Indeed, Route Views provides very few MED values and no IGP costs. IGP costs (and as a consequence MED values, which are usually based on IGP costs) are impossible to determine as the topologies are unknown from our dataset. Here are four possible cases that we considered in our evaluation:

LP: select routes that have the best local preference. Local preference is the metric reflecting, most of the time, the cost of the peering/transit link. Selecting the lowest local preference is equivalent to minimizing the cost of the inter-as forwarding. ASPL: select routes that have the shortest AS path length. AS path length is the technical distance to the destination. Selecting the path only through this criterion is meant to roughly search for better technical quality regardless of the price. LP+ASPL: select routes that have the best local preference and the shortest AS path length. This path selection allows us to select shortest paths while minimizing the transit cost. BGP: select routes that have the best local preference, the shortest AS path length and the lowest router ID. This path selection emulates the BGP decision process. We use it for comparison purposes only. B. Results 1) Major factors impacting route diversity: Our evaluation is based on 36 peering links and 46 transit links (clients). We can see in Figure 3 that the Tier 1 can benefit from a high route diversity. The amount of available paths is greatly correlated with the type of selection. Number of routes 0 5 10 15 20 25 30 35 ALL LP ASPL LP+ASPL Fig. 3. Route filters Tier 1 path diversity In the LP selection case, as almost all the prefixes are known and advertised at least by one client it leads the Tier 1 to only accept diversity coming from clients (i.e. the best local preference). The AS-number (ASN) we used to perform this study is actually known to have a customer cone 2 higher than 90% and therefore receives updates from clients for more than 90% of the prefixes. This filtering can make prefixes with a lot of potential diversity decrease to very little diversity because one or two clients propagate it. In our evaluation, a high proportion of the prefixes (220 000 prefixes or so) decreases to a diversity less than 5 routes. On the other hand, ASPL filtering has quite different consequences. It truncates the diversity 2 Customer cone: for an AS, the customer cone is the ratio between the prefixes belonging to its recursive clients and the number of overall prefixes in the world (cf. http://as-rank.caida.org/?mode0=as-introcustomer-cone). more than the LP filtering. The median values of LP and ASPL filtering are almost the same but a lot more prefixes have a diversity of 1 for ASPL filtering. This makes the mean value of the diversity decrease from LP to ASPL. ASPL and LP filtering are not totally correlated. Given a prefix that is advertised both by clients and peers, the updates announced by clients propose the shortest ASPL with a probability of 60%. Therefore the LP+ASPL filtering roughly cumulates the effect of both filtering and proposes a very low diversity level. First it takes into account routes coming from the best LP (i.e. the clients for most of the prefixes) an then takes only 60% of this diversity due to path length. Churn amount 0 20 40 60 80 100 Linear BGP churn extrapolation ALL LP LPASPL ASPL 5 10 15 20 25 Fig. 4. Route diversity Tier 1: Instability Vs Diversity 2) Tradeoff between path diversity and stability: having a great diversity is a consequence of a relaxation of the route selection. This relaxation may however have two contradictory consequences: it decreases the impact of some updates as the change of BGP metrics may not impact the result of the selection. For example, a change in the AS path that changes the BGP selection would not change the LP selection. contrary to the first point, a routing update that may not change the BGP best route could change the selection of the path diversity. For instance a change in the AS path length would not change the BGP best route if this route comes from the only neighbor with the best local preference whereas it could change the ASPL filtering. Figure 4 presents the instability in relation to its diversity. An extrapolation of the BGP churn is used for comparison. It contains the graph of churn according to the diversity for each path selection. A linear regression has also been calculated for each case for the sake of readability and comparison (with the linear extrapolation of the churn of BGP). The different cases of the Tier 1 are close to that of the BGP extrapolation, except for a few anomalies. These upper points are caused by less than a hundred flapping routes (less than 0.03%). As these flaps affect prefixes with high diversity, the Tier 1 may benefit a lot from the adoption of a flap damping algorithm to decrease this instability. 3) Outcomes: Diversity and instability are present and are not homogeneous among the prefixes. We highlighted in

Figure 3 that some prefixes suffer from a lack of diversity (one or two routes) whereas other prefixes have a lot more routes than necessary (more than 10 routes). This shows that naive mechanisms are not adapted and that one would benefit from more flexible decision policies which is precisely what we aim to offer with LMDs. For instance, a restrictive selection may be used to filter the prefixes with high diversity (potentially taking into account new parameters such as route stability or long term performance measures) whereas a relaxation of the policy may be of interest to increase the diversity for some prefixes with low diversity. Some prefixes may bring a lot of instability and stability analyses may be used to select really stable routes. As we have observed, these unstable prefixes, for the most, benefit from a high diversity. Hence this stability filtering will not make them lose their whole diversity (unless the instability takes place in the destination network). Consequently, an advanced selection process is of interest to wisely select routes. It is very important to highlight that, contrary to BGP, the instability provided by our use of diversity is not propagated to the neighbors. Indeed, the diversity is used locally and the instability impacts only the local choices as we only propagate the BGP best route to neighbors. VI. CONCLUSION This paper presents an architecture aimed at better exploiting the path diversity that is inherently available in the Internet. The proposal is based on the LISP Map-and-Encap mechanisms in order to overcome current BGP limitations. It allows an ISP to offer its path diversity (at least partially) to its customers for the sake of traffic engineering and robustness purposes. We describe the architecture and potential applications. The solution enables the definition of a wide range of possible route selection policies, based for instance on economical, performance or stability criteria. In order to assess the potential benefits of the proposal, we conducted an evaluation based on simple route selection policies (i.e. which routes are offered by the ISP to its customers). First the route diversity that a Tier 1 may offer is studied, depending on the different policies. We also focused on the routing stability criterion by analyzing the quantity of routing updates that cause changes in data forwarding. Our evaluation shows, at least for the studied network, that the increase in diversity comes with a controllable and acceptable overhead. Furthermore, it was shown that taking into account the stability of routes is promising as it does not significantly alter path diversity while being essential for the scalability and robustness of our proposal. For future work, we plan to provide a more detailed and consolidated description of the architecture. We are also currently working on a more comprehensive evaluation of the architecture, in particular evaluating advanced path selection and considering a wider set of ISPs. ACKNOWLEDGEMENTS We would like to acknowledge Pr. Olivier Bonaventure and Virginie Van den Schrieck for their contributions, useful comments and fruitful discussions. This work has been partially supported by the European funded ECODE and ETICS projects, and the ANR project SCATTER. REFERENCES [1] S. Hares, Y. Rekhter, and T. Li, A Border Gateway Protocol 4 (BGP-4). [Online]. Available: http://tools.ietf.org/html/rfc4271 [2] M. Yannuzzi and X. Masip-bruin, Open issues in interdomain routing: a survey, pp. 49 56, 2005. [3] B. Augustin, T. Friedman, and R. Teixeira, Measuring Multipath Routing in the Internet, IEEE/ACM Transactions on Networking, vol. 19, no. 3, pp. 830 840, 2011. [4] Wang Yi, M. Schapira, and J. Rexford, Neighbor-Specific BGP: More Flexible Routing Policies While Improving Global Stability, in Proceedings of the eleventh international joint conference on Measurement and modeling of computer systems, sigmetrics ed. Seattle, WA, USA: ACM, 2009, pp. 217-228. [5] V. Van den Schrieck, P. Francois, and O. Bonaventure, BGP Add-Paths: The Scaling/Performance Tradeoffs, IEEE Journal on Selected Areas in Communications, vol. 28, no. 8, pp. 1299 1307, Oct. 2010. [6] M. Handley, C. Raiciu, A. Ford, S. Barre, and J. Iyengar, Architectural Guidelines for Multipath TCP Development. [Online]. Available: http://tools.ietf.org/html/rfc6182 [7] C. Raiciu, S. Barre, C. Pluntke, A. Greenhalgh, D. Wischik, and M. Handley, Improving datacenter performance and robustness with multipath TCP, in Topology, 2011, pp. 266-277. [8] H. Jiayue and J. Rexford, Toward internet-wide multipath routing, Network, IEEE, vol. 22, no. 2, pp. 16 21, 2008. [9] S. Uhlig and S. Tandel, Quantifying the bgp routes diversity inside a tier-1 network, Lecture Notes in Computer Science, vol. 3976, p. 1002, 2006. [10] B. Quoitin, C. Pelsser, L. Swinnen, O. Bonaventure, and S. Uhlig, Interdomain traffic engineering with BGP, IEEE Communications Magazine, vol. 41, no. 5, pp. 122 128, 2003. [11] J. Scudder, A. Retana, D. Walton, and E. Chen, Advertisement of Multiple Paths in BGP, 2009. [Online]. Available: http://tools.ietf.org/ html/draft-walton-bgp-add-paths-06 [12] bgp best path selection algorithm [ip routing]. [Online]. Available: http://www.cisco.com/en/us/tech/tk365/technologies\ tech\ note09186a0080094431.shtml [13] Juniper, Configure BGP to Select Multiple BGP Paths. [Online]. Available: http://www.juniper.net/techpubs/software/junos/ junos53/swconfig53-ipv6/html/ipv6-bgp-config29.html [14] D. Lewis, V. Fuller, D. Farinacci, and D. Meyer, Locator/ID Separation Protocol (LISP). [Online]. Available: http://tools.ietf.org/ html/draft-ietf-lisp-19 [15] B. Quoitin, L. Iannone, C. D. Launois, and O. Bonaventure, Evaluating the Benefits of the Locator/Identifier Separation, in Proceedings of 2nd ACM/IEEE international workshop on Mobility in the evolving internet architecture. ACM, 2007, pp. 5:1-5:6. [16] D. Lee, LISP Deployment at Facebook, 2010. [Online]. Available: http://www.nanog.org/meetings/nanog50/presentations/tuesday/ NANOG50.Talk9.lee\ nanog50\ atlanta\ oct2010\ 007\ publish.pdf [17] D. Saucez, B. Donnet, L. Iannone, O. Bonaventure, and U. C. D. Louvain, Interdomain Traffic Engineering in a Locator/Identifier Separation Context, 2008 IEEE Internet Network Management Workshop INM, 2008. [18] S. Uhlig and O. Bonaventure, Designing BGP-based outbound traffic engineering techniques for stub ASes, ACM SIGCOMM Computer Communication Review, vol. 34, no. 5, pp. 89 106, Oct. 2004. [19] A. Elmokashfi, A. Kvalbein, and C. Dovrolis, BGP Churn Evolution: a Perspective from the Core, in 2010 Proceedings IEEE INFOCOM. IEEE, Mar. 2010, pp. 1 9. [20] X. Misseri, J.-L. Rougier, and D. Saucez, Technical report - Internet routing diversity for stub networks with a Map-and-Encap scheme, Tech. Rep., 2011. [Online]. Available: http://perso.telecom-paristech.fr/ misseri/files/10-diversity-stubs-technical-report.pdf [21] University of Oregon Route Views Project. [Online]. Available: http://www.routeviews.org [22] CAIDA AS rank. [Online]. Available: http://as-rank.caida.org/