Optimizing Ingress Routing with LISP across Multiple VXLAN/EVPN Sites

White Paper Optimizing Ingress Routing with LISP across Multiple VXLAN/EVPN Sites 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 1 of 18

Contents What You Will Learn... 3 LISP Overview... 3 Why Use LISP in the Data Center... 4 Mobility across Multiple Data Centers with Ingress Route Optimization... 4 VXLAN Overview... 5 VXLAN EVPN Overview... 5 VXLAN EVPN Integration with LISP... 6 Host Move Detection in a VXLAN EVPN Fabric... 6 Host Mobility across VXLAN EVPN Fabrics... 8 Summary... 11 Functional Roles and Configuration... 11 Hardware and Software Details... 12 Border Spine Configuration in Data Center 1 (BGP AS 65001)... 12 Border Leaf Configuration in Data Center 2 (BGP AS 65002)... 14 LISP Map-System Database Configuration... 14 Branch Site Configuration... 15 Verification... 15 Conclusion... 16 Appendix: Other Benefits of LISP in the Data Center... 16 IPv6 Enablement... 16 Multitenancy and Large-Scale VPNs... 17 Efficient Multihoming at the WAN Edge... 17 For More Information... 18 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 2 of 18

What You Will Learn Locator/Identity Separation Protocol (LISP) is a data center interconnect (DCI) solution that provides a simplified way of handling multitenant connectivity in the fabric and mobility semantics across fabrics. This document describes how to integrate Virtual Extensible LAN (VXLAN) Ethernet Virtual Private Network (EVPN) fabric with LISP, using a configuration example. LISP, when integrated with VXLAN EVPN fabric, can help solve route optimization problems that result from workload mobility across data center fabrics. This document assumes that you have a basic knowledge of VXLAN, EVPN, and LISP technologies. LISP Overview Locator/Identity Separation Protocol is a new routing architecture that creates a model by separating the device identity, known as the endpoint identifier (EID), and the routing locator (RLOC). The EIDs are assigned to the end hosts, and the RLOCs are assigned to the devices (primarily routers) that make up the global routing system. This separation adds flexibility to the network in a single protocol, helping enable mobility, scalability, and security. LISP uses a dynamic tunneling approach rather than preconfigured tunnel endpoints. It s designed to work in a multihomed environment and supports communication between LISP and non-lisp sites for internetworking. The main benefits of LISP include simplified WAN edge multihoming with ingress traffic engineering capabilities, multitenancy over the Internet, simplified IPv6 transition support, and IP mobility for geographically dispersed data centers. In the traditional approach, an IPv4/IPv6 address represents both a device s identity and location, as shown in Figure 1. Figure 1. Traditional IP Address In LISP, an IPv4/IPv6 address represents a device s identity only, and the RLOC identifies the location, as shown in Figure 2. 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 3 of 18

Figure 2. IP Address in LISP Why Use LISP in the Data Center The Biggest use case of LISP in a data center environment is ingress route optimization due to workload mobility. Mobility across Multiple Data Centers with Ingress Route Optimization In today s enterprise data center deployments, server virtualization and high availability requires workloads to move from one data center to another across geographically dispersed locations. This mobility brings the challenge of route optimization when virtual servers move: how best to route traffic to the virtual server s current location? It also brings the challenge of maintaining the server s identity (IP address) when the server moves: how to retain the IP address across moves so that clients can continue to send traffic to it regardless of the server s current location. With LISP, when virtual servers move, the IP address and EIDs don t change; and only the RLOC identifiers change. As endpoints move, traffic is routed to these endpoints in their correct location following the best possible path (Figure 3). Figure 3. LISP IP Address Mobility between Data Centers There are other use cases of LISP in the data center, which are discussed in the Appendix section. 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 4 of 18

VXLAN Overview Virtual Extensible LAN is a MAC address-in-user Datagram Protocol (UDP) tunneling mechanism. It identifies the Layer 2 segment through a 24-bit segment identifier called the VXLAN network identifier (VNI). The large VNI range allows the fabric to scale to 16 million segments, whereas a traditional Layer 2 network can scale to only 4096 VLANs. The original Layer 2 frame has a VXLAN header added and is then placed in a UDP-IP packet, thus enabling VXLAN to tunnel a layer packet over a Layer 3 network. Figure 4 shows the VXLAN packet format. Figure 4. VXLAN Packet Format VXLAN is an overlay technology that provides Layer 2 connectivity for workloads residing at noncontiguous points in the data center network. VXLAN provides flexibility by allowing workloads to be placed anywhere, and it offers the traffic separation required in a multitenant environment. Unlike in traditional Layer 2 technologies, VXLAN packets are transported through the underlay using IP information (Layer 3 header) and can take advantage of Equal-Cost Multipath (ECMP) Layer 3 routing. VXLAN EVPN Overview VXLAN Ethernet Virtual Private Network is a standards-based overlay solution that deploys VXLAN fabric with a Border Gateway Protocol (BGP)-based control plane that specifies the BGP EVPN control plane for overlays. The Cisco BGP control-plane solution for VXLAN uses the proven features of BGP to provide a more scalable, flexible, and policy-based alternative. It uses Multiprotocol BGP (MP-BGP) to distribute the required overlay reachability information. MP-BGP introduced new network layer reachability information (NLRI) called EVPN NLRI. This information carries both Layer 2 MAC address and Layer 3 IP address information at the same time (Figure 5). VXLAN EVPN provides significant advantages in the overlay network by getting the Layer 3 routing as close to the end host as possible. The BGP control plane is used to reduce flooding behavior and proactively distribute end-host information to participating VXLAN tunnel endpoints (VTEPs). The BGP control plane is used to: Discover VTEPs dynamically Distribute attached host MAC and IP addresses and avoid the need for the flood-and-learn mechanism for unknown unicast traffic Terminate Address Resolution Protocol (ARP) requests early to avoid flooding Many data centers today deploy a two-tier spine-and-leaf architecture for better scalability and flexibility. The traditional Layer 2 networks are contained in the leaf (top of rack) switches. VXLAN EVPN is used to extend these Layer 2 domains over the Layer 3 network for connectivity between the leaf switches. The leaf switches (which are also VTEP devices) run Multiprotocol Interior BGP (MP-iBGP) and peer with route reflectors that run on the spine switches. The function of the route reflectors is to reflect BGP updates between ibgp peers so that they don t need to form a fully meshed ibgp peering topology. 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 5 of 18

Figure 5. BGP EVPN Control Plane for VXLAN VXLAN EVPN Integration with LISP Host Move Detection in a VXLAN EVPN Fabric In the VXLAN EVPN fabric, the host routes and MAC address information are distributed in the MP-BGP EVPN control plane, which means that the fabric itself performs the host detection. The LISP site gateways use these host routes for triggering the LISP mobility encapsulation and decapsulation. LISP, when integrated with VXLAN fabric, provides ingress route optimization for traffic from the clients to the data center (Figure 6). Figure 6. LISP Functional Roles in A VXLAN Fabric For detailed configuration of VXLAN using the EVPN control plane, please see the following white paper: http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-734107.html - _Toc414541701. 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 6 of 18

When a virtual machine or host attaches to a leaf or top-of-rack (ToR) switch, the Layer 2 information is transported to its peers in the fabric using MP-BGP. This approach helps ensure connectivity between hosts within a data center fabric (Figure 7). Figure 7. Host 1 in VLAN 1000 Attaches to Leaf or ToR Switch 1 and Is Associated with VNI 5000 When the virtual machine or host moves from one leaf switch to another, the new leaf switch detects that a virtual machine has moved behind it by snooping on Domain Host Configuration Protocol (DHCP) or ARP packets. It populates the reachability information in MP-BGP and advertises the updated MAC address route to its peers with an updated sequence number (Figure 8). Figure 8. Host 1 Moves from Leaf or ToR Switch 1 to Leaf or ToR Switch 3 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 7 of 18

When the original leaf or ToR switch receives the route update with the modified sequence number, it sends a withdraw message for the stale reachability information (Figure 9). Figure 9. BGP Control Plane: Old Route Withdrawn from Leaf or ToR Switch 1 Host Mobility across VXLAN EVPN Fabrics When the leaf or ToR switch detects a host movement across data centers, it injects that host route into the MP-BGP EVPN control plane with an updated sequence number. The sequence number is a mobility community attribute that represents the state of mobility. It increments every time the server moves from one location to another. This sequence number attribute has to be carried to the original leaf or ToR switch from which the host moved, because it needs to withdraw that particular host route from BGP. The host route withdrawal happens only when the leaf or ToR switch receives a route with an updated sequence number. LISP currently cannot carry the mobility community attribute across the data center through the WAN. To help LISP achieve mobility semantics across VXLAN EVPN fabrics, you need to establish an Exterior BGP (egbp) relationship between the data centers. This ebgp relationship is used to carry the mobility community attribute in BGP EVPN across the data center sites for the stale reachability information (Figure 10 and figure 13). 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 8 of 18

Figure 10. HOST Mobility across Data Centers with LISP In Figure 10: 1. The end system or server, after moving to a new location, sends a DHCP and ARP packet to join the new network. 2. The leaf or ToR switch detects the new host and redistributes the IP address and MAC reachability information in the MP-BGP EVPN control plane with an updated sequence number. This sequence number attribute is carried across the data centers using an ebgp relationship between AS 65001 and 65002. When the original leaf or ToR switch receives the route information with an updated sequence number, it withdraws its original route from BGP. When the host first comes online (before moving across data centers), the sequence number attribute will be 0. This value indicates that this was the first time that the host is coming online in any data center (Figure 11). Figure 11. Host Mobility with Sequence Number 0 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 9 of 18

After the host moves from one location to another, the sequence number is updated to 1, which triggers the route update through the ebgp connection and the route withdrawal from the original leaf or ToR switch (Figure 12). Figure 12. Host Mobility with Sequence Number 1 3. When the LISP site gateway (also running MP-BGP EVPN in the fabric) detects this new host, it sends a map-register message to the map-system database to register the new IP address in its own data center (BGP AS 65002). 4. When the map system receives the map-register message from BGP, AS 65002 sends a map-notify message to the old LISP site gateways, notifying them that the host has moved from their data center. This message helps ensure that the LISP site gateways install a Null 0 route for that prefix in their routing tables. This Null 0 prefix indicates that the host is in a location remote to that data center. Figure 13. LISP Map System Updates 5. When the clients in the remote branch sites try to send traffic to the LISP site gateways at which the host was present (BGP AS 65001) before the mobility event, the site gateways see that the host is reachable through a Null 0 route. This event triggers a solicit-map request (SMR) from the site gateways to the LISP-enabled router in the branch site asking it to update its database. 6. The branch router then sends a map request to the mapping system asking for the new location of the host. This request is relayed to the LISP site gateways to which the host has moved (BGP AS 65002). 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 10 of 18

7. The LISP site gateways in BGP AS 65002 unicast a map reply to the LISP-enabled branch router asking it to update its database with the new location. Now data traffic starts to flow to the correct data center (BGP AS 65002). Summary LISP as a solution is very easy to configure (with just a few commands, as shown in the configurations that follow), and it provides an optimal way to resolve ingress route optimization challenges that result from workload mobility across data centers. The Cisco Nexus 7000 Series and 7700 platform are switches with comprehensive feature sets that can be used to implement the VXLAN-to-LISP solution discussed in this document using the F3 line cards. F3 line cards provide multiple-data-plane encapsulation in hardware and control-plane protocols. VXLAN encapsulation is implemented in hardware on the southbound side, and LISP is implanted in hardware on the northbound side on the F3 cards, making the Cisco Nexus 7000 Series and 7700 platform with F3 line cards an excellent solution. Functional Roles and Configuration Figure 14 shows the topology of the LISP solution. Figure 14. Topology * In this topology the EBGP EVPN relationship between the two data centers is through an Layer 3 Data-center Interconnect (DCI). The Layer 3 connection between the data centers is highlighted using green dotted lines in the above topology 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 11 of 18

Hardware and Software Details Table 1 summarizes the hardware and software versions used in the configuration example. Table 1. Hardware and Software Used in Configuration Example Functional Role Hardware Platform Software Version Border spine and border leaf Cisco Nexus 7000 Series and 7700 platform with F3 line card Cisco NX-OS Software Release 7.2 Map server and map resolver Cisco ASR 1000 Series Aggregation Services Routers Cisco IOS XE Software Release 3.13.2 Border Spine Configuration in Data Center 1 (BGP AS 65001) This section summarizes the steps for configuring LISP for hand-off from VXLAN on the border spine or border leaf switch. Step 1. Enable the LISP control plane. Step 2. Configure the LISP map-server and map-resolver reachability. Step 3. Configure the LISP hand-off for the tenant VRF instances. The following example shows a configuration for a two-tenant VRF instance. 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 12 of 18

* If you need to configure additional EID (IP address) subnets to map to the VRF instance, then you will have to create another dynamic EID subnet name. Example: The LISP instance ID provides a means of maintaining unique address spaces in the control and data plane. Instance IDs are numerical tags defined in the LISP canonical address format (LCAF). The instance ID has been added to LISP to support virtualization. When multiple organizations within a LISP site are using private addresses as EID prefixes, their address spaces must remain segregated to prevent address duplication. An instance ID in the address encoding can be used to create multiple segmented VPNs within a LISP site at which you want to keep using EID-prefix-based subnets. The LISP instance ID is currently supported in LISP ingress tunnel routers and egress tunnel routers (ITRs and ETRs), map server (MS), and map resolver (MR). The LISP locator VRF is used to associate a VRF table through which the routing locator address space is reachable with a router LISP instantiation. 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 13 of 18

Border Leaf Configuration in Data Center 2 (BGP AS 65002) Configuration of border leaf is the same as the border spine we discussed above For the other Border Spine in Data center 1(BGP AS 65001) and Border Leaf in Data center 2 (BGP AS 65002) the above configuration can be replicated. LISP Map-System Database Configuration Step 1. Configure the map server and map resolver on the switch. The map server and map resolver can be on either the same device or multiple devices. The scenario here uses an ASR 1000 Series router as the map server and map resolver. 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 14 of 18

Branch Site Configuration Verification To check for the EID (host IP address) learned on the LISP site gateway on a Cisco Nexus 7000 Series or 7700 platform switch, use the configuration shown here. To check for LISP map-cache entries on the map server, use the configuration shown here. 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 15 of 18

Conclusion This document provided a brief overview of VXLAN, VXLAN EVPN, and LISP before delving into how to integrate VXLAN EVPN with LISP. Appendix: Other Benefits of LISP in the Data Center LISP also supports these additional capabilities in your data center environment: IPv6 enablement Multitenancy and large-scale VPNs Efficient multihoming at the WAN edge IPv6 Enablement Enterprises wanting to use IPv6 often have problems because their current WAN supports only IPv4 traffic. LISP can help resolve this problem because you can transition to IPv6 in phases while still having other sites and the underlay network on IPv4. This technique is an efficient way to create and operate IPv6 islands within the current network deployment. You can do this using the existing IPv4 underlay by encapsulating IPv6 host packets within IPv4 headers. LISP provides support for both IPv4 and IPv6 EIDs and RLOCs (Figure 15). Figure 15. IPv6 Enablement with LISP 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 16 of 18

Multitenancy and Large-Scale VPNs LISP implements location and ID separation, which creates two namespaces: one for RLOCs (locations) and one for EIDs (IP addresses). These namespaces provide tenant separation using the LISP mapping system because LISP binds virtual routing and forwarding (VRF) to instance IDs. The LISP instance ID is a 24-bit value, which is included in the LISP header to provide control- and data-plane traffic separation. The LISP multitenancy solution also supports VPNs across enterprise networks to extend the network segmentation beyond local network boundaries. This extension is accomplished with multiple VRF instances using the LISP mapping system. Each VRF instance is tied to instance IDs for the address space (EID) in the VRF instance. This use case enables all the new VRF instances to be transported over one WAN network separated logically using VPNs (Figure 16). Figure 16. Multitenancy and Large-Scale VPNs Efficient Multihoming at the WAN Edge The built-in multihoming and traffic engineering features are one of the primary benefits of LISP. Multihoming with LISP is the capability to efficiently adjust the load on each WAN link without having to use advanced BGP traffic engineering. This is accomplished very simply by setting the RLOC weight. This approach enables you to manage and balance the utilization of the ingress bandwidth by setting the priorities. This design offers preference for egress tunnel routers (ETRs) over others, allowing some systems to act as primary ETRs and others to act as backups, thus inherently providing multihoming. This feature is implemented using the priority field, with lowerpriority systems being preferable over higher-priority systems (Figure 17). Figure 17. Multihoming at the WAN Edge 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 17 of 18

For More Information For a detailed understanding of VXLAN and LISP, see: http://www.cisco.com/c/en/us/products/collateral/switches/nexus-5000-series-switches/white-paper-c11-733618.html http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white_paper_c11-693627.html Printed in USA C11-734843-00 06/15 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 18 of 18