WHITEPAPER Bringing MPLS to Data Center Fabrics with Labeled BGP
Bringing MPLS to Data Center Fabrics with Labeled BGP MPLS is a well-known and mature technology typically used in service provider environment. Virtual Private Network (VPN) and Traffic Engineering are the two main MPLS applications. However, MPLS can also be used in data center deployment especially to interconnect data centers to expand L2 boundary across an MPLS network. This may be a good alternative to any emerging overlay technologies such as VXLAN for service providers. This is due to the fact that service providers or even many enterprises have been using MPLS for other services. Using MPLS in data center for overlay is nothing but a natural extension for them. Moreover, they don t have to deal with another overlay technology that adds complexity and difficulty both in administration and operation. Though Labeled BGP is not a new technology, it is new in data center especially with SDN white box switches. PicOS is the first white box switch operating system to support this technology. This white paper outlines how Labeled BGP can be used as an overly solution by using PicOS on SDN white box switches. What is Labeled BGP? Labeled BGP is defined in RFC 3017 that outlines label allocation between BGP peers. Labeled BGP is very useful where MPLS network is segmented within an autonomous system (AS) or spanned across multiple-as. It provides scalability by isolating IP routing between MPLS domains or segments because core devices don t have to learn all IP routes and additionally, routes can be summarized. Figure 1 illustrates this behavior. In principle, Labeled BGP is very similar to MPLS VPN in the sense that in MPLS VPN, provider edge (PE) device allocates two labels to an unlabeled packet. In Labeled BGP, edge devices typically receive packets with two labels. The inner label identifies MPLS service and the outer label identifies LSP. In edge device, outer label is replaced with two labels and hence, resulting packet has three labels. The new outer label is used to forward packet within core network. The middle label is used to forward packet to the final edge or PE device. The third label or bottom label represents MPLS service. Figure 1. Labeled BGP 1 WHITEPAPER
Using Labeled BGP in the Data Center Following Microsoft and Facebook leadership, BGP is becoming the standard protocol to build CLOS based data center. Their design is described in this document: https://datatracker.ietf.org/doc/draft-ietf-rtgwg-bgp-routing-large-dc/ MPLS is the de-facto standard for service provider WAN or core networks. So for a service provider building a large CLOS data center that already decided to use BGP in their data center and MPLS in their core network, it is becoming an obvious choice to use Labeled BGP to have an end-to-end MPLS network. Labeled BGP can be used in the data center as an overlay technology instead of VXLAN. Figure 2, illustrates Labeled BGP in data center. In this scenario, MPLS runs between OVS on VM and leaf switch, and Labeled BGP runs between data center switch and upstream router. Figure 2. Labeled BGP in the Data Center PicOS Architecture 2
Using different encapsulation protocol between the data center and the core brings some complexity in the network are the tunnels need to be stitched together to build an end-to-end service (see Figure 3). Those stitching mechanisms have to be built for all the services type for example L3VPN or L2VPN. Building translation between protocols also incurs the risk of loosing information and adding point of failure in the network. All those issues can be avoided by using an end-to-end MPLS LSP. Figure 3. Stitching VXLAN and MPLS in the Data Center Edge Deploying Labeled BGP in Data Center with PicOS and SDN White Box In a data center using this solution the MPLS ingress router would typically be the host itself. More precisely, the virtual switch of the data centers hosts. Very similar to how most virtual switch are using the VXLAN encapsulation. This section outlines how Labeled BGP can be deployed in data center with PicOS on SDN white box switches. In this use case, the ingress LSPs are routers simulating virtual switch in a real production network. The ToRs will receive (i.e. incoming packet) packets with two MPLS labels. However, PicOS also supports Labeled BGP solution with no (i.e. pure IP packet), one, two or three labels or more labels. But, packets with up to three labels are tested and verified. Additionally, PicOS supports label BGP for IPv4 and IPv6 as well as in ibgp and ebgp environments. LDP or 3 WHITEPAPER
dynamic label distribution protocol is typically used in MPLS deployment, but this is not the case in the data center. In fact, label mapping on the edge of the data center is mainly static and additionally, it is possible to run MPLS between ToR switch and OVS (or vrouter) running on hypervisors where label distribution can be done centrally. Therefore, static label allocation on ToR switches is not only acceptable but also desirable. Test Bed Configuration The test bed consists of three SDN white box switches running PicOS 2.6 and one IXIA test equipment. Figure 4 exhibits this setup. Label BGP is configured on all routers in a single AS. Router B is used, as BGP RR and routers A and C are RR clients. Next-hop-self is enabled on all routers. Static label mapping is used for MPLS label allocation in all routers. PicOS does not support LDP.H1 and H2 are IXIA based hosts used to send and receive label packets to routers A and C. Figure 4. PICA8 Labeled BGP Test Bed Configuration PicOS Architecture 4
Pica8 Labeled BGP Configurations Steps Step Command Description 1 configure terminal admin@xorplus> configure 2 set protocols bgp bgp-id <IPv4> set protocols bgp bgp-id 1.1.1.1 3 set protocols bgp local-as <text> set protocols bgp local-as 1 4 set protocols bgp peer <peer ip address> as <as-number> set protocols bgp peer 1.1.1.1 as 1 5 set protocols bgp peer <peer ip address> local-ip <ip-addess> set protocols bgp peer 1.1.1.1 local-ip 2.2.2.2 6 set protocols bgp peer <peer ip address> ipv4- label-unicast/ ipv6-label-unicast <true/false> set protocols bgp peer 1.1.1.1 ipv4-label-unicast true 7 set protocols mpls in-label [16..1048575] connect/connect6 <IPv4Net>/<IPv6Net> set protocols mpls in-label 44 connect 192.168.2.0/24 8 set protocols mpls in-label [16..1048575] nexthop <IPv4>/ <IPv6> set protocols mpls in-label 44 next-hop 1.1.1.1 Enters the global configuration mode. Configure bgp router-id Configure bgp AS Number Configure bgp peer with corresponding AS number Configure local peering source address for a particular bgp peer Configures Labeled BGP for a particular BGP peer Assigns MPLS label mapping for the top-most label and associates destination network with the top-most MPLS label Enables MPLS label mapping on edge bgp router, in which the next-hop indicates where the MPLS packet will be forwarded with the corresponding MPLS 5 WHITEPAPER
Pica8 Labeled BGP Configuration and Verification Details Routers A and C are edge routers and router B is a core router. Both A and B receive IP packets from H1 and H2 with two labels and send packet to B with three labels. This represents an MPLS VPN use case where top label represents transport LSP and bottom label identifies VPN. Step 1: Configure Loopback interface set vlan-interface loopback address 28.28.28.28 prefix-length 32 set vlan-interface loopback address 17.17.17.17 prefix-length 32 set vlan-interface loopback address 31.31.31.31 prefix-length 32 Step 2: Configure VLAN-Interface set vlans vlan-id 200 l3-interface vlan200 set vlans vlan-id 300 l3-interface vlan300 set vlan-interface interface vlan200 vif vlan200 address 192.168.2.1 prefix-length 24 set vlan-interface interface vlan300 vif vlan300 address 192.168.3.1 prefix-length 24 set interface gigabit-ethernet te-1/1/6 family ethernet-switching native-vlan-id 300 set interface gigabit-ethernet te-1/1/38 family ethernet-switching native-vlan-id 200 set vlans vlan-id 300 l3-interface vlan300 set vlans vlan-id 500 l3-interface vlan500 set vlan-interface interface vlan300 vif vlan300 address 192.168.3.2 prefix-length 24 set vlan-interface interface vlan500 vif vlan500 address 192.168.5.2 prefix-length 24 set interface gigabit-ethernet te-1/1/3 family ethernet-switching native-vlan-id 500 set interface gigabit-ethernet te-1/1/2 family ethernet-switching native-vlan-id 300 set vlans vlan-id 500 l3-interface vlan500 set vlans vlan-id 700 l3-interface vlan700 set vlan-interface interface vlan500 vif vlan500 address 192.168.5.1 prefix-length 24 set vlan-interface interface vlan700 vif vlan700 address 192.168.7.1 prefix-length 24 set interface gigabit-ethernet te-1/1/46 family ethernet-switching native-vlan-id 500 set interface gigabit-ethernet te-1/1/48 family ethernet-switching native-vlan-id 700 Step 3: Configure static route on A, B, C, So that the loopback interface could be reachable set protocols static route 17.17.17.17/32 next-hop 192.168.3.2 set protocols static route 28.28.28.28/32 next-hop 192.168.3.1 set protocols static route 31.31.31.31/32 next-hop 192.168.5.1 set protocols static route 17.17.17.17/32 next-hop 192.168.5.2 PicOS Architecture 6
Step 4: Configure BGP A: B: C: set protocols bgp bgp-id 28.28.28.28 set protocols bgp local-as 17 set protocols bgp fast-external-fallover disable true set protocols bgp peer 17.17.17.17 local-ip 28.28.28.28 set protocols bgp peer 17.17.17.17 as 17 set protocols bgp peer 17.17.17.17 next-hop-self true set protocols bgp bgp-id 17.17.17.17 set protocols bgp local-as 17 set protocols bgp route-reflector cluster-id 17.17.17.17 set protocols bgp peer 28.28.28.28 local-ip 17.17.17.17 set protocols bgp peer 28.28.28.28 as 17 set protocols bgp peer 28.28.28.28 next-hop-self true set protocols bgp peer 28.28.28.28 client true set protocols bgp peer 31.31.31.31 local-ip 17.17.17.17 set protocols bgp peer 31.31.31.31 as 17 set protocols bgp peer 31.31.31.31 next-hop-self true set protocols bgp peer 31.31.31.31 client true set protocols bgp bgp-id 31.31.31.31 set protocols bgp local-as 17 set protocols bgp peer 17.17.17.17 local-ip 31.31.31.31 set protocols bgp peer 17.17.17.17 as 17 set protocols bgp peer 17.17.17.17 next-hop-self true Step 5: Enable Labeled BGP on A,B,C set protocols bgp peer 17.17.17.17 ipv4-label-unicast true set protocols bgp peer 28.28.28.28 ipv4-label-unicast true set protocols bgp peer 31.31.31.31 ipv4-label-unicast true set protocols bgp peer 17.17.17.17 ipv4-label-unicast true Step 6: Advertise bgp route set protocols bgp network4 192.168.2.0/24 7 WHITEPAPER
Step 7: Check BGP route table run show bgp routes ipv4 label Status Codes: * valid route, > best route Prefix Nexthop In-label/Out-label ------------------ ----------------- ---------------------- *> 192.168.2.0/30 0.0.0.0 nolabel/nolabel run show bgp routes ipv4 label Status Codes: * valid route, > best route Prefix Nexthop In-label/Out-label ------------------ ----------------- ---------------------- *> 192.168.2.0/30 28.28.28.28 16/imp-null run show bgp routes ipv4 l Possible completions: <IPNet> Print BGP IPv4 routes of specified prefix label Print BGP IPv4 routes with labels run show bgp routes ipv4 label Status Codes: * valid route, > best route Prefix Nexthop In-label/Out-label ------------------ ----------------- ---------------------- *> 192.168.2.0/30 17.17.17.17 17/16 Step 8: Check MPLS forwarding table on A,B,C run show mpls forwarding-table ipv4 all run show mpls forwarding-table ipv4 all 16 imp-null 192.168.2.0/30 te-1/1/2 192.168.3.1 run show mpls forwarding-table ipv4 all 17 16 192.168.2.0/30 te-1/1/44 192.168.5.2 PicOS Architecture 8
Step 9: Configure MPLS label mapping on C set protocols mpls in-label 1000 connect 192.168.2.0/30 Note : The label 1000 is the top label that incoming BGP domain Step 10: Configure MPLS label mapping on A set protocols mpls in-label 2000 next-hop 192.168.2.2 Note : The label 2000 is the second-level label that incoming BGP domain, and it s the outgoing label that out of BGP domain the next-hop is indicate the next-hop ip address where the mpls packet forwarding and with a label 2000 Step 11: Check the mpls forwarding table on A,B,C run show mpls forwarding-table ipv4 all 2000 2000 -- te-1/1/38 192.168.2.2 run show mpls forwarding-table ipv4 all 16 imp-null 192.168.2.0/30 te-1/1/2 192.168.3.1 run show mpls forwarding-table ipv4 all 17 16 192.168.2.0/30 te-1/1/44 192.168.5.2 1000 16 192.168.2.0/30 te-1/1/44 192.168.5.2 Step 12: H2 sends mpls packet with 2-level mpls label to H1 9 WHITEPAPER
Pica8, Inc. Corporate Headquarters 1032 Elwell Court, Suite 105 Palo Alto, California 94303 USA 650-614-5838 www.pica8.com Pica8, Inc., 2015. All rights reserved. Produced in the United States 04/15. Pica8 and PicOS are trademarks of Pica8, Inc. Pica8 and PicOS trademarks are intended and authorized for use only in countries and jurisdictions in which Pica8, Inc. has obtained the rights to use, market and advertise the brand. Pica8, Inc. shall not be liable to third parties for unauthorized use of this document or unauthorized use of its trademarks. References in this publication to Pica8, Inc. products or services do not imply that Pica8, Inc. intends to make these available in all countries in which it operates. Contact Pica8, Inc. for additional information.