MPLS L3 VPN Supporting VoIP, Multicast, and Inter-Provider Solutions Luyuan Fang ATT MPLSCon 2005, NYC The world s networking company SM
Outline Overview of the L3 VPN deployment VoIP over MPLS VPN MPLS L3 Multicast VPN Inter-provider VPNs Conclusions 1
Overview of MPLS L3 VPN Deployment (1) MPLS VPN the fast growing services Carriers see big growth in IP-based VPN services, a few providers, tripled or nearly tripled the number of companies buying their monthly IP VPN services in 2004, an upswing they expect to continue in 2005. [Computerworld, 1/6/05.] Advanced MPLS L3 VPN deployment - Features supported and under development» Enterprise VPN» Carrier s Carrier VPN» Inter-AS VPN within in single provider» Inter-provider VPN» QoS support for all types of VPNs» VoIP VPN support» Multicast VPNs» IPv6 VPNs 2
Overview of MPLS L3 VPN Deployment (2) Observations: Access speed: T1 is still the dominating port speed, >90%, but all speeds are required by customers QoS: More than 50% of the VPN interfaces requires QoS, which is not the case in Internet Services. PE-CE protocol: Majority using BGP than Static. This is the reverse case from Internet Services. VPN routes: VPN routes per site on average: 2-3 routes per VPN sites. Most VPNs are well behaved, with small percentage VPNs routes/sites ratio > 10 Routes per port Routes per customer Total routes today Ave. 2-3 Ave. ~ 300 >> 180,000 VPN Ave. <3 Ave. ~7 ~ 180,000 Internet 3
VoIP over MPLS VPN (1) Advantages of VoIP over MPLS VPN Compare with Leased line, FR/ATM: MPLS VPN provides any-to-any connection, avoid full mesh - cost effective Compare with Internet: MPLS VPN provides routing isolation MPLS VPN for VoIP applications VoIP VPN services for external customers VoIP backbone transport for internal VoIP networks Connecting calls / conference bridge with various access methods VoIP VPN and data VPN via single physical interfaces CoS for VoIP VPN Marking at the PE, e.g. with 90% real time for VoIP traffic Core QoS support, high priority treatment for VoIP traffic Reliability consideration Consider CE, PE diversity, POP diversity, AS diversity, provider diversity BGP convergence time: considering tuning the BGP timers - advertisement interval, keepalive/hold timers, and use bgp external fast failover 4
VoIP over MPLS VPN (2) Extranet VPN for Customer VoIP, and support VoIP VPN and data VPN via single physical interface interface BE, GW, Servers VRF VoIP Complex Import/export VoIP server rts Import VPN-R VoIP rts Import VPN-G VoIP rts GK, CCE, Servers VoIP Server complex VoIP Server complex VRF R-VoIP Import VoIP server rts Import/export VPN-R VoIP rts VRF R-data Import/export VPN-R data rts MPLS Backbone VRF G-VoIP Import VoIP servers routes Import/export VPN-R VoIP rts VRF G-data Import/export VPN G-data rts Data LAN Data LAN PBX PBX IP PBX IP PBX 5
VoIP over MPLS VPN (3) Separate provider core VoIP VPN and extranet cust. VoIP VPN Call signaling/setup CCE Media Servers BE NG-BE VoIP complex 2 Negotiate connection CCE Media Servers BE NG-BE VoIP core VPN Import/export VoIP core rts of VoIP complex, CCE, BE VoIP core VPN Cust. Extranet VPN Import/export cust. VoIP rts Inport local BE rts BE Cust. VoIP VPN Data LAN PBX 1 Call request BE 3 Alert and connect the call Cust. Intranet Conf. bridge Local access Via Cust. Extranet VPN Via VoIP Core VPN 6
VoIP over MPLS VPN (4) Separate provider core VoIP VPN and extranet cust. VoIP VPN VoIP packets transport after call setup CCE Media Servers BE NG-BE VoIP complex CCE Media Servers BE NG-BE VoIP core VPN Import/export VoIP core rts of VoIP complex, CCE, BE BE VoIP core VPN Cust. VoIP VPN Cust. Extranet VPN Import/export cust. VoIP rts Inport local BE rts Data LAN BE Cust. Intranet PBX Conf. bridge Local access VoIP packet flow via Cust. Extranet VPN VoIP packet flow Via VoIP Core VPN 7
VoIP over MPLS VPN (5) VoIP VPN application applications: call connections Local, long distance, and 800 calls with legacy or IP phones CCE CCE Long distance 800 numbers Media Servers BE VoIP complex Media Servers BE PSTN NG-BE NG-BE PSTN 1 VoIP core VPN Cust. VoIP VPN 3 Data LAN 2 Cust. Intranet Conf. bridge PBX Local access IP PBX 8
VoIP over MPLS VPN (6) VoIP VPN application applications: Conference connections VoIP VPNs connecting employees to enterprise conference bridge With Local, long distance, and 800 access, using legacy or IP phones CCE CCE Long distance 800 numbers Media Servers BE VoIP complex Media Servers BE PSTN NG-BE NG-BE PSTN 1 VoIP core VPN Cust. VoIP VPN Data LAN 2 3 Cust. Intranet PBX Conf. bridge Local access IP PBX 9
Multicast for BGP/MPLS VPNs The growing customer demands for Multicast VPN (MVPN) support With Multicast widely deployed in enterprise network, the need of using Multicast to connect the enterprises sites via provider VPNs grows Multicast VPN application examples Finance Video conference, corporate communication Video broadcast Media, Enterprise, Content distributor Large files distribution, software upgrade to all remote offices E-education 10
MVPN Customer Support Requirements No additional configuration or new routing and management protocols required for on CE devices Support all features currently supported by Unicast VPNs Support all access types, e.g. PPP, MLPPP, FR encap, etc. Support all access speeds, e.g. FT1 to OC192 QoS support for all types of MVPN Addressing: allow cust. Addr. Overlaping, no restrictions to cust. multicast group addresses Multihoming load sharing: network based e.g. eibgp; customer controlled addr. splitting Support all VPN topologies, intranet, extranet, any-to-any, hob-and-spoke Support Carrier s Carrier VPN, with labeled connections of PE-CE Inter-provider solutions all options (a, b, and c) must be supported IPv4 and IPv6 VPN Security protecting provider infrastructure, and customer networks SLA measurements Management, monitoring and troubleshooting Support Multicast and Unicast in the same VPN Additional Multicast customer support RP Engineering, RP-outsourcing Fragmentation Minimum delay and optimal routing 11
Current MVPN implementation (Based on IETF drafts listed on page 17) Advantages of MVPN implementation for customers No CE-CE PIM adjacency is required no full mesh CE-CE tunnels, PIM adjacency is between PE-CE No changes required in customer network, no impact of existing cust. Multicast deployment No restrictions to cust. multicast group addresses Exchange MVPN routing information PE is VPN aware, processing PIM messages within the context of VRF» A VRF supports Unicast and Multicast tables» Per VRF routing and forwarding PE maintains PIM adjacency with all other PEs supporting the same VPN PE maintains distinct PIM instance per VPN RPF checks using unicast routing information in the same VRF Multicast Tunnels (MT) are built to exchange PIM messages» MT is like a LAN for the VRFs Note that the # of provider states is function of # of VPNs and # of PEs, not # of (S,G)/(*,G) states of all customers» Aggregate cust. (S,G) or (*,G) in MVPN to one or more (S,G) or (*,G) in SP network 12
Multicast routing info exchange VPN A VPN B PIM PIM PIM Multicast Tunnel B Multicast Tunnel A PIM VPN B PIM PIM VPN A VPN A VPN B PIM adjacency between VRFs for VPN R PIM adjacency between VRFs for VPN B 13
Forwarding MVPN traffic with Multicast tunnels/trees Trade-offs PIM-SM vs. PIM-SSM PIM-SM or Bidirectional PIM -> One multicast tree per VPN» Less states, may not be optimal path as all traffic need to go through RP PIM-SSM -> One multicast tree per VPN per PE» More states, optimal path in terms of delay, not optimal in terms of bandwidth Using default Multicast Data Tree (MDT) + Data MDT to avoid sending high rate traffic to PEs with no receivers Build Default MDT for customer PIM control traffic (using PIM-SSM, Bidrectional PIM, or PIM SSM), it will send multicast traffic to all PEs (with receivers or not) in the MVPN Build on demand Data MDT for high rate traffic between PEs connecting high rate Mcast sources and actives receivers only (with configurable threshold)» Saved bandwidth by not sending to PEs with no receivers» Optimal path in terms of delay, still not optimal in terms of network bandwidth consumption 14
MVPN data forwarding Using data multicast tree - avoid sending to non-receivers VPN A VPN B VPN B G1 PE2 P PE3 P RP P VPN B PE1 P PE4 VPN A S1 G1 VPN A VPN B G1 Default Multicast Tree for VPN B Data Multicast Tree for VPN B on demand 15
Engineering it Right Understand customer applications Important especially for initial deployment» How many sites are using Multicast among all sites in a VPN» How many sites are sending MVPN traffic simultaneously» What are the applications? Packet size? rt or nrt? Fragmentation sensitive?» QoS requirements» Other features required, e.g. CoS, MLPPP, multi-homing load sharing, etc. Engineering objectives Meeting all customer requirements» Providing full set of features as Unicast MPLS VPN» Minimum delay and optimal routing» High reliability in the core and access network» Same level of Security as Unicast MPLS VPN Engineering the network to meet the objectives Sizing up the PE and network ensure high stability and performance» Total number of States can be supported per PE, P, RP (if PIM-SM), and per network» Total number of MVPNs, and MVPN rts can be supported per PE, and per network» Total number of simultaneous Multicast stream can be supported per PE, network» Replication, and packet size impact to PE performance» Total percentage of real time (rt) vs. non real time (nrt) can be supported per PE, P, network 16
Dealing with Scalability, optimality challenges, and tunneling techniques Work in Progress in IETF (not a complete list) New proposals for MVPN requirements and solutions Requirements for Multicast in L3 Provider-Provisioned VPNs <draft-ietf-l3vpn-ppvpn-mcast-reqts-00.txt> Multicast in MPLS/BGP IP VPNs <draft-to-become-l3vpn-2547bis-mcast-00.txt> Multicast in BGP/MPLS VPNs and VPLS <draft-raggarwa-l3vpn-mvpn-vpls-mcast-01.txt> Existing proposals for current MVPN implementation Multicast in MPLS/BGP IPVPNs <draft-rosen-vpn-mcast-08.txt> Base Specification for Multicast in BGP/MPLS VPNs <drat-raggarwa-l3vpn-2547-mvpn-00.txt> Recent proposals for MPLS tunneling support for Multicast Signaling Requirements for Point to Multipoint Traffic Engineered MPLS LSPs <draft-ietf-mpls-p2mp-sig-requirement-01.txt> Extensions to RSVP-TE for Point to Multipoint TE LSPs <draft-ietf-mpls-rsvp-te-p2mp-01.txt> Label Distribution Protocol Extensions for Point-to-Multipoint Label Switched Paths <draft-minei-mpls-ldp-p2mp-00.txt> Multicast Extensions for LDP <draft-wijnands-mpls-ldp-mcast-ext-00.txt> Additional Ref: Multicast support in 2547 VPNs, is it good enough? Yakov Rekhter, MPLS World Congress, Feb. 2005 17
Inter-Provider L3 MPLS VPNs Needs for Inter-provider VPNs Global connectivity for enterprise customer VPNs Limited footprint of a single provider to cover all regions Deployment Status Inter-provider VPN has been deployed by many providers Most inter-provider connections today are one-to-one negotiation/arrangement between the two providers» Business models» Connection technical options» WAN address assignment» CoS consistency between the providers» Operation models» Network management capabilities and boundary» VPN feature consistency between the providers» End-to-end LSA 18
Provider Responsibility Models CE CE Provider A Provider B CE CE Responsibility options A Managed service - Provider A Unmanaged service - customer B Managed service - Provider B Provider A Provider A Provider B Provider AB Provider C A B C D E Provider B Optional: Provider B: including interconnect to provider C A Managed service - Provider A Unmanaged service - customer B Managed service - Provider B Example 1: Provider A buys whole sale VPN services from Provider B, provider A manages CEs connecting to both provider A and B. Responsibilities: A->A, B->A, C->A, D->B, E->A Example 2: Provider A extends its footprint through B and B s global connection to C, with unmanaged services. Responsibilities: A->customers, B->A, C->B, D->B (including inter-connection and VPN service arrangement to C), E->customers 19
Inter-provider Option A (RFC2547bis)! "#" CE1 PE1 VPN X VRF import routes with RT=500:2 RR 1 VRF ASBR 1 IPv4 routes exchange VRF ASBR2 $%'$( RR 2 PE2! "#" CE2 VPN X (1) Option A: Back-to-back VRF VPN Connectivity VPN X LDP PE1 label VPN CUST1 label PE1 RR 1 RR 2 P1 ASBR 1 ASBR2 P2 PE2 LDP ASBR2 label VPN CUST1 label CE1 CE2 VPN X (2) Option A: Back-to-back VRF Packet forwarding VPN X 20
Inter-provider Option B: (RFC2547bis)! "#" RR 1! "#" RR 2! "#" ) VPN X CE1 PE1 ASBR 1 ASBR2 Label exchange using MP-eBGP ASBRs Maintain all Inter-AS VPN v4 routes, No VRF PE2 CE2 VPN X LDP PE1 label VPN CUST1 label (1) Option B: MP-eBGP between ASBRs Connectivity ) PE1 RR 1 RR 2 P1 ASBR 1 ASBR2 P2 PE2 LDP ASBR2 label VPN CUST1 label ) VPN X CE1 (2) Option B: MP-eBGP between ASBRs Packet forwarding CE2 VPN X 21
Options AB comparison Option C is not considered for Inter-provider for security reason Implementation Scalability Security QoS ibgp / eibgp multipath Multicast VPN VPN routing exchange VPN topology Per VPN traffic count at ASBR Supported Per VPN IPv4 rts exchange e.g. per Hub ipvc in Hub-and-spoke Easy Option A VRF to VRF Easy - adopted by many providers as the first step VRF, logical link, BGP session need to be provisioned at ASBR on per VPN basis Authentication work needed May need to implement label to be able to carry EXP bit Supported same as under intra-as Option B MP-eBGP Medium adopted by some providers and more planned No VRF on ASBR, no per VPN provisioning on ASBR Route filtering and authentication work needed EXP bit naturally carried by labels between ASBRs Supported if implemented correctly to get around ASBR NH rewrite issue Supported with BGP and PIM extensions Need to filter out the unneeded VPN routing info, and remap RT Simple to implement Not easy need solutions 22
Option B implementation supporting ibgp and eibgp 456 (-5%- ) ) 3 "+ *+",-,+- *,.-/*-"0$ -/*1"#"2 3 - - ibgp and eibgp are ingress features - the multipath selection decision is made at the ingress PE. Every LSP need to be considered in the path selection process - PE3 in London receives both paths advertised for network X through PE1 and PE2 with different RDs, labels, and same next hop there are two distinct LSPs - Only one path will be used if PE3 compare BGP next hop alone it is ASBR2 for both paths - Both paths (via PE1, and PE2) will be used if PE3 compare BGP next hop and labels - Same for Intra-AS case if 2 or more VRFs for a single VPN on the same PE 23
Option B implementation supporting MVPN %7-" 456 %7-" Originating PE prefix preserved %7-" 3 9:.0 4, (4 ;,-,0 %7-" 58.-9-(4-7*# '6+,, 3 + PIM extension PIM RPF vector carry the proxy address when P routers don t know the rte of ingress PE New PIM Hello option to indicate the capability of processing RPF vector BGP extension RPF attribute to preserve the originating PE prefix to over come the problem when next hop is rewritten by ASBR. Using the route distinguisher (RD), the VPN specific BGP MDT table can be selected to enable the RPF interface for the source to be found 24
Conclusions BGP/MPLS VPN the fast growing services Continue advanced development to meet all customers needs Continue scaling improvement by working with vendors» VPN routes, BGP sessions, QoS policies, aggressive timers, MVPN, etc. Feature requirements VoIP VPN is a new highly in demand feature QoS support for all types of VPNs, especially VoIP Multi-homing load-balancing and redundancy in all scenarios Multi-service port: Internet and VPNs through the same customer interface Global reachability - Inter-AS and Inter-provider Multicast VPN, and Multicast Inter-AS VPN IPv6 VPN - e.g. 6PE solution with min impact to the core Major Challenges End-to-end QoS solutions for Inter-provider 25