White Paper Carrier Ethernet 26601 Agoura Road, Calabasas, CA 91302 Tel: 818.871.1800 Fax: 818.871.1805 www.ixiacom.com Rev. A, September 2007
2
Table of Contents Introduction... 4 Benefits of Carrier Ethernet... 4 The Goal: "Carrier Grade" Ethernet... 5 Technology That Enables Carrier Ethernet...7 Aggregation: Metro Ethernet Technologies... 7 Provider Ethernet... 9 Multi-protocol Label Switching (MPLS)...10 Service Management...10 Testing Challenges...12 Summary...12 References...12 3
Introduction Ethernet is far and away the dominant LAN technology used in networks today. Due to its speed, simplicity, plug-and-play capability, multipoint connectivity, and low cost it has been widely adopted and deployed. In its first stage of adoption, advanced features such as full-duplex interfaces, and advanced management capabilities, increasing speed (from 10 Mbps to 100 Mbps, and virtual LANs, or VLANs), enabled Ethernet to overcome other technologies such as token ring, FDDI, and ATM in the enterprise. Service providers face the challenge of offering scalable services that make full use of their converged legacy and Ethernet networks. In the second stage, Ethernet switches became more reliable and feature rich and Ethernet became the technology of choice for network POP interconnections and pointto-point WAN links. Service providers began using Ethernet instead of the more expensive packet over SONET (PoS) to interconnect routers, and chose Ethernet over token ring and FDDI to interconnect servers and storage in major data centers. As a result, Ethernet became widely used and understood in enterprise networks, while service providers chose Ethernet for inexpensive, reliable Layer 2 transport whenever they could. Then customers began asking for native Ethernet connectivity in metro and access networks over legacy technologies such as lower-speed time division multiplexed (TDM) circuits, highly complex ATM, or high-cost PoS. This lead to the third stage the adoption of Ethernet in access and metro networks and the development of carrier Ethernet services such as Ethernet Internet access, IPbased virtual private networks, and multicast networks. With Ethernet transport speeds from 1 Gbps to 10 Gbps now available, and customer demand continuing to grow, service providers face the challenge of offering scalable services that make full use of their converged legacy and Ethernet networks. Benefits of Carrier Ethernet For Service Providers Technology convergence provides CAPEX and OPEX reductions Access network technologies leverage Ethernet to provide backhaul the ability to move volumes of traffic from subscribers to broadband aggregation devices adjacent to the core network for IP DSLAMs, PON, WiMAX, and direct Ethernet over fiber/ copper Flexible Layer 2 VPN services, including private line, virtual private line, or emulated LAN offer new revenue streams 4
For the Enterprise A converged network for VoIP, data, video conferencing, and other services reduces overall costs Standardizing on Ethernet reduces complexity and benefits IT staff support and training budgets High-speed, low-latency service is easily upgraded by changing the service policy The Goal: "Carrier Grade" Ethernet The first native Ethernet services to emerge were point-to-point-based, followed by emulated LAN (multipoint to multipoint-based). Services were first defined and limited to metro area networks. They have now been extended across wide area networks and are available worldwide from many service providers. The term carrier Ethernet implies that Ethernet services are carrier grade. The benchmark for carrier grade has been set by the legacy TDM telephony networks to describe services that achieve five nines (9.9999%) uptime. Although it is debatable whether carrier Ethernet will reach that level of reliability, the goal of one particular standards organization is to accelerate the development and deployment of services that live up to the name. The term "Carrier Ethernet" implies that Ethernet services are "carrier grade." Carrier Ethernet looks to become the major component of next-generation metro area networks, which serve as the aggregation layer between customers and core carrier networks. A metro Ethernet network, which uses IP Layer 3 MPLS forwarding, is currently the hotbed of carrier Ethernet activity. The Metro Ethernet Forum (MEF) is a global industry alliance started in 2001. In 2005, the MEF formally defined carrier Ethernet services. Members of the MEF include the majority of the top network equipment manufactures and service providers in the world. Members serve on technical committees that author technical specifications defining services, architecture, management, and test and measurement. The MEF 6 specification defines carrier Ethernet services. MEF 6-defined basic connection point terminology: UNI: user network interface UNI-C: UNI-customer side UNI-N: UNI-network side NNI: network-to-network interface E-NNI: external NNI I-NNI: internal NNI EVC: Ethernet virtual circuit 5
MEN: metro Ethernet Network TLS: transparent LAN services SLA: service-level agreement SLS: service-level specification MEF 6 also defines service types: EPL: Ethernet private line EVPL: Ethernet virtual private line E-Line: Ethernet line ELAN: Ethernet LAN Service providers offer customers service-level agreements (SLAs) that guarantee the details of a service's performance. E-Line defines a service type that uses a point-to-point EVC, which can be delivered with an EPL or EVPL. An EPL uses a unique physical UNI port at each end. An EVPL uses a virtual port (VLAN) at each end of the service. ELAN defines a service type that uses a multipoint-to-multipoint EVC. It is important to note that, while the standard does specify Ethernet as the technology connecting customer equipment to a service provider network, it does not define what underlying technology is used to deliver the service. Typically, service providers provide customers service-level agreements or service-level specifications that guarantee the details of a service s performance. Ethernet service attributes are found in MEF specification 10.1. Service attributes include: Physical layer service attributes 10 Mbps full duplex 100 Mbps full duplex 10/199 Mbps Auto-negotiation full duplex 1 Gbps full duple 10 Gbps Full duplex Bandwidth Profile Service Attribute CIR: Committed Information Rate CBS: Committed Burst Size EIR: Excess Information Rate EBS: Excess Burst Size 6
EVC 1 Bandwidth Profileper EVC 1 UNI EVC 2 Bandwidth Profile per EVC 2 EVC 3 Bandwidth Profile per EVC 3 Figure 1: An Ethernet virtual connection is defined between UNI(s). Service attributes are first Technology That Enables Carrier Ethernet Figure 2: In typical carrier Ethernet architecture, there are three main components: access, aggregation/metro, and core. By definition, access must be Ethernet. Technologies used in ggregation networks and the core can vary. Aggregation / Metro Ethernet Technologies Legacy metro transport networks are built primarily of time-division multiplexing technology. TDM :circuit-switched services are optimized for delivering voice. The underlying technology of a TDM network is a SONET/SDH ring. A TDM network consists of digital multiplexers, digital access cross-connects (DACs), SONET/SDH add/drop multiplexers (ADMs), and SONET/SDH cross-connects. 7
The benefit of this type of service is that subscribers receive the specific bandwidths they have ordered. A major challenge is that TDM interfaces do not offer much flexibility in bandwidth options. Significant and expensive steps are required, such as fractional T1 (64 Kbps), full T1 (1.5 Mbps), DS3 (45 Mbps), OC3 (155 Mbps), OC12 (622 Mbps), OC 192 (10 Mbps). An increase in bandwidth requires new circuit provisioning plus, typically, a truck roll to install the service. In many markets, particularly in the United States, there is a significant installed base and investment in the TDM infrastructure. As service providers look to enable new packet-based services, they would like to leverage legacy investments where possible. Several new protocols are designed to accomplish that. Generic Framing Procedure (GFP) As service providers look to enable new packet-based services, they would like to leverage legacy investments where possible. Generic framing procedure (GFP) is defined by ITU-T G.7041. It allows mapping of variable-length, higher-layer client signals over SDH/SONET. GFP can be used to encapsulate native Ethernet frames into a SONET payload. There are two modes of GFP: generic framing procedure-framed (GFP-F) and generic framing procedure-transparent (GFP-T). GFP-F maps each client frame into a single GFP frame. GFP-F is used where the client signal is framed, or packetized, by the client protocol. GFP-T allows mapping of multiple 8b/10-b block-coded client data streams into an efficient 64b/65b block code for transport within a GFP frame. GFP-F is used when optimal bandwidth efficiency is desired at the expense of latency. GFP-T is used when low latency transport is required. When GFP is enabled it is commonly used in conjunction with virtual concatenation (VCAT). VCAT is an inverse multiplexing technique used to split SONET/SDH bandwidth into logical groups, which may be transported or routed independently. VCAT specified in ITU-T G.7043 provides the ability to multiplex many services onto the same transport. However, VCAT inherently induces a varying propagation delay known as differential delay, which can affect service. Another protocol, link capacity adjustment scheme (LCAS), is typically used with VCAT. LCAS is a method to dynamically increase or decrease the bandwidth of virtual concatenated containers in a hitless manner, which enables service providers to add bandwidth on demand to a data service. The LCAS protocol is specified in ITU-T G.7042. With the addition of GFP, a new breed of transport products emerged, known collectively as the multi-service provisioning platform (MSPP). MSPP supports SONET/SDH and Ethernet over SONET services. With it, Ethernet services can be easily added with the addition of new line cards. Resilient Packet Ring (RPR) IEEE 802.17 Resilient Packet Ring (RPR), defined by IEEE 802.17, is a Layer 2 transport architecture based on a dual, counter-rotating ring topology. RPR runs over SONET and provides packet-switching technology. With RPR, service providers can leverage their existing SONET/SDH infrastructures without significant upgrades. RPR provides high-bandwidth utilization by means of spatial reuse within the ring. A fairness algorithm provides efficient data transport under congested conditions without starving a single station (255-station limit). RPR has a protection switching mechanism that provides sub-50 ms service restoration with no reserved bandwidth requirements. There is integrated, three-level traffic class support built into the MAC with additional rate-limiting features. RPR rings also provide optimization for multicast traffic delivery. Existing SONET/SDH MSPPs can add RPR capability by adding RPR-capable modules. 8
Provider Ethernet Provider Ethernet is a new term that defines Ethernet technologies used in aggregation/ metro service networks. It refers primarily to an IEEE standards-based Ethernet switched network. There are several IEEE protocols that can be used to provide carrier Ethernet services in aggregation/metro networks. Many new greenfield deployments are using or considering pure Ethernet switched networks. Advantages over legacy technologies include bandwidth flexibility, flexibility in service provisioning, and lower costs. IEEE 802.1Q (Virtual LANs) The virtual LAN (VLAN) standard provides a method for tagging Ethernet frames with VLAN information. There are various ways the VLAN can be assigned, including port based, MAC based, and protocol based. A port-based implementation is the most common. With it, traffic from a specific port can be added to a virtual LAN. VLAN technology has been widely used in enterprise networks. Service providers are also using VLAN tagging to logically separate customer traffic in aggregation/met o networks. IEEE 802.1ad Provider Bridges (aka Q-in-Q) A challenge service providers face when using VLANs arises when dealing with traffic that has already been VLAN-tagged. That is where provider bridging also known as Q-in-Q or VLAN stacking comes into play. Q-in-Q provides customer VLAN transparency by adding an additional tag called the service tag, or S-tag. The original customer VLAN tag is then known as the C-tag. One of the limitations of Q-in-Q is that the 12-bit VLAN tag is limited to 4094 unique customer addresses that are not highly scalable for service provider networks. IEEE 802.1ah Provider Backbone Bridges (PBB, aka MAC-in-MAC) Provider Backbone Bridges (PBB) were designed to address the limitations of Q-in-Q and provide additional capabilities. PBB is similar to Q-in-Q in the fact that it encapsulates customer data. However, it uses a second MAC header and tag. The new tag, called the I-tag, or service instance identifier, is 24 bits long and provides highly scalable services. PBB also provides significant scale improvements by using a MAC address called the B-SA/B-DA (backbone source/destination) for forwarding traffic. There is inherent backwards compatibility for 802.1Q and 802.1ad networks because the address is encapsulated. PBB even be used in the core to provide hierarchy for Q-in-Q networks. However, in most cases, it is used in the aggregation/metro networks that connect to an IP/MPLS core. IEEE 802.1Qay Provider Backbone Bridge Traffic Engineering Before PBB-TE became an official project within the IEEE, it was known as provider backbone transport. This emerging standard, championed by Nortel, is now moving forward within the IEEE. PBB is used as the data plane that provides a mechanism to tunnel traffic over a service provider network, yet a critical requirement for service 9
providers is to support traffic-engineered paths. To accomplish this, the typical sourceaddress MAC learning capability and broadcasting unknown traffic function is disabled. Instead, traffic is explicitly mapped to a precisely configured network tunnel. The provisioning of these paths is typically supported by SNMP MIB commands set to each switch. Another critical component of PBB-TE is resiliency. Using PBB-TE 1:1, tunnel path protection is configured and Ethernet CFM (802.1ag) continuity check messages (CCMs) are used over each path for fault-detection-triggering notification. PBB-TE provides theoretical high scalability with a 58-bit space for tunnel ID and a 24-bit space for service ID. Service providers are interested in PBB-TE as an alternative to extending IP/MPLS with expensive routers. Multiprotocol Label Switching (MPLS) Multiprotocol Label Switching (MPLS) Multiprotocol label switching has established itself as a proven technology in service provider core networks. Most service providers have launched successful IP/MPLS Layer 3 VPN services. Additional development by the IETF has lead to two additional technologies that can be used to enable new Layer 2-based carrier Ethernet services. Psuedowire Emulation Edge-to-Edge (PWE3) The Internet Engineering Task Force (IETF) standard pseudowire emulation edge-to-edge (PWE3) provides a tunneling service over a MPLS core. The label switched path (LSP) can be manually or dynamically built. Specific traffic is then mapped on to the LSP. PWE3 can carry legacy technologies like frame relay and ATM and can also carry Ethernet traffic. PWE3 provides the inherent benefits of an MPLS network, including the ability to define backup paths with fast failover and QoS support. PWE3 can be used to build carrier Ethernet E-Line services. Virtual Private LAN Switching (VPLS) Virtual private LAN switching provides a virtual LAN service over an MPLS network. There are two standards within the IETF that define VPLS: VPLS-LDP (RFC 4762) and VPLS-BGP (RFC 4761). Similar to IP/MPLS L3 VPNs, VPLS is enabled at a provider s edge router, where a virtual switching instance (VSI) is defined. The virtual switch performs MAC address learning and floods broadcast, multicast, and unknown unicast traffic. MPLS uses split horizon and a full mesh of pseudowires (PW) for loop avoidance in the core. Spanning tree is not run in the core. Two standards, VPLS-LDP and VPLS-BGP, primarily differ in the control plane used to establish the mesh of pseudowires. VPLS is attractive to service providers since they can leverage their existing IP/MPLS networks and overlay a new L2 VPN service. Challenges include a requirement for full mesh PWs and the handling of flooding traffic, which is a scalability concern. Because of these issues, new developments are underway to implement hierarchical VPLS (H-VPLS). Service Management A major shortcoming in native Ethernet is the requirement to provide carrier-grade services. It has no management capability that would allow it to detect and report failures at Layer 2 over a virtual link. Other transport technologies in this space like frame relay, ATM, and SONET have a suite of services that provide the ability to monitor and troubleshoot the network. These services are called operations, administration, and maintenance (OAM). To replace these legacy technologies, Ethernet needs similar 10
capabilities. To address this, work has begun in IEEE and ITU-T. Within the IEEE there are two standards: IEEE 802.3ah E-OAM, Ethernet in the first mile (used in conjunction with access technologies), and IEEE 802.1ag connectivity fault management (CFM). Similar to the IEEE 802.1ag specification, the ITU-T has authored the Y.1731 OAM functions and mechanisms for Ethernet-based networks standard. These standards provide the tools to manage a network in the data plane and the control plane over an Ethernet service (E-Line or E-LAN) and to verify continuity, connectivity, and performance. The alternative, a higher-layer management solution like SNMP, is slow and does not provide easy correlation of information to verify a specific path an Ethernet virtual connection may take. IEEE 802.3ah Ethernet Operations, Administration, and Maintenance (OAM) (Clause 57) 802.3ah Ethernet OAM is known as link OAM since it runs over a point-to-point 802.3 link. EOAM provides mechanisms such as link monitoring, remote failure indication, remote loopback control, and OAM discovery. E-OAM uses a simple OAM PDU frame that runs directly over a standard Ethernet MAC header. OAM provides useful fault management capability, including fault isolation, fault detection, and link performance testing. E-OAM is considered most useful in the last mile to monitor Ethernet access services, but can also be used on any Ethernet link in the network. IEEE 802.1ag Ethernet Connectivity Fault Management (E-CFM) 802.1ag Ethernet CFM runs end-to-end over an Ethernet virtual circuit and is viewed as a critical component for delivering carrier-grade Ethernet services. E-CFM includes protocols that provide the ability to detect, verify, and isolate connectivity faults. It also runs directly over standard Ethernet frames that travel in-band with customer traffic. Devices that do not support E-CFM will forward the frames and not participate in the messaging. The three E-CFM protocols are continuity check, loopback, and link trace. Continuity check provides a keep-alive type of message over an Ethernet service and is used to detect and notify that a fault has occurred. The loopback protocol is like an IP ping at Layer 2. A loopback message (LBM) and a loopback reply (LBR) verify any faults. The link trace protocol is like an IP trace route at Layer 2. There is a link trace message (LTM) and a link trace reply (LTR) that isolate faults. E-CFM can define hierarchy within a domain, which gives an administrator visibility into specific nodes along the path. For example, a customer may only have visibility from end to end, but a service provider can see every hop. ITU-T Y.1731 The Y.1731 specification is a superset of 802.1ag. It provides all the features of E-CFM, plus the following signals: Ethernet locked, Ethernet test, multicast loopback, and alarm indication. Performance management capabilities include frame loss measurement, frame loss delay, and throughput measurement. At this point, Y.1731 is not widely implemented, but with its enhanced capabilities, it is of great interest to service providers. 11
Testing Challenges Each stage of carrier Ethernet deployment has its own testing requirements. It is expected that deployments will start with Q-in-Q and/or MAC-in-MAC. PBT or other transport protocols will be implemented as networks increase in size. Whatever the means, interfaces to the core networks, usually via IP/MPLS, need to be thoroughly exercised and at full line rates. New and legacy devices must be tested for interoperability. Performance testing of network subsystems and entire end-to-end networks, both at and beyond expected capacity, is required to ensure proper forwarding and handling of over capacity. Security over exclusively Ethernet transport is another concern because everyone s traffic travels side by side, separated only as VLAN, MAC-in-MAC, and VPN traffic. Tests must ensure no leakage between those methods. Look for the deployment of more carrier Ethernet services to a fastgrowing customer base, which will require continued innovation in Carrier Ethernet technologies. To do all this, the MEF has defined set of conformance and performance tests. And, since metro Ethernet protocols are used in densely layered combinations tunnels transport other protocols and services, maintenance protocols enable resiliency, billing, and other services each network configuration requires a different testing scenario. Ixia s testing tools not only combine multiple protocols, but enable network emulations thatsimultaneously include Q-in-Q, MAC-in-MAC, and PBB-TE/PBT at line speeds up to 10 Gbps. Ixia s applications test conformance and interoperability, function and performance, and service scalability. For more information about Ixia s suite of test applications, go to http://www.ixiacom.com/ Summary Carrier Ethernet is made up of three main components: the definition of services, network implementation, and service management. While the services defined by the MEF have been widely accepted, the technologies to enable and manage them can vary greatly from one service provider to another because of all the possible combinations available. Because of customer demand, service providers are highly motivated to roll out and enhance their carrier Ethernet networks. According to Infonetics, service providers continue to shift CAPEX spending from legacy TDM to IP/MPLS and Ethernet-based services. And network equipment manufacturers report that carrier Ethernet routers and switches comprise their fastest-growing routing and switching market segment. For the future, look for the deployment of more carrier Ethernet services to a fast-growing customer base, which will require continued innovation in Carrier Ethernet technologies. References Metro Ethernet Forum - http://www.metroeternetforum.org/ IEEE - http://www.ieee802.org/ IETF - http://www.ietf.org/ ITU-T - http://www.itu.int/itu-t/ Ixia - http://www.ixiacom.com/ 12
White Paper Ixia Worldwide Headquarters 26601 Agoura Rd. Calabasas, CA 91302 (Toll Free North America) 1.877.367.4942 (Outside North America) +1.818.871.1800 (Fax) 818.871.1805 www.ixiacom.com Other Ixia Contacts Info: info@ixiacom.com Investors: ir@ixiacom.com Public Relations: pr@ixiacom.com Renewals: renewals@ixiacom.com Sales: sales@ixiacom.com Support: support@ixiacom.com Training: training@ixiacom.com This material is for informational purposes only and subject to change without notice. It describes Ixia's present plans to develop and make available to its customers certain products, features, and functionality. Ixia is only obligated to provide those deliverables specifically included in a written agreement between Ixia and the customer.