Carrier Ethernet. Operations, Administration & Maintenance. Santanu Dasgupta

Transcription

1 Carrier Ethernet Operations, Administration & Maintenance Santanu Dasgupta 1

2 Acknowledgement Jose Liste Sudarshan Pathalam 2

3 Agenda OAM Overview Ethernet OAM Protocol Overview IEEE 802.3ah Link OAM, also known as EFM IEEE 802.3ag Connectivity Fault Management (CFM) ITU Y.1731 Fault & Performance Management MEF E-LMI Configuration Management Ethernet Service Activation Testing OAM Interworking Sample Fault Management Scenarios Ethernet OAM Deployment Use Cases Summary 3

4 Operations, Administration & Maintenance F Fault Management FCAPS C Configuration Management A Accounting P Performance Management S Security Management OAM Protocols and Mechanisms helps operator to achieve some of the FCAPS functionality OAM capability is one of the key differentiator to make a SP network truly Carrier Grade 4

5 Problem Scope A few possible scenarios Excessive Encoding Errors Equipment CE Cross-connected Service CE Physical link failure UNI port failure Ethernet UNI Access Excessive FCS Errors U-PE A Native Ethernet U-PE B U-PE device failure Physical link failure Uni-directional Link N-PE 1 N-PE 2 SP Network Core MPLS Unexpected Endpoint PW PW failure VFI failure P-Router failure D-LDP session failure N-PE 3 N-PE 4 SONET/SDH LSP Failure MPLS VC failure U-PE D U-PE C Physical link failure C-VLAN to EVC mismatch Ethernet UNI VLAN to VFI xconnect failure Duplex mismatch CE CE AC failure Speed mismatch 5

6 OAM: The Concept Operations, Administration, Maintenance & Provisioning: fault indication performance monitoring security management diagnostic functions configuration & service provisioning OAM covers both Nß à S and Wß à E interfaces Management Plane (NMS / EMS) N ß à S Network Plane (Elements) W ß à E 6

7 Scope of Ethernet OAM Operations, Administration, Maintenance & Provisioning: fault indication performance monitoring security management diagnostic functions configuration & service provisioning OAM covers both Nß à S and Wß à E interfaces Management Plane (NMS / EMS) Primary Focus of Ethernet OAM protocols is on W ß à E interactions (across NEs) N ß à S Network Plane (NEs) W ß à E 7

8 Focus Areas of Ethernet OAM UNI Access Aggregation Edge Core Edge Aggregation Access UNI U-PE PE-AGG N-PE Core Core N-PE PE-AGG U-PE EPL EVPL EP- LAN EVP- LAN EP- TREE EVP- TREE AC EPL AC EVPL CE Services & Network Infrastructure Fault Management Config Management Performance Management Fault Detection Fault Notification Fault Verification Fault Isolation Fault Recovery Service Provisioning Frame Loss Measurement Delay Measurement Delay Variation Measurement Availability Measurement 8

9 Ethernet OAM Building Blocks Performance Management Fault Management Configuration Management Management E-LMI E-LMI 802.1ag / Y.1731 IP SLAs Performance Performance Management Configuration Management Fault Management 802.3ah Performance Management Fault Fault Management Configuration Management IEEE 802.1ag: Connectivity Fault Management (CFM) IEEE 802.3ah: Ethernet Link OAM (EFM OAM) ITU-T Y.1731: OAM functions & mechanisms for Ethernet based networks MEF E-LMI: Ethernet Local Management Interface IP SLA s: Performance Management using CFM and Y.1731 mechanisms 9

10 Carrier Ethernet OAM Protocol Positioning Business Aggregation E-LMI Access Connectivity Fault Management Core Aggregation Y.1731 Performance Management Business MPLS OAM Residential Ethernet MPLS/IP Ethernet Link OAM MPLS/IP Ethernet MPLS/IP Residential UNI NNI NNI NNI UNI E-LMI User to Network Interface (UNI) Link OAM Any point-point link CFM, Y.1731 PM End-to-End UNI to UNI MPLS OAM within MPLS cloud 10

11 IEEE 802.3ah (Clause 57) Link OAM 11

12 Link OAM (IEEE 802.3ah, Clause 57) Provides mechanisms useful for monitoring link operation, such as: Link Monitoring Remote Failure Indication Remote Loopback Control Defines an optional OAM sublayer Intended for single point-to-point IEEE links Uses Slow Protocol 1 frames called OAMPDUs which are never forwarded by MAC clients Standardized: IEEE 802.3ah, clause 57 (now in IEEE ) OSI Model Application Presentation Session Transport Network Data Link Physical LAN CSMA/CD Layers Higher Layers LLC OAM (Optional) MAC Physical Layer (1) No more than 10 frames transmitted in any one-second period 12

13 IEEE 802.3ah Key Functions OAM Discovery Discover OAM support and capabilities per device Link Monitoring Basic error definitions for Ethernet so entities can detect failed and degraded connections Fault Signaling Mechanisms for one entity to signal another that it has detected an error Remote MIB Variable Retrieval Ability to read one or more MIB variables from the remote DTE Remote Loopback Used to troubleshoot networks, allows one station to put the other station into a state whereby all inbound traffic is immediately reflected back onto the link 13

14 IEEE 802.3ah OAM Events Set of events that may impact link operation Critical Link events Link fault fault in the Rx direction of local DTE Dying gasp unrecoverable local failure condition Critical event unspecified critical event Link events Errored Symbol Period Event Errored Frame Event Errored Frame Period Event Errored Frame Seconds Summary Event 14

15 IEEE 802.3ah Remote Loopback Fault localization and intrusive link performance testing Loopback Control OAMPDU is used to control a remote OAM client Traffic sent from master loopback port is loopback by slave port, except Pause and OAMPDU OAM MAC PHY X Returned traffic dropped at Master to prevent MAC moves Slave loops service frames without swapping MAC addresses X OAM MAC Slave drops traffic from LLC on looped interfaces PHY Master OAM Slave OAM 15

16 IEEE 802.1ag: Connectivity Fault Management (CFM) 16

17 CFM Overview Family of protocols that provides capabilities to detect, verify, isolate and report end-to-end Ethernet connectivity faults Employs regular Ethernet frames that travel in-band with the customer traffic Devices that cannot interpret CFM Messages forward them as normal data frames CFM frames are distinguishable by Ether-Type (0x8902) and dmac address (for multicast messages) Standardized by IEEE in 2007 IEEE std ag Cisco Systems, Inc. All rights reserved. Cisco Public 17

18 CFM Overview (Cont.) Key CFM Mechanisms Nested Maintenance Domains (MDs) that break up the responsibilities for network administration of a given end-to-end service Maintenance Associations (MAs) that monitor service instances under a given MD Maintenance Points (MPs) that generate and respond to CFM PDUs Protocols (Continuity Check, Loopback and Linktrace) used for Fault Management activities 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 18

19 CFM Concepts Maintenance Domain (MD) CE CE Operator A Operator B Service Provider Defined by Operational/Contractual Boundaries e.g. / Service Provider / Operator MD may nest and touch, but never intersect Up to eight levels of nesting : MD Level (0..7) The higher the level, the broader its reach MD Name Format: null, MAC address, DNS or string-based 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 19

20 CFM Concepts Maintenance Association (MA) CE Operator A Operator B CE Monitors connectivity of a particular service instance in a given MD (e.g. one service traversing four MDs = four MAs) Defined by a set of Maintenance End Points (MEP) at the edge of a domain Identified by MAID == Short MA Name + MD Name Short MA Name Format: Vlan-ID, VPN-ID, integer or string-based 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 20

21 CFM Concepts Maintenance Point (MP) MEP CE Operator A Operator B CE MEP MEP MEP MEP MEP MEP MEP MEP Maintenance Association End Point (MEP) Define the boundaries of a MD Support the detection of connectivity failures between any pair of MEPs in an MA Associated per MA and identified by a MEPID (1-8191) Can initiate and respond to CFM PDUs 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 21

22 CFM Concepts Maintenance Point (MP) MIP CE Operator A Operator B CE MEP MIP MIP MEP MEP MIP MIP MEP MEP MIP MIP MEP MEP MIP MIP MIP MIP MEP MIP MIP MIP MIP Maintenance Domain Intermediate Point (MIP) Support the discovery of paths among MEPs and location of faults along those paths Can be associated per MD and VLAN/EVC (manually or automatically created) Can add, check and respond to received CFM PDUs 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 22

23 CFM Concepts UP MEP CFM PDUs generated by the MEP are sent towards the Bridge s Relay Function and not via the wire connected to the port where the MEP is configured CFM PDUs to be responded by the MEP are expected to arrive via the Bridge s Relay Function Applicable to switches Port A Bridge Relay Entity Port B CFM PDUs 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 23

24 CFM Concepts DOWN MEP CFM PDUs generated by the MEP are sent via the wire connected to the port where the MEP is configured CFM PDUs to be responded by the MEP are expected to arrive via the wire connected to the port where the MEP is configured Port MEP special Down MEP at level zero (0) used to detect faults at the link level (rather than service) Applicable to routers and switches Port A Bridge / Router Relay Entity Port B CFM PDUs 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 24

25 CFM Concepts MAs and UP/DOWN MEPs Applicability of UP/DOWN MEPs in switches: DOWN MEPs are typically used for MAs spanning a single link UP MEPs are commonly used for MAs with a wider reach (e.g. endto-end, beyond a single link) DOWN MEP to UP MEP Bridge Port Bridge 1 Relay Entity Bridge Port Monitored area Bridge Port Bridge 2 Relay Entity Bridge Port UP MEP to UP MEP Bridge 1 Bridge 2 Bridge Bridge Bridge Bridge Port Relay Entity Port Port Relay Entity Port Monitored area 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 25

26 CFM Protocols There are three (3) protocols defined by CFM Continuity Check Protocol Fault Detection Fault Notification Fault Recovery Loopback Protocol Fault Verification Linktrace Protocol Path Discovery and Fault Isolation 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 26

27 CFM Protocols Continuity Check Protocol CE Operator A Operator B CE Catalogue Catalogue Catalogue and Terminate MEP MIP MIP MEP X Continuity Check Message (CCM) Used for Fault Detection, Notification and Recovery Per-Maintenance Association multicast heart-beat messages Transmitted at a configurable periodic interval by MEPs (3.3ms, 10ms, 100ms, 1s, 10s, 1min, 10min) Uni-directional (no response required) Carries status of port on which MEP is configured Catalogued by MIPs at the same MD-Level, terminated by remote MEPs in the same MA 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 27

28 CFM Protocols Loopback Protocol CE Operator A Operator B CE S D MEP MIP MIP MEP Loopback Message (LBM) 2. Loopback Reply (LBR) Used for Fault Verification Ethernet Ping MEP can transmit a unicast LBM to a MEP or MIP in the same MA MEP can also transmit a multicast LBM (defined by ITU-T Y.1731), where only MEPs in the same MA respond Receiving MP responds by transforming the LBM into a unicast LBR sent back to the originating MEP 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 28

29 CFM Protocols Linktrace Protocol CE Operator A Operator B CE S MEP 2 MIP 4 MIP 6 D MEP X Y 1, 3, 5 Linktrace Message (LTM) 2, 4, 6 Linktrace Reply (LTR) Used for Path Discovery and Fault Isolation Ethernet Traceroute MEP can transmit a multicast message (LTM) in order to discover the MPs and path to a MIP or MEP in the same MA Each MIP along the path and the terminating MP return a unicast LTR to originating MEP 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 29

31 ITU-T Y.1731 Overview ITU-T recommendation that provides mechanisms for user-plane OAM functionality in Ethernet networks. It covers: Fault Management mechanisms Performance Management mechanisms Standardized by ITU-T SG 13 in May 2006 Ethernet Service OAM MPLS Service OAM MPLS-TP Tunnel OAM Latest published version dated Feb after IEEE 802.1ag standardization Frame formats (Multicast Address, Ethertype, and common OAM PDU fields) and base functionality are agreed upon across IEEE 802.1ag and Y.1731 IP / MPLS Tunnel OAM 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 31

32 Covered by IEEE 802.1ag ITU-T Y.1731 Overview OAM Functions for Fault Management Ethernet Continuity Check (ETH-CC) (Y.1731 adds unicast CCM) Ethernet Loopback (ETH-LB) (Y.1731 adds multicast LBM) Ethernet Linktrace (ETH-LT) Ethernet Remote Defect Indication (ETH-RDI) Ethernet Alarm Indication Signal (ETH-AIS) Ethernet Locked Signal (ETH-LCK) In addition: ETH-TEST, ETH-APS, ETH-MCC, ETH-EXP, ETH-VSP OAM Functions for Performance Management Frame Delay Measurement (ETH-DM) Frame Loss Measurement (ETH-LM) Frame Loss Measurement (ETH-SLM) 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 32

33 ETH-AIS Alarm Indication Signal LOC Alarm Suppression Without AIS, a MEP would report Loss of Continuity (LOC) for each of its remote MEPs upon timer expiration With AIS, if a MEP receives AIS from the network, it suppresses LOC alarms from peer MEPs LOC 1 LOC 3 LOC 4 2 Network outage AIS defect 2 mpid = 1 X LOC 1 LOC 2 LOC 4 mpid = 1 AIS AIS X AIS AIS defect LOC 2 LOC 3 LOC 4 3 AIS defect AIS 3 4 WITHOUT AIS Timer-driven LOC notification LOC 1 LOC 2 LOC Cisco Systems, Inc. All rights reserved. Cisco Public 33 4 AIS defect WITH AIS Event-driven AIS notification

34 Alarm Indication Signal Transport Path Failure propagation AIS Defect Northbound Syslogs Equipment Operator A Bridges NMS Link Failure Operator B Bridges Equipment MEP(7) MIP(7) MEP(5) MIP(5) MEP(3) MIP(3) MEP(2) MIP(2) SMEP/link-status AIS 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 34

35 Y.1731 Performance Management Functions Frame Loss Ratio percentage (%) of service frames not delivered / Total number of service frames delivered in T time interval Frame Delay round-trip/one-way delay for a service frame Frame Delay Variation Variation in frame delay between a pair of service frame 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 35

36 Ethernet Performance Management ITU-T Y.1731 Technology Overview Ethernet Delay Measurement ETH-DM Ethernet Loss Measurement ETH-LM Ethernet Synthetic Loss Measurement ETH-SLM One-Way ETH-DM Two-Way ETH-DM Single-Ended ETH-LM Dual-Ended ETH-LM Single-Ended ETH-SLM Scope One-Way delay measurements Specifics Two-Way delay measurements Also One-Way delay when synchronized Unidirectional frame loss measurements Applicable to P2P services only Unidirectional frame loss measurements Applicable to P2P services only Unidirectional frame loss measurements Applicable to P2P and MP services 1DM PDU Synthetic traffic with two (2) timestamps Need for Time-of-Day synchronization DMM / DMR PDUs Synthetic traffic with two (2) mandatory and two (2) optional timestamps LMM / LMR PDUs On-demand operation Based on actual Service Frame Loss Exchange of service frame counters CCM PDUs Proactive operation Based on actual Service Frame Loss Exchange of service frame counters SLM / SLR PDUs Based on statistical sampling Exchange of synthetic frame counters 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 36

37 Y.1731 Frame Delay Measurement Frame Delay calculated based on timestamps applied to synthetic traffic Applicable to point-topoint and multipoint services Two (2) mechanisms defined One-Way ETH-DM Two-Way ETH-DM Ethernet Delay Measurement ETH-DM One-Way ETH-DM Performance Metrics One-Way FD/FDV Time-of-Day (ToD) synchronization required for OW FD Two-Way ETH-DM Performance Metrics Two-Way FD/FDV One-Way FD/FDV when synchronized 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 37

38 ITU-T Y.1731 Overview One-Way ETH-DM Equipment Service Provider Equipment MEP NEs must be synchronized (ToD) for one-way delay MEP 1DM TxTimeStampf 0 1DM t p TxTimeStampf RxTimeStampf 1DM 1DM TxTimeStampf 0 1DM t c TxTimeStampf RxTimeStampf 1DM One-Way Delay One-Way Delay Variation Delay = RxTimeStampf TxTimeStampf Delay Var.[t c ] = Delay[t c ] Delay[t p ] t p Time Previous t c Time Current 1DM One-way Delay Measurement 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 38

39 ITU-T Y.1731 Overview Two-Way ETH-DM Equipment Service Provider Equipment MEP NEs may be synchronized (ToD) for one-way delay MEP Two-Way Delay D M M D M R TxTimeStampf TxTimeStampf RxTimeStampf TxTimeStampb RxTimeStampb (RxTimeStampb TxTimeStampf) (TxTimeStampb RxTimeStampf) DMM DMR One-Way Delay (Forward) RxTimeStampf TxTimeStampf Optional Timestamps TxTimeStampf D RxTimeStampf M 0 0 M TxTimeStampf D RxTimeStampf M TxTimeStampb 0 R One-Way Delay (Backward) RxTimeStampb TxTimeStampb DMM Delay Measurement Message DMR Delay Measurement Reply 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 39

40 Y.1731 Frame Loss Measurement Frame Loss calculated based on actual in-profile service counters Applicable to point-to-point services only (with ETH-LM) Near-End Frame Loss measurement Loss associated with Ingress Data Frame Far-End Frame Loss measurement Loss associated with Egress Data Frame Per-CoS counters maintained per MEP TxFCl in-profile data frames transmitted towards the peer MEP RxFCl in-profile data frames received from the peer MEP Single-Ended LM Ethernet Loss Measurement ETH-LM ETH- Performance Metrics Unidirectional Frame Loss Frame Loss Ratio Availability Dual-Ended ETH-LM 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 40

41 ITU-T Y.1731 Overview Single-Ended (On-demand) ETH-LM Equipment Service Provider Equipment Service Frames MEP MEP Per-CoS Service Frame counters TxFCl RxFCl LMM LMR TxFCf 0 0 TxFCf RxFCf TxFCb t p LMM LMR LMM TxFCf 0 0 TxFCf RxFCf TxFCb LMM LMR Per-CoS Service Frame counters RxFCl TxFCl t c LMR Frame Loss (Far-end) (TxFCf[t c ] TxFCf[t p ]) (RxFCf[t c ] RxFCf[t p ]) Frame Loss (Near-end) (TxFCb[t c ] TxFCb[t p ]) (RxFCl[t c ] RxFCl[t p ]) t p Time Previous t c Time Current LMM Loss Measurement Message LMR Loss Measurement Reply 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 41

42 Synthetic Loss Measurement (ETH-SLM) Y.1731 ETH-LM cannot be used for frame loss in multipoint EVCs A new protocol (SLM) based on synthetic frames Statistical Frame Loss Covers P2P or MP EVCs Implementable by ALL platforms ITU agreed (June 2010) to allocate code points for new PDUs (SLM / SLR) A SLR SLM B ETH-SLM included ITU-T G / Y.1731 (07/2011) C SLM Synthetic Loss Message SLR Synthetic Loss Reply 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 42

43 ITU-T Y.1731 Overview Single-Ended ETH-SLM Equipment Service Provider Equipment Synthetic Frames MEP MEP Per-Test ID Synthetic Frame counters TxFCl RxFCl SLM SLR TxFCf 0 TxFCf TxFCb t p t c SLM SLR SLM SLR TxFCf 0 SLM TxFCf TxFCb SLR Per-Test ID Synthetic Frame counters RxFCl TxFCl RxFCl = TxFCl at Responder Frame Loss (Far-end) Frame Loss (Near-end) (TxFCf[t c ] TxFCf[t p ]) (TxFCb[t c ] TxFCb[t p ]) (TxFCb[t c ] TxFCb[t p ]) (RxFCl[t c ] RxFCl[t p ]) t p Time Previous Start of Measurement Period t c Time Current End of Measurement Period SLM Synthetic Loss Message SLR Synthetic Loss Reply 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 43

45 Ethernet LMI Overview Provides protocol and mechanisms used for: Notification of EVC addition, deletion or status (Active, Not Active, Partially Active) to CE Communication of UNI and EVC attributes to CE (e.g. CE-VLAN to EVC map) CE auto-configuration Notification of Remote UNI count, name and status to CE Asymmetric protocol based on Frame Relay LMI, mainly applicable to the UNI (UNI-C and UNI-N) Specification completed by MEF: org/pdfs/standards/mef16.doc User Network Interface (UNI) UNI-C CE UNI-N E-LMI Metro Ethernet Network 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 45

47 Service Activation Testing (SAT) Issuance of Birth Certificate Validation of Service Configuration Validation of SLA Throughput Latency Loss Jitter Control Plane Signaling Data Plane Loopback Traffic Generation/ Reception Test Methodology 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 47

48 SAT Concepts Bi-directional Test Operation Ethernet Data-Plane Loopback TG Test Traffic EVC Test Initiator Responder Uni-directional Test Operation Start Reply / Request TG Test Traffic EVC RL Test Initiator Stop Reply / Request Responder TG = Traffic Generation function HW-accelarated (Platform-Dependent) RL=Responder Logic 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 48

49 MEF Projects Project Name Highlights Status Service Activation Testing project SAT PDU project Latching loopback project Defines set of tests needed to run before handing off a service to subscriber Leverages ITU-T Y.1564 Defines the test PDUs that can be used to perform the tests defined by the SAT project Defines control protocol to be used for unidirectional tests - to set up the collector and/or responder at the remote end. Defines control protocol to be used for bidirectional tests - to put remote device into latching loopback Approved Project Editor: CableLabs Targeting completion in Q2, 2013 Approved Project Editor: Verizon Targeting completion in Q4, 2013 Approved Project Editor: Adtran Targeting completion in Q1, Cisco Systems, Inc. All rights reserved. Cisco Public 49

50 ITU-T Y.1564 RFC2544 methodology shortcomings (as stated by Y.1564) Not Ethernet service aware Tests run as a single flow at a time Tests are performed sequentially Does not measure Frame Delay Variation Does not verify CIR, CBS, EIR, EBS and CM Y.1564 advantages Tests all KPIs at same time Test to CIR to verify SLA performance (FD, FDV, FLR) Tests to EIR limit and just beyond to verify policing behavior (no SLA performance expected for yellow frames (above CIR and below CIR +EIR) Device Centric Sequential Tests Missing KPI Service centric All KPI measured in 1 test Includes Jitter, CIR/ EIR,CBS/EBS Faster Execution 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 50

52 What Is OAM Interworking? Operator A Operator B Edge Bridge Strict OAM layering should be honored: messages should not cross layers OAM Messages should not leak outside domain boundaries within a layer Bridge switch switch Bridge Router Router Router Bridge Bridge ATM Switch ATM Switch Edge Router Bridge Interworking is event translations & not necessarily 1:1 message mapping Interworking may be inter-layer and intra-layer Ethernet over SONET Network OAM Ethernet over MPLS Service OAM Event Translation ATM interdomains RFC1483 intra-layer 10G Ethernet Network OAM Server Layers feed events into Client Layers Transport OAM Transport OAM Transport OAM Transport OAM 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 52

54 Ethernet OAM Deployment Use Case #1 Ethernet L2 VPN SLA Monitoring Ethernet Service Provider L2 VPN L3 CE EVC EVC L3 CE NID / Access / Demarcation Device L2 VPN Shadow Router NID Optional L2 VPN L3 CE EVC = Ethernet Virtual Circuit NID = Network Interface Device = CFM Maintenance Association End Point (MEP) = IP-based probe 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 54

55 Ethernet OAM Deployment Use Case #2 Mobile Backhaul Service Monitoring Ethernet Service Provider Distribution Router enodeb (4G) NodeB (3G) BTS (2G) Cell Site Router EVC Demarc Demarc EVC Demarc SAE GW RNC BSC 4G 3G 2G Probe (Optional) Distribution Router Mobile Operator Cell Site(s) Wireline Provider / Mobile Backhaul Operator Mobile Operator MTSO enode B = Enhanced Node B SAE GW = System Arch Evolution GW RNC = Radio Network Controller MTSO = Mobile Telephone Switching Office BTS = Base Transceiver Station BSC = Base Station Controller EVC = Ethernet Virtual Circuit NID = Network Interface De`vice = CFM Maintenance Association End Point (MEP) 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 55

56 Ethernet OAM Deployment Use Case #3 Monitoring of Ethernet Access to L3VPN Ethernet Service Provider VRF Internet L3 VPN Managed L3 CE G.SHDSL NID DSLAM EVC EVC Aggregator Demarc Device L3 PE Shadow Router Optional IP / MPLS Core Managed L3 CE IP Department Third Party Ethernet Service Provider Or Transport / Aggregation Department IP Department EVC = Ethernet Virtual Circuit NID = Network Interface Device = CFM Maintenance Association End Point (MEP) = IP-based probe 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 56

57 Ethernet OAM Deployment Use Case #4 Monitoring of Broadband & Wholesale Ethernet Access Service Residential Subscribers Ethernet Service Provider VRF Internet DSLAM PON OLT Aggregator EVC Aggregator BNG/ BRAS SP Retail ISP EVC (Bitstream Wholesale) Internet Wholesale SP subscribers Broadband Department Third Party Ethernet Service Provider Or Transport / Aggregation Department Retail ISP Broadband Department / Wholesale SP EVC = Ethernet Virtual Circuit NID = Network Interface Device = CFM Maintenance Association End Point (MEP) 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 57

58 Ethernet OAM Deployment Use Case #5 Monitoring of Ethernet Service over DOCSIS Cable L2 VPN Cable MSO (HFC) Ethernet Service Provider EVC Cable MSO (HFC) NID / Demarcation L2 VPN NID / Demarcation L2 VPN EVC = Ethernet Virtual Circuit NID = Network Interface Device = CFM Maintenance Association End Point (MEP) = IP-based probe 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 58

60 Ethernet OAM Deployment Use Case #7 Fault Management - Fault detection, Loop detection, Node monitoring, LAG port monitoring, reachability check. Ethernet Service Provider NID / Demarcation Ethernet Transport Ethernet Or MPLS Core Ethernet Transport NID / Demarcation NID / Demarcation EVC = Ethernet Virtual Circuit NID = Network Interface Device = CFM Maintenance Association End Point (MEP) = IP-based probe Link OAM Sessions 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 60

61 Deployment Use Case #8 Service Activation and Throughput Test with Loopback Application: Service Turn-Up (out-of-service) Throughput Test - POP-Local CPE (shown) or POP-Remote CPE site Ethernet Service Provider L3 Provider POP Loopback EVC X,Y A,B Test Traffic EVC ip = Y mac = C Central Test Head VRF Managed CPE ip = X mac = A A,B X,Y ip = W mac = B L3 PE L3 Loopback Ethernet Data-plane Loopback Hardware-accelerated (no CPU intervention) IPv4/IPv6-aware Loopback Configurable source IPv4/IPv6 of packets to loop IP address swap (implicit MAC swap) 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 61

63 High Level Topology E-LMI Cisco 3800 L3 VPN A Catalyst 3750-ME L2VPN P2P / MP (EoMPLS / VPLS) MPLS Aggregation L3 VPN B MPLS Core Ethernet access to MPLS VPN (EoMPLS) CFM IEEE 802.1ag Cisco 1800 Link OAM IEEE 802.3ah Cisco 7600 Access Cisco ME-3400 Cisco Cisco Systems, Inc. All rights reserved. Cisco Public 63

64 Fault Management Scenarios Number Description 1 End-to-End Service/Path Verification Fault Verification/Isolation 2 Using E-OAM for Ethernet Access to L3VPN 3 E2E CPE Fault Notification & CPE Corrective Action using Ethernet OAM Interworking 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 64

65 Operational Scenario 1 Problem Statement Fault Verification and Fault Isolation of ethernet connectivity issues Problem Solution IEEE 802.1ag (CFM) Ping and Traceroute utilities for reactive troubleshooting of service connectivity issues Service Provider Ethernet Access MPLS Aggregation Ethernet Access CE 11 upe 11 PE-Agg npe 11 npe 31 upe 31 CE 31 Domain Service Provider Domain Operator Domain 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 65

66 Operational Scenario 1 (cont.) Problem Statement Fault Verification and Fault Isolation of ethernet connectivity issues Problem Solution IEEE 802.1ag (CFM) Ping and Traceroute utilities for reactive troubleshooting of service connectivity issues upe11 Service Provider upe31 Ethernet upe11#ping ethernet MPLS mpid 3100 Core level 4 vlan 100 Ethernet Access Access Type escape sequence to abort. CE 11 upe 11 PE-Agg Sending npe 5 11 Ethernet CFM loopback messages, npe timeout 31 is 2 upe seconds: 31 CE 31!!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms Domain Service Provider Domain Operator Domain 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 66

67 Operational Scenario 1 (cont.) Problem Statement Fault Verification and Fault Isolation of ethernet connectivity issues Problem Solution IEEE 802.1ag (CFM) Ping and Traceroute utilities for reactive troubleshooting of service connectivity issues upe11 Service Provider npe11 npe31 upe31 upe11#traceroute ethernet Ethernet MPLS Core c.3d00 level 4 Ethernet vlan 100 Type escape sequence to abort. TTL 255. Per-Hop Timeout is 10 seconds Access Access Tracing the route to c.3d00 on Domain PROVIDER_DOMAIN, Level 4, vlan 100 CE 11 Traceroute sent via GigabitEthernet0/16 upe 11 PE-Agg npe 11 npe 31 upe 31 CE MAC Ingress Ingress Action Relay Domain Action Hops Host Forwarded Egress Egress Action Next Hop Service Provider B 1 npe f21.cec5 Gi3/1 IngOk RlyCCDB Domain Forwarded B 2 npe RlyFDB Forwarded Gi3/1 EgrOK upe31 Operator Domain! 3 upe c.3d00 Gi1/1/1 IngOk RlyNone Not Forwarded 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 67

68 Operational Scenario 1 (cont.) Problem Statement Fault Verification and Fault Isolation of ethernet connectivity issues Problem Solution IEEE 802.1ag (CFM) Ping and Traceroute utilities for reactive troubleshooting of service connectivity issues upe11 Service Provider npe11 npe31 X npe31(config)#int gig3/1 npe31(config-if)#shutdown upe31 Proactive Fault Notification Ethernet Access MPLS Core Ethernet Access Jan 26 03:14:10.608: %ETHER_SERVICE-6-EVC_STATUS_CHANGED: status of EVC_P2P_100 changed to InActive upe11#traceroute ethernet c.3d00 level 4 vlan 100 CE 11 upe 11 PE-Agg Type npe escape 11 sequence to abort. TTL 255. Per-Hop npe Timeout 31 is 10 upe seconds 31 CE 31 Tracing the route to c.3d00 on Domain PROVIDER_DOMAIN, Level 4, vlan 100 Traceroute sent via GigabitEthernet0/16 Domain Reactive Fault Isolation MAC Ingress Ingress Action Relay Action Hops Host Forwarded Egress Egress Action Next Hop Domain B 1 npe f21.cec5 Gi3/1 IngOk RlyCCDB Forwarded B 2 npe RlyCCDB Not Forwarded Gi3/1 EgrDown 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 68 Service Provider Operator Domain

69 Operational Scenario 2 Problem Statement Troubleshooting Ethernet access connectivity problems by L3VPN PE Problem Solution IEEE 802.1ag CFM with Outward-facing / Down MEPs at L3VPN PE Service Provider Ethernet Access MPLS Aggregation CE 11 upe 11 PE-Agg npe 11 npe 21 L3VPN PE Domain Service Provider Domain Operator Domain 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 69

70 Operational Scenario 2 (cont.) Problem Statement Troubleshooting Ethernet access connectivity problems by L3VPN PE Problem Solution IEEE 802.1ag CFM with Outward-facing / Down MEPs at L3VPN PE L3VPN-PE#show running-config begin GigabitEthernet3/0/0 interface GigabitEthernet3/0/0 Service Provider description L3VPN PE to npe21 gig3/3 ethernet cfm mep level 4 outward domain PROVIDER_DOMAIN mpid 2450 vlan 450 ethernet cfm mep level 4 outward domain PROVIDER_DOMAIN mpid 2350 vlan 350! interface GigabitEthernet3/0/0.350 description To CE31 encapsulation dot1q 350 ip vrf forwarding BLUE Ethernet MPLS Core ip address Access! interface GigabitEthernet3/0/0.450 description CE 11 To upe CE21 11 PE-Agg npe 11 encapsulation dot1q 450 ip vrf forwarding RED ip address L3VPN-PE#show ethernet cfm maintenance-points remote Can only Ping/Traceroute to remote MEPs marked with * npe 21 Ethernet Access upe 21 L3VPN PE Domain Service Provider Domain Operator Domain MPID Level Mac Address Vlan PortState InGressPort Age(sec) Service ID 3350* c.3d UP Gi3/0/ customer_350_provider 2451* c.0b UP Gi3/0/ customer_450_provider 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 70

71 Operational Scenario 3 Problem Statement End to End Fault Notification of service status to Equipment Problem Solution IEEE 802.3ah to IEEE 802.1ag (CFM) Inter-Working (IW) for propagation of EVC status and CFM to E-LMI IW for notification to CE Service Provider Ethernet Access MPLS Aggregation Ethernet Access CE 11 upe 11 PE-Agg npe 11 npe 31 upe 31 CE 31 Domain Service Provider Domain Operator Domain 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 71

72 Operational Scenario 3 (cont.) Problem Statement End to End Fault Notification of service status to Equipment Problem Solution IEEE 802.3ah to IEEE 802.1ag (CFM) Inter-Working (IW) for propagation of EVC status and CFM to E-LMI IW for notification to CE 4 3 Service Provider 2 1 E-LMI Link OAM Ethernet Access CFM MPLS Core Ethernet Access E-LMI Link OAM CE 11 upe 11 PE-Agg npe 11 npe 31 upe 31 CE 31 Domain Service Provider Domain Operator Domain 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 72

73 Operational Scenario 3 (cont.) Problem Statement End to End Fault Notification of service status to Equipment Problem Solution IEEE 802.3ah to IEEE 802.1ag (CFM) Inter-Working (IW) for propagation of EVC status and CFM to E-LMI IW for notification to CE 4 3 Service Provider 2 X 1 Ethernet Access MPLS Core Ethernet Access CE 11 upe 11 PE-Agg CE31(config)#int npe 11 fast 0/0 npe 31 upe 31 CE 31 CE31(config-if)#shutdown Domain Jan 26 03:40:08.176: %ETHERNET_OAM-6-EXIT_SESSION: The client on interface Fa0/0 has left the OAM session. Service Provider Jan 26 03:40:10.180: %LINK-5-CHANGED: Interface FastEthernet0/0, changed Domain state to administratively down Jan 26 03:40:11.180: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to down Operator Domain 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 73

74 Operational Scenario 3 (cont.) Problem Statement End to End Fault Notification of service status to Equipment Problem Solution IEEE 802.3ah to IEEE 802.1ag (CFM) Inter-Working (IW) for propagation of EVC status and CFM to E-LMI IW for notification to CE 4 3 Service Provider 2 1 Jan 26 03:40:08.176: %ETHERNET_OAM-6-RFI: The client on interface Fa1/0/1 has received a remote failure Indication from its remote peer (failure reason = remote client administratively turned off) Ethernet MPLS Core Ethernet Jan 26 03:40:08.184: %ETHER_SERVICE-6-EVC_STATUS_CHANGED: Access status of EVC_P2P_100 Access changed to InActive Jan 26 03:40:09.191: %ETHERNET_OAM-6-EXIT_SESSION: The client on interface Fa1/0/1 has left the OAM session. CE 11 upe 11 PE-Agg npe 11 npe 31 upe 31 CE 31 upe31#show ethernet service evc Identifier Type Act-UNI-cnt Status Domain EVC_P2P_100 P-P 1 InActive upe31#show ethernet lmi evc UNI Id: CE31_UNI St Evc Id CE-VLAN ?I EVC_P2P_ Key: St=Status, A=Active, P=Partially Active, I=Inactive, *=Default EVC,?=Link Down Service Provider Domain Operator Domain 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 74

75 Operational Scenario 3 (cont.) Problem Statement End to End Fault Notification of service status to Equipment Problem Solution IEEE 802.3ah to IEEE 802.1ag (CFM) Inter-Working (IW) for propagation of EVC status and CFM to E-LMI IW for notification to CE 4 3 Service Provider 2 1 Jan 26 03:40:36.093: %ETHER_SERVICE-6-EVC_STATUS_CHANGED: status of EVC_P2P_100 changed to InActive upe11#show ethernet Ethernet cfm maintenance-point remote MPLS Core Ethernet MPID Level Mac Address Access Vlan PortState InGressPort Age(sec) Service Access ID c.3d DOWN Gi0/16 21 customer_100_provider CE 11 upe 11 PE-Agg npe 11 npe 31 upe 31 CE 31 upe11#show ethernet service evc Identifier Type Act-UNI-cnt Status EVC_P2P_100 P-P 1 InActive Domain upe11#show ethernet lmi evc UNI Id: CE11_UNI St Evc Id CE-VLAN I EVC_P2P_ Key: St=Status, A=Active, P=Partially Active, I=Inactive, *=Default EVC,?=Link Down Service Provider Domain Operator Domain 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 75

76 Operational Scenario 3 (cont.) Problem Statement End to End Fault Notification of service status to Equipment Problem Solution IEEE 802.3ah to IEEE 802.1ag (CFM) Inter-Working (IW) for propagation of EVC status and CFM to E-LMI IW for notification to CE 4 3 Service Provider 2 1 Ethernet MPLS Core Ethernet Access Access CE11#show ethernet lmi evc UNI Id: CE11_UNI CE 11 upe 11 PE-Agg npe 11 npe 31 upe 31 CE 31 St Evc Id CE-VLAN I EVC_P2P_ Domain Proactive E2E Fault Notification Proactive CPE Action Key: St=Status, A=Active, P=Partially Active, I=Inactive, *=Default EVC,?=Link Service Down Provider Domain CE11#show ip interface brief Interface IP-Address OK? Method Status Protocol <snip> Operator Domain GigabitEthernet0/ YES manual down down 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 76

77 Summary You can perform fault, performance and configuration management with E-OAM for Ethernet based Services There are multiple suites of protocols from various standard bodies (& vendors) that work in different bucket And they do interwork to give you a comprehensive troubleshooting and maintenance platform Cisco Systems, Inc. All rights reserved. Cisco Public 77

79 Acronyms Acronym AIS CCM CCMDB CE CFM EFM E-LMI E-OAM EVC IEEE ITU LBM LBR LMI LTM LTR MA MAID MD Alarm Indication Signal Continuity Check Message CCM Data Base (see CCM) Edge Connectivity Fault Management Ethernet in the First Mile Ethernet LMI (see LMI) Ethernet OAM (see OAM) Ethernet Virtual Connection Institute of Electrical and Electronics Engineers International Telecommunication Union Loopback Message Loopback Reply Local Management Interface Linktrace Message Linktrace Reply Maintenance Association MA Identifier (see MA) Maintenance Domain Acronym MEF MEN MEP MEPID MHF MIB MIP MP OAM PDU PE RDI RFI TLV UNI UNI-C UNI-N VID VLAN Metro Ethernet Forum Metro Ethernet Network Maintenance Association End Point MEP Identifier (see MEP) MIP Half Function (see MIP) Management Information Base Maintenance Domain Intermediate Point Maintenance Point Operations, Administration and Maintenance Protocol Data Unit Provide Edge Remote Defect Indicator Remote Failure Indicator Type, Length, Value User to Network Interface side of UNI (see UNI) Network side of UNI (see UNI) VLAN Identifier Virtual LAN 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 79