Customer White paper. SmartTester. Delivering SLA Activation and Performance Testing. November 2012 Author Luc-Yves Pagal-Vinette

Transcription

1 SmartTester Delivering SLA Activation and Performance Testing November 2012 Author Luc-Yves Pagal-Vinette Customer White paper

2

3 Table of Contents Executive Summary I- RFC-2544 is applicable for WAN and Service Activation II- Introducing SmartTester for improved RFC 2544 Service Testing III- Further applicability to real-life scenarios 1. Real-Time applications and services 2. Differentiated Services validation and testing Conclusion 3

4 Executive Summary For the past few years, Service providers have accelerated their deployments of Carrier-Ehernet solutions due to its low cost-of-ownership, global standardization and its relative simplicity compared to IP-VPN and MPLS-based solutions. To enhance competitiveness, time-to-market and decrease issues during field deployments, CSPs (Communication Service Providers) and network service managers need visibility into the envisaged performance of their Ethernet links and the QoS / QoE (Quality of Service / Quality of Experience) associated. Service qualification is a key differentiator to analyze and report on network performance as defined through three key criteria: throughput, latency and frame loss. The OneAccess implementation of the IETF RFC 2544 SmartTester provides the benefits of service activation and SLA testing at the level of sixteen concurrent service flows in order to test, report and guarantee service quality without compromising on accuracy. 4

5 I- RFC-2544 is applicable for WAN and Service Activation RFC 2544 describes a benchmarking model optimized for testing communication service platforms in laboratory and field environments and is the de-facto methodology for providers and network engineers installing or troubleshooting Ethernet-based active links and the services to run across. In real-life scenarios, four main service activation tests are considered necessary to characterize performance in an environment where CoS and QoS are tested intensively: thoughput, latency, frame loss rate and recovery. Below is a description of a comprehensive set of RFC 2544 tests: Fig.1: RFC 2544 Tests and Applications on Ethernet or IP-based networks Test Throughput Latency Application Testing for maximum speed with no frame loss Testing how long it takes for one frame to go from source to destination across the network Frame Loss Rate Testing the percentage of frames lost during a packet transmission Back-to-Back Recovery Reset A buffering capacity test that measures the maximum number of frames received before a frame is lost Testing the recovery time of a switch after overstressing its capacity Testing time necessary for a switch to start forwarding frames after a hardware or software reset II- Introducing SmartTester for improved RFC 2544 Service Testing The OneAccess Networks SmartTester Service Activation tool is a proprietary Service Activation implementation of RFC 2544 for performance testing of live WAN links regardless of rates over copper, electrical and optical fibre media types. SmartTester is fully available over TDRE (the Total Dynamic Routing Engine, from OneAccess, an embedded service platforms software delivering the full scope of service intelligence. With OneAccess s proprietary implementation, SmartTester improves significantly the accuracy and power of RFC 2544 performance and service activation testing. The four necessary tests are combined with advanced predictability, flexibility, control and automation, which streamline processes and accentuates time saving while simultaneously delivering accurate data: Deployment being the most critical element of customer satisfaction, QoS/SLA policy verification brings a strong sense of service perspective. 5

6 Throughput is the maximum rate at which data can be transported from source to destination with zero lost frames. Nevertheless, highly stressed rate accuracy tests provide unpredictable results therefore require to be combined with additional tests to provide a much granular visibility. Latency is the total time it takes for a frame to traverse a network from source to destination. The context for a relevant latency is while the link is under load. The network needs to be able to deliver latency at specified utilization. It is considered that it is more appropriate to exercise that test under a full throughput load. Test resulting in: A table with optimum value per frame size tested Test resulting in: A statistics log table each 60 seconds including the last latency measurement test Test resulting in: A table with percentage of frame loss at specified rate and packet size per sessions identified by differentiated CoS marking (IP ToS or L2 PCP) A Frame Loss Rate test determines the percentage of frame that left the source but never actually reached its destination while traversing the network. The usual recommendation is to start the Frame Loss Rate at 100%, then reducing gradually to the next rate to test for lost frames. Subsequently, a successful test is defined by two consecutive test results without frame loss. A Recovery test consists in stressing a service platform with abnormal traffic typically at 110% traffic until the targeted system starts to lose frames. Traffic pressure is subsequently reduced to 50%. Therefore, the gap of time between the last dropped frame and the recovery of frames accepted is the system recovery time. A Back-to-Back or burst test assesses the buffering capacity of a service platform. It measures the maximum number of frames transmitted at full speed before frames are dropped. In both IP and Carrier-Ethernet networks, this test is useful to determine the Excess Information Rate (EIR) as defined by providers in Service Level Agreements. Carrier-Ethernet is often used as an SDH and long-haul service replacement and requires SmartTester functions to measure and qualify throughput and bandwidth from an end-to-end perspective. 6

7 The OneAccess implementation of RFC 2544 enables running up to 16 concurrent RFC-2544 testing sessions. As an additional flexibility, all sessions can be started, stopped and initialized individually or all at once. SmartTester can be run on both IP and Ethernet service layers with different QoS information for differentiated and priority services, together with configurable parameters, which enables an additional level of control and flexibility. Additional options include Echo IP, Echo Ethernet and optional values such as IP and Ethernet encapsulation, frame size, time interval between packets, bandwidth, packets per second, duration of test, packet amount for test, stop upon errors, payload amount (bytes or bandwidth use) and other incremental options. Test resulting in: A table with percentage of frame loss at specified rate and packet size III- Further applicability to real-life scenarios 1. Real-Time applications and services Initially not included in RFC-2544, packet jitter is a crucial aspect in delivering real-time applications (VOIP, Video-Conferencing and TelePresence) over a PSN (Packet-Switched Network). The measurement is crucial to ensure QoE (Quality of Experience), as excessive jitter can cause major issues for user experience such as image degradation, lip-sync issues and lack of synchronization between voice, video and data streams. As covered in the latency test section above, frame latency can vary over time; it is precisely that variation in inter-packet arrival time that is generating the packet jitter depicted below. Without Jitter With Jitter With OneAccess s SmartTester, it is possible to ensure agile testing over the networking environment carrying real-time application packets. Indeed, real-time application packets are highly sensitive to delay variation. To facilitate further testing, it SmartTester can introduce additional variables such as: packet delay variation, CoS/QoS differentiated services marking and different bandwidth levels, thus testing the network service infrastructure s ability to cope with strict- SLA voice requirements. Example of testing VOIP services with intra-packet delay With mission-critical applications delivery, OneAccess SmartTester provides a granular visibility of the network infrastructure to cope with packet-delay sensitive variations. 7

8 2. Differentiated Services Validation and Testing A few useful definitions are presented below that can help in understanding the testing concepts for Ethernet services: Layer-2 and Layer-3 Virtual Circuits Virtual Circuits: an association of two or more interfaces that limits the exchange of Service frames between those interfaces. Bandwidth Profile CIR is the Committed Information Rate, i.e. the average rate in bits/s of service frames up to which the network delivers service frames and meets the performance objectives (SLA). EIR is the Excess Information Rate, i.e. the average rate in bits/s of service frames up to which the network may deliver service frames, but without any performance objectives (SLA). Fig.2: RFC 2544 Tests and Applications for Ethernet or IP-based networks Bandwidth (Mbps) 100% of link EIR CR Time (s) RED Traffic: Everything over EIR: traffic discarded ORANGE Traffic: Everything between CIR and EIR: traffic transmitted only if network capacity allows and with no performance guarantee. GREEN traffic: Everything below CIR. All performance metrics are guaranteed by SLA Testing of differentiated services over Ethernet consists of stressing the way that flow priorities are handled and to ensure maximum throughput for rates within the CIR or above it. To test this in a logical fashion, multiple test phases are required: 8

9 Phase 1 Ramp a stream gradually to reach and exceed the CIR (or expected bandwidth for that service/ application). Measure simultaneously latency, delay variation and throughput for rates below and equal to CIR. Measure maximum throughput for rates over CIR. Repeat testing procedure for each service. Result: Verifies that throughput on the expected service values are below / equal or above the CIR values expected. Phase 2 Aggregate all configured/expected services together and generate a global multi-session. Output to measure simultaneously throughput, latency, and packet delay variation on all services and compare against expected results. This procedure needs only to be performed once but over a long period of Result: Verifies how each service influences other ones and if they respectively comply to SLAs. 9

10 Conclusion CSPs (Communication Service Providers) face the continuous challenge of guaranteeing quality of services delivery to their customers. Nowadays, Ethernet services need to be delivered to customers with service assurance and in compressed time frames without compromising the reliability of the services. OneAccess s SmartTester is the right tool to turn-up and troubleshoot new services for operational staff and their customers. SmartTester provides them with a simple, integrated and highly OPEX-efficient approach to delivering service activation processes, measuring service efficiency and providing SLA reporting. It does this in a significantly faster manner than alternative approaches and with a significantly higher level of accuracy. 10

11 ...

12 about us Incorporated in 2001, OneAccess is a leading manufacturer of multi-service routers and carrier Ethernet access devices enabling major telecoms service providers to deliver business-grade managed services profitably. OneAccess supplies routers to over 110 communications service providers including 4 of the top 5 largest telecoms operators in Europe. By using mass customization techniques OneAccess CPE solutions can be precisely tailored to meet the stringent demands of the business managed services market enabling superior performance, management, reliability and services innovation. Smart Enterprise Access Solutions for Service Providers For more information visit OneAccess, Pentagone Plaza, 381 Avenue du Général de Gaulle, Clamart, France Phone: +33 (0) Fax: +33 (0) marketing@oneaccess-net.com