1 Whitepaper Quality of Service Testing in the VoIP Environment
2 Carrying voice traffic over the Internet rather than the traditional public telephone network has revolutionized communications. Initially, voice over IP (VoIP) simply promised to dramatically lower long distance charges. Today, because of the broad acceptance and implementation of SIP, it is significantly more interesting. Along with voice traffic, a SIP-based IP network can simultaneously handle , video, chat, text, and more. It enables smooth integration with both contact center systems (IVR, CTI, routing, etc.) and enterprise applications (CRM, ERP, knowledge bases, skills tables, etc.). Additionally, IP communications has given birth to a new piece of information, presence, which tracks each user s availability status and how best to contact them at the moment (text, phone, , etc.). Unified Communications (UC) has arisen from the unification of these elements on a SIP network. UC is a broad term applied to a wide range of new solutions enabled by the integration of all communications, data, presence, and more over a SIPbased IP network. It is an evolving solution set with applications that both optimize the contact center and improve enterprise communications and processes. Many organizations are leveraging UC to: Enhance the customer experience with multi-channel contact centers that let customers choose how they want to interact with an organization s service team Speed time to market with new collaboration tools that allow colleagues worldwide to interact via video, phone, or chat and seamlessly switch between devices as needed Eliminate lag time with notify and respond solutions that use presence to contact decision makers via their preferred communication device of the moment for approvals Reduce churn with a first contact resolution policy that uses skills tables, CRM, and presence to easily incorporate experts into the customer service process Assuring Voice Quality No matter how innovative UC solutions become, if voice quality suffers, no one will use them. Due to its real-time nature, an effective voice conversation requires a high level of continuity. That continuity can be negatively impacted by large numbers of packets (representing other types of data) competing with voice packets for network bandwidth, a situation that never existed in the circuit-switched world. The equipment responsible for processing voice for transport over an IP network must be able to retain all the nuances, inflections, and pauses that comprise effective human communication. This is not always an easy task, especially as network demand increases and new applications are continuously introduced. Assuring voice quality in a VoIP environment is simply a matter of testing. First, establish baseline performance. Then analyze the impact of each successive application on voice quality. To accomplish this step, you must be able to test, measure, and evaluate the performance of the various elements needed to create a VoIP transmission. This paper will identify those elements and suggest some strategies for testing that can help ensure the level of quality required to make VoIP a viable service offering. This information is critical for any manufacturer, system integrator, service provider, or enterprise for whom guaranteeing solid voice quality performance is a critical issue. VoIP Call Elements A VoIP call consists of a combination of any of the following elements: endpoints, session boarder controllers (SBC), gateways, call servers, proxy servers, a packet-switched network, and sometimes also a circuit-switched network. Which of these elements are present in a particular call depends on what types of endpoints are being used and what type of call is being made. An endpoint in a VoIP scenario can be an IP telephone (a PC with a softphone installed or an IP phone itself), a mobile phone, or a traditional PSTN or POTS line. In a teleconference situation, all types of endpoints will likely be participating, as well as media servers. Additionally, an important element of a VoIP call is the packet-switched network itself - the cloud that provides the data transport between the other elements. The network, consisting of various physical media, network protocols, and routers and switches controlling the flow of traffic, is the most problematic of the connection elements. While the PSTN interface is an important component of a VoIP call, and should be tested, it does not generally impact voice quality and is not discussed here.
3 What Needs to be Tested, Measured, and Evaluated In the previous section, we identified the elements that comprise a VoIP transmission. This section will discuss the potential problems these elements can introduce, usually when trying to perform under heavy traffic demands: Connection Failure: Call not completed Latency: Delay for packet delivery Jitter: Variations in delay of packet delivery Possible Elements in a VoIP Call Packet loss: Too much traffic in the network causes the network to drop packets Burstiness of Loss and Jitter: Loss and discards (due to jitter) tend to occur in bursts Connection Failure The endpoint applications and devices discussed above need to be able to place and receive calls, so this capability needs to be verified. A gateway needs to be able to receive and send circuit-switched traffic on one side and packet-switched traffic on the other, and this basic functionality needs to be verified as well. Latency Voice signals need to be processed for transport over a packet-switched network. The necessary compression and packetization (and the reverse of these processes) is completed either by the intelligent endpoints or a gateway. Execution of these functions requires a small amount of time, which can vary depending on the architecture of each device (DSPs, compression algorithms, distributed signaling, and media) and the amount of traffic to be processed. This processing time introduces delay, which is called latency. The human ear, being a subjective evaluator, can tolerate some latency, usually up to around 250ms, before perceiving a drop in the quality of a connection. So, knowing how much latency an endpoint, network, or gateway introduces, especially when traffic load is high, is important to test in order to ensure the 250ms threshold is not exceeded. As it happens, the major portion of the delays are introduced after the packets leave the endpoint or gateway. Depending on how busy each successive router in the network is, it can introduce another few milliseconds or more into the cumulative latency. Outside of a carefully managed network, there is no control over the number of routerto-router legs or hops a packet has to take. Therefore, monitoring the total end-to-end latency that packets are experiencing is necessary to maintain a good quality VoIP transmission. Jitter Not only is it impossible to predict or control (using current networks) how many hops packets from a VoIP call will traverse, packets from the same call can be assigned different routes, with varying numbers of hops and different traffic volumes along the way. As a result, packets from the same conversation can experience different amounts of delay on their way to their destination. These variable delays produce a condition called jitter, where packets arrive at their destination at different intervals. Most gateways have buffers to collect packets and return acceptable continuity to the data, and these must be suitably tuned so that the process itself does not introduce excessive delay. The human ear can tolerate some jitter, usually up to around 75ms, before perceiving an unacceptable drop in quality. Therefore, another area of testing involves monitoring jitter to make sure it is being dealt with effectively. Dropped Packets When a router becomes overloaded with traffic, it may intentionally drop packets to relieve the congestion. Routers, SBCs, or gateways may often drop packets when the packets in question arrive out of order with an excessive amount of delay. Too much jitter can also result in overflow of the jitter buffers and thereby loss of packets. With traditional data traffic, for which these networks are optimized, there are error-checking methods
4 built into the protocols to address these situations and maintain data integrity. These methods require some overhead (which is not conducive to real-time traffic) and were not implemented for voice transport. Again, a certain number of missing packets (generally between 1% and 3%, depending on the data represented) can be forgiven by the human ear. Beyond this guideline, the call quality can degrade to unacceptable levels, so it is important to monitor and test for dropped packets. How to Test, Measure, and Evaluate We have identified several conditions which, if they occur, can negatively impact a user s perception of the quality of the VoIP transmission. The failure of a call to connect is an obvious and easily measured call control problem, but how the other conditions affect voice quality is more difficult to quantify. How an audio signal is perceived by humans is very subjective. It is therefore important to closely simulate real world conditions so that testing is done on what humans are actually hearing. Best practice recommends a methodical, end-to-end approach to IP communications testing and monitoring first validating foundational elements (carrier to IP network) and each technology layer (IVR, CTI, CRM, presence, chat, etc.) successively. That means automating actual calls, in a controlled manner, and measuring the result at every stage. All of the elements that should be in a complete test plan are listed and defined below. Baseline Testing The baseline test is a low volume automated functional test. It essentially assures that all the IP network elements are working as expected. If this test is not performed, issues that come out later may be difficult to link to specific functions or stress under load. DTMF Testing Because users may be using VoIP services to access systems that require DTMF inputs (IVRs, for example), and because DTMF tones are handled differently than speech when processed for IP transport, there should be a test dedicated specifically to checking their integrity. Several DTMF tones should be played sequentially into the SUT using many telephony interface channels at once while their integrity is verified at the receiving end. Voice Quality Testing Voice quality is the ultimate measure of a VoIP solution and there are two approaches to evaluating speech quality in test situations: human evaluation vs. automated analysis of speech clips. In some ways, voice quality is subjective a voice call that sounds acceptable to some may not sound acceptable to others. To obtain a consensus, a group of listeners would have to rate the quality of the call on a scale from 1 to 5 as defined by the ITU (International Telecommunication Union). These scores are averaged and reported as the VoIP solutions Mean Opinion Score (MOS) rating. This method is not particularly viable. It is hard to get a room full of people to listen to the number of clips needed to stress a system and continue to listen to them as improvements are made. For automated test situations, the ITU created the Perceptual Evaluation of Speech Quality (PESQ) algorithm which compares the transmitted and received waveform from the speech clip and produces a perceptually weighted objective score (PESQ score). Like MOS, the PESQ score is also a measurement of perceived voice quality on the familiar voice quality scale from zero to five. When used with narrow band codecs, as virtually all IP telephony is today, the PESQ score is a very close approximation of the Absolute Category Rating MOS score that a large listening panel would determine. Automated Load Testing This test steadily increases the call volume, thereby identifying at what level of stress a problem might occur. In this test cycle, it is easy to spot the level where calls are being dropped or blocked by network issues such as insufficient bandwidth, improper provisioning, incorrect QoS settings, etc. A heavy traffic load is the primary contributor to system and network delay, jitter, and dropped packets. Therefore, an effective test system should be measuring these manifestations of performance degradation while the VoIP system is dealing with telephony load. In order to detect these conditions, the test system must be capable of sniffing packets on the network with audio content, and understand routing and control information embedded in the packet. For instance, out of place or
5 missing sequence numbers would indicate a level of jitter or that packets were missing. By time-stamping call events as they are generated and comparing the stamps to the synchronized clock upon receipt, endto-end latency can be measured. By steadily increasing the call volume beyond expected peak levels, an organization can understand the conditions under which the network will fail. Any company that is serious about voice quality should have these capabilities in their plans. Ongoing Monitoring The key for long term service and application quality and reliability is end-to-end monitoring, performance management, and accurate and meaningful reporting. The goal is to identify issues long before customers or end users find them is essential for superior quality of experience and minimized customer churn. Of equal importance is the use of performance management systems to track applications and systems to highlight where issues may occur in the future based on usage patterns, response or queueing timing, and IVR performance. Ideally, the monitoring platform shows all the statistics in a dashboard and raises alerts where parameters are out of band or trending in the wrong direction. Assuring Voice Quality for Multi- Channel Unified Communications Assuring voice quality in a multi-channel environment is more complicated but certainly feasible. Once you have baseline performance data for your network, you can then test each layer successively to determine its impact on voice quality. At the same time, the assessment should include an analysis of security vulnerabilities at each step. Best-practice methodologies suggest covering the following elements: Multi-service: voice, video, web, chat, , file-sharing/transfer Devices: Session Control and Management Servers, application servers, media servers, PBXs, SBCs, gateways, and SIP-based endpoints Contact Center Solutions: IVR, CTI, routing Presence Enterprise: CRM, ERP, employee directory, skills tables Multi-vendor interoperability, collaboration Conclusion The pace of innovation today is astounding. VoIP has the power to transform how companies do business and offers new ways to improve customer service, boost productivity, respond to opportunity, and lower costs. One thing is certain; the multi-channel, unified communications environment is becoming more complex. It is an evolving mix of voice, video, social media, presence, contact, and data applications all of which can come from different vendors. The challenge is to ensure the quality of each and every user experience. Poor voice quality in the contact center can result in lost sales, customer churn and tarnished brands. In a complex, evolving technical environment, the key to preempting issues is testing. A strong commitment to VoIP testing in pre-deployment lowers implementation timelines and eliminates costly problems that arise during production. Continuous monitoring assures operational performance and enables you to correct issues before they affect the customer and end-user experience. New ideas and technologies are coming fast and furiously. With the right VoIP testing solution in place, you can make informed decisions on how to best utilize new technologies today and for the future. For a complete list of offices worldwide, or to find an authorized distributor in your area, please visit: /contactus Empirix. All rights reserved. All descriptions, specifications and prices are intended for general information only and are subject to change without notice. Some mentioned features are optional. All names, products, services, trademarks are used for identification purposes only and are the property of their respective organizations. Empirix and the star symbol design are trademarks of Empirix, Inc., Billerica, MA ENT:WP:QOSTITVE:0113