Key findings and conclusions: Cisco Unified Border Element Enterprise Edition, enterprise class Session Border Controller, handles 16,000 concurrent SIP-to-SIP calls at 150 calls per second Lab Testing Summary Report February 2011 Report SR111215 Product Category: Enterprise-Class Session Border Controller Vendor Tested: In-box High Availability feature supports active/standby failover of Route Processor and Embedded Services Processor with no dropped calls Transcodes over 9,900 SIP-to-SIP calls from G.711ulaw to G.711alaw Performs In Service Software Upgrade (ISSU) while processing active calls C isco engaged Miercom to evaluate the call handling capabilities of the Cisco Unified Border Element Enterprise Edition, abbreviated as CUBE (Ent), under specific adverse use-case scenarios. The CUBE (Ent) is a highly scalable enterprise-class Session Border Controller for VoIP networks, peering connections, international gateways, and enterprise access. Since Cube (Ent) is integrated into Cisco s router platforms, it can function as either a standalone or a router integrated session border controller. The large scale redundancy and flexibility of the CUBE (Ent) make it an ideal candidate for unified communication systems in an enterprise scenario, allowing smooth transitions between isolated VoIP and TDM systems. Cisco ASR 1000 Series Aggregation Services Routers or ASR1K Routers come in 1RU, 2RU, 4RU or 6RU configurations in the ASR Figure 1: Cisco Unified Border Element (CUBE) Enterprise- Class SBC - Basic Call Scale Test Products Tested: Cisco Unified Border Element Enterprise Edition In basic scalability tests with three iterations of 16,000 calls, no calls were dropped. Maximum CPU utilization was 31% during ramp-up of calls. This performance level was maintained during an overnight test, where over two million successful calls were made.
1001, 1002, 1004, and 1006 models respectively. Cube (Ent) achieves upgradeable media processing performance levels of 2.5, 5, 10, 20 or 40 Gbps using the embedded Cisco QuantumFlow processor for media processing. For data connectivity, the ASR supports multiple modular data interfaces from T1/E1 to 10 Gigabit Ethernet connectivity. The tested version of the Cube (Ent) on the ASR1K routers is based on the ASR IOS XE software version 3.4.1 software. Cube (Ent) allows enterprises to connect isolated voice and unified communications networks directly over an IP connection, avoiding traditional TDM based public switched telephone networks (PSTNs). The resulting end-to-end IP can support a variety of network functions, including: SIP trunking and unified communications application interconnects, enhanced quality of service (QoS), increased throughput, and reduced network complexity. Miercom verified that CUBE (Ent) on the ASR 1006 handled 16,000 concurrent SIP-to-SIP calls with no difficulty whatsoever. Calls were ramped up at a pace of 150 cps, and then held steady for 7 minutes, and disconnected at a rate of 150 cps. No calls were interrupted or prematurely dropped. The intercall interval, the waiting period between termination and initiation of calls, was 3 minutes. See Figure 1 on page 1. Additionally, we verified that the CUBE software, running on a Cisco Integrated Services Router Generation 2, the Cisco ISRG2 specifically the 3945E, was capable of 2,500 concurrent SIP-to-SIP calls at a rate of 20 cps. We allowed the repetition of the ASR 1006 tests to run overnight (ramp up to 16,000, hold, and ramp down), totaling over two million connected calls. No premature disconnections or unexpected behavior occurred. High Availability CUBE (Ent) supports several high availability options. There are software redundancies as well as hardware redundancies. The software redundancy creates two virtual threads on each natural CPU-core and reserves one for standby. Constant synchronization occurs between threads ensuring that the failover is seamless. There was no recordable downtime and no detectable audio faults. Hardware redundancy has a few forms. In the ASR1006, there are two boards for each main system processer, with the system requiring only one of each, Route Processor (RP) and Embedded Services Processor (ESP). If one board is pulled, the other RP or ESP immediately takes over without any packet loss. All active calls remained up. There was no impact on the audio observed on all sampled active calls. In addition to board redundancy, there are two different box-to-box redundancies. The first is a sophisticated form of redundancy with keepalive synchronization, using signaling to ensure the link between devices is not broken. The second form of box-to-box redundancy uses loadbalanced call routing. Both the box-to-box redundancy with keepalive signaling and the loadbalanced call routing worked flawlessly and no problems were detected. 16,000 simulated SIP calls, originating from a NavTel SIP call generator, were used for the in-box High Availability (HA) testing on the ASR 1006. 12,000 simulated SIP calls were used for box-tobox HA using a pair of ASR 1004s. All calls were loaded and released at a rate of 150 cps with a 7 minute steady state hold time for HA testing. High availability was maintained and none of the simulated calls were impacted during inbox or boxto-box failover. We also monitored a random phone stream during all testing to determine if there was any type of audio glitch. In all tests, we heard continuous, streaming audio with no pause at all. The software redundancy and board redundancy, for both RP and ESP, were demonstrated on the ASR 1006. See Figure 2 on page 3. In addition to standard voice calls, we tested videoto-video calls. There was a momentary freeze of approximately three seconds in the repeated trials testing video calls. The audio did not cut out, and the calls remained active throughout all failovers. Media Forking Media Forking allows full-duplex audio of calls to be directed to a recording server. This feature is critical for quality and training purposes, such as in contact centers, or where calls are recorded for quality or compliance purposes. Each SIP call has two RTP streams, one for inbound audio and another for outbound audio. Each stream is forked for recording, for a total of four RTP streams. The two recording RTP streams are combined into one, so that only three streams exist per call. This procedure saves bandwidth, a primary limiting factor in most configurations. We performed Media Forking tests using CUBE (Ent) on the Cisco ISRG2 3945E platform. In repeated trials, we successfully forked 1,000 calls Copyright 2012 Miercom Cisco Unified Border Element Session Border Controller Page 2
Figure 2: Cisco Unified Border Element (CUBE) Enterprise- Class SBC High Availability at Scale Box-to-box failover was performed at the peaks in the chart, when all 12,000 calls were in steady-state. In repeated trials, none of the simulated calls were dropped, and call audio was not impacted when an additional call, made between handsets, was placed. for recording at a rate of 100 cps with a peak CPU utilization of 38%. System memory used 218 MB out of 509 MB. Transcoding Transcoding resolves the mismatch between dissimilar voice codecs. Universal transcoding in CUBE (Ent) on ASR is provided through the use of a Service Port Adapter (SPA). The SPA used was a Digital Signal Processor (DSP) SPA, utilizing three cores on each of the seven LSI SP2603 DSP chips, for a total of 21 DSP cores, to handle the transcoding duties. We performed transcoding for three profiles of complexity: Low (G.711 ulaw to G.711 alaw), Medium (G.711 to G.729), and Mixed (G.711 to several codec types). With eleven DSP SPA modules installed in an ASR 1006 chassis, there were call transcoding capacities of 9,933 calls for the Low profile, 6,468 calls for the Medium profile, and 3,927 calls for the Mixed profile. Ramping up and ramping down calls is CPU intensive. Memory utilization of 2.1 GB and 50% CPU utilization for the medium complexity profile was observed. The mixed profile used 1.6 GB memory and 50% CPU utilization. The low profile used 2.7 GB memory and 36% CPU utilization. Increasing the call rate to 150 cps with a call hold duration of 7 minutes for low profile had 68% CPU and 2.7 GB memory utilization. Advanced Features Advanced functionality testing was performed for various features. An In Service Software Upgrade (ISSU) was executed successfully with 500 calls active at 150 cps. We executed ISSU on both systems in a redundant scenario by upgrading the primary device, pushing the calls to the secondary during failover and reboot, and then upgrading the secondary device by pushing the calls back to the primary. No unexpected behavior or dropped connections were experienced. Header manipulation for any kind of SBC is a vital feature. Service providers require specific formats, order or headers for SIP messages for seamless operation. The CUBE software performed 10 concurrent SIP header manipulations at 150 cps with no issue, validating that it is a capable system for interconnectivity and interoperability. The header manipulation tests were performed at full load with 16,000 active calls. Random calls were spot checked to verify that headers were being manipulated and responded to appropriately. Bottom Line CUBE (Ent) provides a unique product which combines Session Border Controller functionality with the feature set already inherent within the Cisco ASR 1000 and ISR G2 class router families. It offers resiliency, interoperability, and high availability at scale, which makes it a serious choice for enterprise customers who are looking to make a graceful migration from TDM to SIP-based business telephony systems. Copyright 2012 Miercom Cisco Unified Border Element Session Border Controller Page 3
Test Bed Diagram Inbox HA Mu-4000 Service Analyzer ASR 1006 Box-to-Box HA IP 10.x.x.x IP 20.x.x.x ASR 1004 CUCM Media Forking ISRG2 3954E Media Recorder 9971 Phone Navtel How We Did It We performed a security and protocol integrity analysis of the CUBE (Ent) using the Mu Dynamics Mu- 4000 Service Analyzer. The behavior of the SBC was examined while it was subjected to mutations of several protocols including SIP, UDP and TLS. CUBE maintained all call processing functionality and was impervious to our attempts to compromise its performance. The test bed diagram displays the various chassis we tested, as well as the installed test equipment. There were an ASR 1006 and a pair of ASR 1004s in parallel in the network. Our SIP call generator was provided by Navtel and is able to handle higher throughput than required. In addition to the ASR systems, we tested an ISRG2 with a media recorder for media forking. Several IP phones were attached for manual testing and audio verification. The Cisco Unified Communications Manager (CUCM), an enterprise-class communications processing system, managed calls. CUBE (Ent) that is enabled on IOS XE version 3.4.1 was used. The ASR routers were running IOS version 15.1(3)s1. Mu Studio Security (www.mudynamics.com) provides a complete service assurance solution for determining the reliability, availability and security of IP-based applications and services. The Mu solution is highly automated, with lights-out fault isolation. Mu Studio Security speeds the remediation of software flaws by providing actionable reports and complete data on any faults. Mu-based testing is managed via a variety of interfaces, including its highly visual web-based graphical user interface or remotely controlled using REST- or XML-based APIs for integration into common laboratory automation frameworks such as HPQC or STAF. The tests in this report are intended to be reproducible for customers who wish to recreate them with the appropriate test and measurement equipment. Miercom recommends customers conduct their own needs analysis and testing specifically for the expected environment for the product deployment before making a product selection. Current or prospective customers interested in repeating these results may contact reviews@miercom.com to receive assistance from Miercom professional services to conduct these tests. Copyright 2012 Miercom Cisco Unified Border Element Session Border Controller Page 4
Miercom Performance Verified The performance of Cisco Unified Border Element Enterprise Edition, an enterprise-class SBC, was verified by Miercom. In hands-on testing, CUBE demonstrated advanced performance features such as: Handling 16,000 concurrent SIP-to-SIP calls at 150 calls per second Using an in-box High Availability feature to support active/standby failover of Route Processor and Embedded Services Processor with no dropped calls Transcoding over 9,900 SIP-to-SIP calls from G.711ulaw to G.711alaw Performing In Service Software Upgrade (ISSU) while processing 500 active calls at 150 calls per second Cisco Aggregation Services Router 1006 with Cisco Unified Border Element Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134 1-800-553-6387 www.cisco.com About Miercom s Product Testing Services Miercom has hundreds of product-comparison analyses published over the years in leading network trade periodicals including Network World, Business Communications Review - NoJitter, Communications News, xchange, Internet Telephony and other leading publications. Miercom s reputation as the leading, independent product test center is unquestioned. Miercom s private test services include competitive product analyses, as well as individual product evaluations. Miercom features comprehensive certification and test programs including: Certified Interoperable, Certified Reliable, Certified Secure and Certified Green. Products may also be evaluated under the NetWORKS As Advertised program, the industry s most thorough and trusted assessment for product usability and performance. Report SR111215 reviews@miercom.com www.miercom.com Before printing, please consider electronic distribution Product names or services mentioned in this report are registered trademarks of their respective owners. Miercom makes every effort to ensure that information contained within our reports is accurate and complete, but is not liable for any errors, inaccuracies or omissions. Miercom is not liable for damages arising out of or related to the information contained within this report. Consult with professional services such as Miercom Consulting for specific customer needs analysis. Copyright 2012 Miercom Cisco Unified Border Element Session Border Controller Page 5