ZXTM (Zeus Extensible Traffic Manager) In Virtual Mode With VMware Server A Broadband-Testing Report
First published February 2007 (V1.0) Published by Broadband-Testing La Calade, 11700 Moux, Aude, France Tel : +33 (0)4 68 43 99 70 Fax : +33 (0)4 68 43 99 71 E-mail : info@broadband-testing.co.uk Internet : HTTP://www.broadband-testing.co.uk 2007 Broadband-Testing All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the authors. Please note that access to or use of this Report is conditioned on the following: 1. The information in this Report is subject to change by Broadband-Testing without notice. 2. The information in this Report, at publication date, is believed by Broadband-Testing to be accurate and reliable, but is not guaranteed. All use of and reliance on this Report are at your sole risk. Broadband-Testing is not liable or responsible for any damages, losses or expenses arising from any error or omission in this Report. 3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY Broadband-Testing. ALL IMPLIED WARRANTIES, INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT ARE DISCLAIMED AND EXCLUDED BY Broadband-Testing. IN NO EVENT SHALL Broadband-Testing BE LIABLE FOR ANY CONSEQUENTIAL, INCIDENTAL OR INDIRECT DAMAGES, OR FOR ANY LOSS OF PROFIT, REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 4. This Report does not constitute an endorsement, recommendation or guarantee of any of the products (hardware or software) tested or the hardware and software used in testing the products. The testing does not guarantee that there are no errors or defects in the products, or that the products will meet your expectations, requirements, needs or specifications, or that they will operate without interruption. 5. This Report does not imply any endorsement, sponsorship, affiliation or verification by or with any companies mentioned in this report. 6. All trademarks, service marks, and trade names used in this Report are the trademarks, service marks, and trade names of their respective owners, and no endorsement of, sponsorship of, affiliation with, or involvement in, any of the testing, this Report or Broadband-Testing is implied, nor should it be inferred. ii Broadband-Testing 1995-2007
TABLE OF CONTENTS EXECUTIVE SUMMARY: ZXTM THE NEXT STEP - VIRTUALISATION... 1 INTRODUCTION: WHAT IS ZXTM?... 2 WHAT IS VIRTUALISATION?... 3 Partitioning... 3 Isolation... 3 Encapsulation... 3 ZXTM PUT TO THE TEST IN A VIRTUALISED ENVIRONMENT... 4 Testbed Details... 4 Test: Requests Per Second (RPS) At Layer 4... 5 Test: Requests Per Second (RPS) At Layer 7 (http)... 7 Test: SSL Performance... 8 Test: Cache Performance... 10 SUMMARY AND CONCLUSIONS... 11 APPENDIX: MORE TESTBED DETAILS... 12 TABLE OF FIGURES Figure 1 ZXTM Within The Backend Network...2 Figure 2 ZXTM As A Virtual Application Within VMware...3 Figure 3 Our Testbed For The Virtualisation Testing...4 Figure 4 L4 Requests Per Second...6 Figure 5 L4 Throughput...6 Figure 6 RPS At Layer 7...7 Figure 7 RPS At Layer 7...7 Figure 8 SSL Conns Per Second ID Reuse = 0...8 Figure 9 SSL Conns Per Second ID Reuse = 10...9 Figure 10 Cache Performance...10 Figure 11 Spirent Avalanche 2500...12 Figure 12 Creating A Spirent Avalanche Test...13 Broadband-Testing 1995-2007 iii
BROADBAND-TESTING Broadband-Testing is Europe s foremost independent network testing facility and consultancy organisation for broadband and network infrastructure products. Based in the south of France, Broadband-Testing offers extensive labs, demo and conference facilities. From this base, Broadband-Testing provides a range of specialist IT, networking and development services to vendors and end-user organisations throughout Europe, SEAP and the United States. Broadband-Testing is an associate of the following: NSS Network Testing Laboratories (specialising in security product testing) Broadband Vantage (broadband consultancy group) Limbo Creatives (bespoke software development) Broadband-Testing Laboratories are available to vendors and end-users for fully independent testing of networking, communications and security hardware and software. Broadband-Testing Laboratories operates an Approval scheme which enables products to be short-listed for purchase by end-users, based on their successful approval. Output from the labs, including detailed research reports, articles and white papers on the latest network-related technologies, are made available free of charge on our web site at HTTP://www.broadband-testing.co.uk The conference centre in Moux in the south of France is the ideal location for sales training, general seminars and product launches, and Broadband-Testing can also provide technical writing services for sales, marketing and technical documentation, as well as documentation and testhouse facilities for product development. Broadband-Testing Consultancy Services offers a range of network consultancy services including network design, strategy planning, Internet connectivity and product development assistance. iv Broadband-Testing 1995-2007
EXECUTIVE SUMMARY: ZXTM THE NEXT STEP - VIRTUALISATION To date with Zeus, we have proved that ZXTM: - Outperforms any other Layer 7 traffic management product on the market in terms of price:performance and sometimes in terms of performance at any price. - Scales perfectly; every time another appliance is added, performance increases accordingly, as well as adding extra resilience in each instance. So redundancy comes free - Is capable of performing highly complex Layer 7 operations at multi-gigabit speeds. - Is able to dramatically improve the performance of enterprise applications and application servers, such as BEA WebLogic and Apache Web Server respectively. In all of the above cases, ZXTM was running as a hardware appliance, but Zeus also provides its own software in a form that can be run on a Virtual Appliance. This report investigates how ZXTM performs when running in a virtualised environment. Our chosen platform is the free VMware Server. Note: we'll look at VMware's commercial ESX hypervisor another time. Our key findings show that: ZXTM installs and runs in a VMware environment with a minimum of deployment time and zero additional training or support requirements. ZXTM performance in a virtual environment is more than satisfactory and capable of supporting complex Layer 7 applications and transactions. While below those performance figures achieved with ZXTM running in a native environment, using a standard AMD Opteron platform (equivalent to a ZXTM 7000 appliance), we achieved performance figures that exceeded many results recorded by Broadband-Testing when testing dedicated Layer 7 hardware products from other vendors. We proved that it is entirely possible to run ZXTM in a virtualised environment alongside other applications on the same server. So where out and out performance is less of an issue than data management and manipulation, ZXTM can be a very cost-effective add-on to an existing application server environment. Both in terms of pure performance and flexibility ZXTM effectively rubberstamps the use of a virtualised environment. In our opinion, performance was outstanding and should be seen by VMware as a real proof point for its own product. It is possible to run in a virtual environment and achieve good performance. Broadband-Testing 1995-2007 1
INTRODUCTION: WHAT IS ZXTM? ZXTM from Zeus Technology operates at both Layer 4 (L4) load-balancing and Layer 7 (L7) intelligent traffic management levels and it is Ethernet-based but it is not a switch, or really any kind of Ethernet device per se, but effectively a server-based network appliance which is sold as software or an appliance. It therefore typically sits in front of the server farm, behind the Internet gateway, from where it conducts traffic management in a wide number of different ways, none of which simply involve throwing raw bandwidth at it. Figure 1 ZXTM Within The Backend Network Being an appliance, rather than a switch, means ZXTM works on a simple gateway principle one way in, one way out (though in practise this is likely to be multiple Gigabit NIC connections) sharing Gigabit Ethernet switch capacity with the server farm. With its multi-faceted redundancy configurations, it also means that huge clusters of distributed ZXTM devices can be created offering both extreme levels of performance and extreme levels of resilience). It can also run in a virtualised environment, as tested here with VMware Server. ZXTMs feature set is extensive, covering intelligent load-balancing and every aspect of L7 traffic management: throughput, compression, data manipulation, security such as DoS protection server and application optimisation, migration tools One excellent example of this attention to detail lies in ZXTMs TrafficScript feature for deep packet inspection and manipulation. This is quite simply the most comprehensive, rules-based methodology for traffic control available on anything we ve seen. So what ZXTM is all about is not throwing more bandwidth at the problem but, instead, throwing intelligence at it. Never mind the width, feel the quality as you might say 2 Broadband-Testing 1995-2007
WHAT IS VIRTUALISATION? Virtualisation decouples the physical hardware from the operating system with a view to producing a more efficient and flexible operating environment. It allows multiple virtual machines to run in isolation, side-by-side on the same physical machine. Each virtual machine has its own set of virtual hardware (e.g., RAM, CPU, NIC, etc.) upon which an operating system and applications are loaded. The operating system therefore sees a normal, working environment, regardless of the actual physical hardware components. Some of the key benefits of virtualisation are summarised as follows: Partitioning Isolation Multiple applications and operating systems can be supported within a single physical system. Servers can be consolidated into virtual machines on either a scale-up or scale-out architecture. Computing resources are treated as a uniform pool to be allocated to virtual machines in a controlled manner. Virtual machines are completely isolated from the host machine and other virtual machines. If a virtual machine crashes, all others are unaffected. Data does not leak across virtual machines and applications can only communicate over configured network connections. Encapsulation Complete virtual machine environment is saved as a single file, so it is easy to back up, move and copy. Standardised, virtualised hardware is presented to the application - guaranteeing compatibility. Figure 2 ZXTM As A Virtual Application Within VMware Broadband-Testing 1995-2007 3
ZXTM PUT TO THE TEST IN A VIRTUALISED ENVIRONMENT Testbed Details Broadband-Testing created a twin testbed in order to put ZXTM to the test in a virtualised environment, using a VMware Server platform. For the testing we were using VMware Server 1.0.1 running on Windows Server 2003 R2, on a dual AMD Opteron platform, typical of a standard ZXTM appliance environment. Benchmarking wise, we used both Zeus BenchBot, Linux-based web client simulators and a testbed based upon Spirent s Avalanche and Reflector 2700 client and server traffic simulator appliances (see appendix for more details). Figure 3 Our Testbed For The Virtualisation Testing As always, the basis of the testing was not to simply run a series of white gloves on technical laboratory tests, but to create as realistic as possible a virtual world for the ZXTM, with real web traffic and real-world applications. It was our goal to define tests in a very clear way that use parameters that would be very meaningful to a customer for use as a sizing-guide in a purchasing decision. It's not uncommon to see a vendor claim "connections per second" by measuring "null connections", or by only measuring the new connections established (i.e. SYN's per second, not full connections where there is an actual transfer of data) in order to boost metrics. All the tests performed were carrying out real end user transactions, which is why we are able to state that these were real world conditions. 4 Broadband-Testing 1995-2007
The scenarios we created and tested, were as follows: Requests Per Second (Layer 4 and Layer 7 - http) Maximum SSL Sessions (termination handled by ZXTM) L4/L7 Throughput Cache RPS and Throughput Note: For more reports on ZXTM, please go online to the Broadband-Testing website: www.broadband-testing.co.uk where you will find several reports available to download for free One important point to make, in terms of analysing these results against previous ZXTM testing we have carried out, is that in native form ZXTM runs as a 64-bit application. However, in virtual mode it is only able to run as a 32-bit application, so there is an obvious potential performance loss point, in addition to the extra overhead involved in the layered approach that virtualisation creates. Realistically, then, we were looking for ZXTM to achieve anywhere approaching 25% of its native performance which, in the real world, would equate to excellent standalone performance in its own right and outstanding price:performance levels. Anything achieved above this would be a real bonus. Test: Requests Per Second (RPS) At Layer 4 Here we were looking for a measure of how many concurrent connections effective Internet sessions - ZXTM could support during an extended period of user activity. This not only gives us an indication of the scalability per device of the ZXTM, but also its resilience, vital within a virtualised environment. Specifically, we looked to see how many requests per second each device was capable of sustaining across a range of file request sizes. Max requests is with a minimal file size, then we tested again with 2K and 8K file sizes, both common in the real world. We began by testing at Layer 4, then tested at Layer 7, where based on previous ZXTM tests we would expect to see some performance loss compared with the Layer 4 figures but still excellent results (see next section). We also measure data throughput achieved. While not critical for all Layer 7 applications, it is important to measure the total throughput capabilities of a traffic management device, in order to avoid accidentally creating bottlenecks at the device itself. Better overall throughput also means that device is more capable at performing at Layer 4 intensive load-balancing as well as at Layer 7; a better all-rounder in other words. Broadband-Testing 1995-2007 5
L4 Requests Per Second 30000 25700 25000 20000 13700 15000 10000 5700 5000 0 L4 requests/second L4 2k file requests/second L4 8k file requests/second Figure 4 L4 Requests Per Second As we can see from the graph and, as we expected, ZXTM-VMware performance was excellent. Indeed, these performance figures exceed some recorded by Broadband- Testing for dedicated Layer 7 hardware from other vendors. An RPS level of 25,700 is more than enough for most, high-level real world applications. The bandwidth story shows a similar trend, with VMware sustaining in excess of half a gigabit per second. L4 Throughput (Gbps) 0.522 0.6 0.5 0.4 0.3 0.2 0.1 0 L4 throughput (GBits/sec) Figure 5 L4 Throughput 6 Broadband-Testing 1995-2007
Test: Requests Per Second (RPS) At Layer 7 (http) HTTP Requests Per Second 20000 19100 18000 16000 14000 12000 11200 10000 8000 6000 5090 4000 2000 0 HTTP requests/second HTTP 2k file requests/second HTTP 8k file requests/second Figure 6 RPS At Layer 7 Here we effectively repeated the tests but as http requests at Layer 7. The theme continued at Layer 7 with our http traffic tests, with VMware showing excellent results, max ing at close on 20,000 RPS excellent for Layer 7 performance. Sustained throughput levels were very similar to those recorded at Layer 4. HTTP Throughput (Gbps) 0.53 0.6 0.5 0.4 0.3 0.2 0.1 0 HTTP throughput (GBits/sec) Figure 7 RPS At Layer 7 Broadband-Testing 1995-2007 7
Test: SSL Performance As more and more secure services appear on the Internet, so more and more SSL transactions will occur. Real-world SSL transaction performance, whether terminating at the device under test, or then re-encrypting to send the request onto a target server, is therefore increasingly vital. So it is important for a Layer 7 device to offer a high level of sustainable SSL support where the device terminates the SSL sessions, rather than passing them to the target server. SSL session handling is an absolute killer for server performance, so this is a vital function of a Layer 7 traffic management device. For the SSL tests, we ran two separate tests using the RC4-MD5 cipher - with two different session ID reuse values this is the number of times an SSL connection is allowed to use the same ID. For reasons of working the test device as hard as possible, our first test allowed a once-only use of an ID. This is the most secure methodology but is rarely used in the real world. It is an excellent way of measuring the capabilities of a L7 device under the most trying circumstances however. We also ran a more typical 10-session ID reuse test in order to get something much closer to real world figures. Below we see the Connections Per Second figure for the VMware deployment reaching 995cps. Bear in mind that it s not long ago since dedicated hardware platforms exceeding 800cps SSL performance were deemed as exceptional. SSL CPS (SID reuse = 0) SSL connections/second SSL throughput (GBits/sec) 1200 1000 800 0.409 0.45 0.4 0.35 0.3 600 400 0.25 0.2 0.15 200 0 0.1 0.05 0 Figure 8 SSL Conns Per Second ID Reuse = 0 8 Broadband-Testing 1995-2007
With session ID re-use set to 10, we see a significant improvement in SSL transaction performance, with VMware max ing at an excellent 7259cps. SSL CPS (SID reuse - 10) 8000 7259 7000 Conns Per Sec 6000 5000 4000 3000 2000 4660 2420 1000 0 SSL CPS Max SSL CPS 2k file SSL CPS 8k file Figure 9 SSL Conns Per Second ID Reuse = 10 Broadband-Testing 1995-2007 9
Test: Cache Performance Here we tested the cache performance of ZXTM, both in terms of Requests Per Second (RPS) achieved and throughput generated and, again, the VMware implementation excelled, achieving 45,300rps while sustaining a throughput rate of almost 1Gbps. Cache Performance 50000 1.2 45000 40000 45300 0.973 1 35000 30000 25000 20000 15000 10000 5000 0.8 0.6 0.4 0.2 Cache requests/second Cache throughput (GBits/sec) 0 1 0 Figure 10 Cache Performance 10 Broadband-Testing 1995-2007
SUMMARY AND CONCLUSIONS Within the scope of this test we set out to show, primarily, that the ZXTM traffic management application is able to run, and run well, in a virtual environment. Our testing proved that this is indeed the case, subjecting as we did ZXTM to lengthy and repeated tests in a VMware Server environment. What this means is that Layer 7 traffic management solutions can be deployed in an incredibly cost-effective and efficient manner by those users valuing the intelligence and manageability of such a deployment over out and out performance. That said, we were delighted with the performance results we obtained, where we exceeded our own expectations and at the same time fully validated VMware itself as a virtual platform capable of supporting high-performance applications. In A Nutshell We proved that it is entirely possible to run ZXTM in a virtualised environment alongside other applications on the same server. So where out and out performance is less of an issue than data management and manipulation, ZXTM can be a very cost-effective add-on to an existing application server environment. Both in terms of pure performance and flexibility ZXTM effectively rubber-stamps the use of a virtualised environment, notably when running under VMware, where performance was outstanding and should be seen by VMware as a real proof point for its own product. Broadband-Testing 1995-2007 11
APPENDIX: MORE TESTBED DETAILS Internet architectures are becoming increasingly complex. Whether you're building network equipment or providing a service, you must deliver consistent performance under all conditions. Until now, capacity assessment at high-loads has been a costly and complex process. For this reason Spirent Communications introduced the Avalanche and Reflector appliances to assist with the challenge. At Broadband-Testing we have taken these web application simulation and planning products and integrated them into our test-bed simulating real-life Internet conditions; those that the average user experiences daily. Figure 11 Spirent Avalanche 2500 Avalanche is described by Spirent as a capacity assessment product that challenges any computing infrastructure or network device to stand up to the real-world load and complexity of the Internet or intranets The system determines the architectural effectiveness, points of failure, and the performance capabilities of a network or system. Using Avalanche to generate Internet user traffic and Reflector to emulate large clusters of data servers, you can simulate even the world's largest customer environments. The system provides invaluable information about a site's architectural effectiveness, points of failure, modes of performance degradation, robustness under critical load, and potential performance bottlenecks. It is able to set up, transfer data over, and tear down connections at very high rates - all while handling cookies, IP masquerading for large numbers of addresses, and traversing tens of thousands of URLs. Avalanche initiates and maintains more than a million concurrent connections, each appearing to come from a different IP address. This allows realistic and accurate capacity assessment of routers, firewalls, load-balancing switches, and Web, application, and database servers. It helps identify potential bottlenecks from the router connection all the way to the database. This accuracy is especially critical for gauging Layer 4-7 performance. The ability to additionally simulate error conditions such as HTTP aborts, packet loss, and TCP/IP stack idiosyncrasies can help anticipate-and avoid-significant and previously unknown impacts on performance. 12 Broadband-Testing 1995-2007
To enable more accurate load simulations across multi-tiered Web site architectures, the system also supports extremely realistic user modelling behaviours such as think times, click stream, and HTTP aborts that cause Web servers to terminate connections while back-end application servers continue to process requests. Configuring in this way is simple as both Avalanche and Reflector directly from a desktop browser to set up tests, review feedback in real time, and easily reconfigure test parameters. Figure 12 Creating A Spirent Avalanche Test The Avalanche also supports browser cookies, html forms, HTTP posts, and SSLencrypted traffic. The system therefore gives you the flexibility to specify data sources and mix and match data sets to recreate accurate user behaviour at very high performance levels. It also simulates SSL loads that can stress the world's most sophisticated secure e- commerce platforms. It also includes configurable cipher suites that enable you to emulate different types of browsers. Avalanche includes a high-accuracy delay factor that mimics latencies in users' connections by simulating the long-lived connections that tie up networking resources. Long-lived, slow links can have a far more detrimental effect on performance than a large number of short-lived connections, so this approach delivers more realistic test results. While Avalanche focuses on the client activity, Reflector realistically simulates the behaviour of large Web, application, and data server environments. Combined with Avalanche it therefore provides a total solution for recreating the world's largest server environments. Broadband-Testing 1995-2007 13
By generating accurate and consistent HTTP responses to Avalanche's high volume of realistic Internet user requests, Reflector tests to capacity any equipment or network you connect between the two systems. Its protocol-level accuracy helps you assure the stability and performance of switches, routers, load balancers, firewalls, caches, and other Layer 4-7 devices. The system is ideal for helping infrastructure service providers validate, enforce, and maintain service level agreements (SLAs). 14 Broadband-Testing 1995-2007