PARALLELS CLOUD SERVER

PARALLELS CLOUD SERVER Performance and Scalability 1

Table of Contents Executive Summary... Error! Bookmark not defined. LAMP Stack Performance Evaluation... Error! Bookmark not defined. Background... Error! Bookmark not defined. Test Results... 6 Analysis... 6 vconsolidate Performance Evaluations... 8 Background... 8 Test Results... 8 Analysis... 9 Conclusion... 11 Appendices... 12 LAMP Test Configuration... 13 vconsolidate Test Configuration... 14 2

Executive Summary In today s cloud infrastructure, virtualization plays a crucial role. In fact, many of the benefits of a cloud environment, including high availability, flexibility, and simplified resource management, stem from virtualization. This white paper compares two key aspects of virtualization technologies performance and scalability for several common virtualization solutions. Strong performance is a primary requirement for customer satisfaction, as faster technologies deliver a better customer experience. Faster technologies reduce latency, require less hardware investment, and consume less electricity. It s equally important, however, to have a highly scalable virtualization solution, so that data centers can continue to deliver high performance as load levels increase. Easy scalability is especially important during heavy sales periods, such as Black Friday in the US, where data centers must handle workloads that are several times heavier than normal. If performance degrades with higher workloads, cascading services outages may result because as one server becomes overloaded, it puts heavier loads on the other servers, which then become overloaded in turn. In our performance and scalability comparisons, we looked at the most widely used virtualization infrastructure solutions: Parallels Cloud Server 6 (which includes both Parallels Containers and Parallels Hypervisor), Parallels Virtuozzo Containers 4.7 (the previous version of Parallels Containers), Citrix Xen 6, and Linux KVM. We ran these virtualization solutions on hardware with 48GB memory and 24 CPU cores, which our statistics show is a typical configuration used by hosting providers for cloud infrastructure. 1 To ensure that their data centers are operating as efficiently as possible, system engineers must test every aspect of their cloud infrastructure, from hardware stability to the aggregated performance of software and hardware stacks. Like many companies, we performed our tests using Dell s DVD Store benchmark, an industry-standard workload simulator for LAMP (Linux, Apache, MySQL, PHP) stacks, which are the most frequently used software stacks. We also used Intel s vconsolidate, a benchmark for consolidation scenarios. 1 See Appendix for more information about hardware and software configurations. 3

To determine the best virtualization solution overall, in terms of both performance and scalability, we used the geometric mean of the values in each test a metric that is commonly used to determine the aggregated score of a set of benchmark numbers. These aggregated results show that Parallels Containers significantly outperforms all other virtualization solutions tested: For the LAMP benchmark, Parallels Containers outperformed Linux KVM by 250% and Citrix Xen by 350%. For the vconsolidate benchmark, Parallels Containers outperformed Linux KVM by 80% and Citrix XEN by 140%. Moreover, according to the benchmark results, the current version of Parallels Containers showed an 11% improvement over the previous version, Parallels Virtuozzo Containers 4.7. In addition, the Parallels Hypervisor component of Parallels Cloud Server 6 outperformed all the other hypervisor solutions. The high performance and scalability of Parallels Containers is due to the fact that the container-based approach eliminates the need to duplicate operating system drivers inside each container. In contrast, hypervisor-based solutions emulate CPUs and devices in each virtual machine (VM), adding a huge amount of overhead. The flexible design of containerbased virtualization also allows containers to use all the available resources based on their needs, resulting in more efficient use of hardware resources than is possible with KVM and Xen. Additionally, Parallels Containers use numerous advanced optimizations, such as system-wide caching and page-sharing. The remainder of this paper details the tests and benchmark results. LAMP Stack Performance Evaluation Background This test analyzed the performance and scalability of Parallels Cloud Server 6 (consisting of Parallels Containers and Parallels Hypervisor), Parallels Virtuozzo Containers 4.7, Citrix Xen, and Linux KVM all running the industry-standard LAMP web application stack. The LAMP stack, which is widely used for building web sites on Linux, measures how many LAMP virtualization instances can operate on a single piece of hardware. For our tests, we 4

used Dell s DVD Store benchmark 2, a fully functional DVD Internet store, as a workload simulator for LAMP stacks. The DVD Store version 2.1 is a completely open-source ecommerce test application, with a backend database component, a Web application layer, and driver programs. The software set consists of CentOS 5.8 x64; Apache, an industry standard for web servers today; MySQL, the most popular opensource database; and PHP, one of the most popular languages for Web applications 3. Together, these components represent a typical web environment for many independent software vendors that offer software as a service (SaaS). The DVD Store test simulates typical user activity at an ecommerce site: logging in; browsing for DVDs by title, actor, or category; adding the selected DVDs to their shopping cart; and then purchasing the DVDs. This scenario actually has broader application beyond ecommerce, as most common PHP use cases involve accessing a database, getting and processing data, and displaying the HTML results to the user. The benchmark metric is the total number of service requests processed per second. Tests of all virtualization solutions were performed on the same server to ensure that the test conditions would be identical. Each virtual machine or container was configured with 2 GB of RAM, plus one vcpu for uniprocessor (UP) guest scenarios or four vcpus for symmetrical multiprocessing (SMP) guest scenarios. The tests used the same web pages for all guest scenarios, with workloads for all guests running in parallel. Virtualization of high-performance production workloads typically involves some sort of performance penalty, created by the virtualization layer overhead. As the number of guests on the hardware node increases, so does the impact of the virtualization layer overhead on LAMP stack performance. The LAMP test demonstrates this phenomenon by measuring the total number of requests per second that a hardware node can support as the number of guests increases. For each virtualization technology that we tested, we began with one guest to establish a baseline, and then gradually increased the number of guests. By comparing the results across all virtualization technologies tested, we determined the scalability and overhead of each virtualization solution when running a LAMP stack in a production cloud environment. 2 For more information about the DVD-Store benchmark, please visit http://linux.dell.com/dvdstore and http://en.community.dell.com/techcenter/extras/w/wiki/dvd-store.aspx. 3 See language statistics at http://lang-index.sourceforge.net. 5

Test Results Figures 1 and 2 show the consolidated performance score of multiple virtualized LAMP guests, ranging from 1 to 21 guests, all running simultaneously. Results are given for Parallels Containers, Parallels Virtuozzo Containers 4.7, Parallels Hypervisor VMs, Citrix Xen VMs, and KVM VMs. The performance score of each virtualization solution is the total rate of requests/sec for all guests. Analysis The performance test results clearly demonstrate that Parallels Containers outperforms hypervisor solutions and even performs faster than its previous version, Parallels Virtuozzo Containers 4.7. In addition, as illustrated in Figure 3 (which aggregates the results of Figures 1 and 2), Parallels Containers outperforms KVM by 250% and Citrix Xen by 350%. And for those who prefer hypervisors to containers, Parallels Cloud Server 6.0 is still the best solution, with Parallels Hypervisor outperforming KVM by 90% and Citrix Xen by 140%. Figure 1. LAMP benchmark results with uniprocessor guests (1 vcpu, 2GB) 6

Figure 2. LAMP benchmark results on SMP guests (4 vcpus, 2GB) Figure 3. Aggregated LAMP benchmark results 7

vconsolidate Performance Evaluations Background The vconsolidate benchmark, which was developed by Intel in cooperation with other vendors, measures aggregated server performance and scalability in a scenario involving consolidation of several different workloads. We chose this benchmark because it uses commodity software similar to that of a typical hosting environment. The vconsolidate benchmark uses a set of four guests running different, unrelated applications (in contrast to the LAMP benchmark, in which all guests run the same set of applications). With vconsolidate, each application runs in a separate guest, with all four guests running concurrently. The applications consist of Java (SPECjbb test), web (Webbench), database (Sysbench), and mail (Microsoft Exchange Load Tool). (Note that because the mail stack runs only on Windows servers and we wanted our test results to be comparable across both Windows and Linux cases, we left the mail stack idle in all cases.) Each set of four such VMs or containers constructs a Consolidation Stack Unit (CSU). The performance metric is the geometric mean of the throughput of each workload type, measured in Java operations/sec for Java applications, requests/sec for web applications, and transactions/sec for databases. As with the LAMP stack test, this benchmark was run on Parallels Cloud Server 6 (consisting of Parallels Containers and Parallels Hypervisor), Parallels Virtuozzo Containers 4.7, Citrix Xen, and Linux KVM. Tests of all virtualization solutions were performed on the same server to ensure that the test conditions would be identical. Each virtual machine or container was configured with 1 vcpu and 1 GB of RAM. We performed the test using two different guest operating systems: CentOS 5.8 x86_64 and Microsoft Windows Server 2008 R2 SP1 DC. (Note that for the Windows operating system, we did not include results for Parallels Containers or Parallels Virtuozzo Containers 4.7, as these containers can operate only on Linux-based operating systems.) Test Results Figures 4-6, below, show the consolidated performance scores of multiple virtualized CSUs, with each CSU consisting of four guests, each running a different workload. The workloads were the same in all tests, and were equally divided among all CSUs participating in each 8

test. The performance score of each virtualization solution (the geometric mean of the various workload types) is given for different numbers of simultaneously running CSUs. Analysis The results show that the Parallels Containers component of Parallels Cloud Server 6.0 significantly outperforms all hypervisor solutions, beating KVM by 80% and Xen by 140%. And its speed is equal to or faster than that of Parallels Virtuozzo Containers 4.7. Figure 4. vconsolidate benchmark results with CentOS 5.8 uniprocessor guests (1vCPU, 1GB, 4VMs/CTs in CSU) 9

Figure 5. vconsolidate benchmark results with Windows 2008 uniprocessor guests (1vCPU, 1GB, 4VMs/CTs in CSU) Figure 6. Aggregated vconsolidate benchmark results 10

Conclusion In using the industry-standard LAMP and vconsolidate benchmarks to test the performance and scalability of a variety of virtualization solutions, we found that Parallels Containers scored significantly higher than all hypervisor solutions tested and as the number of containers or VMs increased, the advantage of Parallels Containers continued to grow. The aggregated overall test scores show that: In the LAMP test, Parallels Containers outperformed KVM by 250% and Xen by 350%. In the vconsolidate test, Parallels Containers outperformed KVM by 80% and XEN by 140%. These tests demonstrate that the Parallels Containers component of Parallels Cloud Server 6 is both the highest-performing and the most scalable of all virtualization solutions. 11

Appendices Hardware (The Same Hardware was used for All Tests) Server Processor: Intel Xeon X5650; 2.67 GHz; 2x hex-cores (= 12 cores; 24 hyper threads) Memory: 48 GB RAM (two NUMA nodes) Storage: Adaptec 5405 RAID-0 with four WD 7200 RPM SATA drives Network: Ethernet Intel 82574L 1Gigabit network connection Client Processor: Intel Xeon X7460; 2.136 GHz; 2x hex-cores (= 12 cores; 24 hyper threads) Memory: 32 GB RAM Network: Ethernet Intel 82575EB 1Gigabit network connection Special Notes To exclude possible network bottlenecks in lab infrastructure (e.g., slow Ethernet routing), we tested LAMP and vconsolidate with a direct connection, established via a crossover 1Gb Enternet. Server RAID controller was configured with RAID cache and disk writer cache enabled. KVM Virtualization Settings Note that KVM shows especially poor performance with the default settings (without virtio). Consequently, we added the virtio option (which requires drivers for Windows on the guest operating system) to the KVM bridge network connection and storage disk, thereby turning on paravirtualization. For more information, see http://wiki.libvirt.org/page/virtio. 12

LAMP Test Configuration Software Virtualization Software Parallels Cloud Server 6 (2.6.32-042stab061.8) Parallels Virtuozzo Containers 4.7 (4.7.0-99, 2.6.32-042stab031.1) KVM (RHEL 6.3 2.6.32-279) Citrix Xen 6.0.2 Guest Configuration CentOS 5.8 x86_64 Each VM/CT was configured with 1 vcpu for UP (4 vcpus for SMP), 2 GB RAM Guest LAMP stack includes: Linux: CentOS 5.8 x86_64 Apache: 2.2.3-65 MySQL: 5.0.95 PHP: 5.1.6 (Zend v2.1.0) DVD Store Configuration The test creates workloads independently for each VM/CT, with all workloads running in parallel. Each value on the graphs represents an average of 3 loads, at 10 minutes per load. We used a medium-size database (1GB), tuning MySQL for that size: innodb_buffer_size=1536mb. The database had 2,000,000 customers, 100,000 orders per month, and 100,000 products. Load driver command line: dvd-store-client --nthreads=3 --rampup=300 --seconds=600 --mixes=3 --db_size=1024 -- n_line_ items=1 --pct_newcustomers=1 --think_time=0 13

vconsolidate Test Configuration Software Virtualization Software Parallels Cloud Server 6 (2.6.32-042stab061.8) Parallels Virtuozzo Containers 4.7 (4.7.0-99, 2.6.32-042stab031.1) KVM (RHEL 6.3 2.6.32-279) Citrix Xen 6.0.2 Guest Configuration CentOS 5.8 x86_64; Windows 2008 R2 SP1 DC Each VM/CT was configured with 1 vcpu, 1 GB RAM Software All tests used: A custom vconsolidate profile was used for the test. Specifically, there were four load threads for the Java workload, four for the database workload, and eight for the web workload (standard settings). The firewall was turned off. All other settings were left as default. 14