RED HAT ENTERPRISE VIRTUALIZATION PERFORMANCE: SPECVIRT BENCHMARK AT A GLANCE The performance of Red Hat Enterprise Virtualization can be compared to other virtualization platforms using the SPECvirt_sc2010 benchmark SPECvirt_sc2010 measures hypervisor performance using realistic workloads As of Jan. 1, 2012, RHEV leads VMware in performance on comparable servers defined by CPU count The RHEV subscription model provides high performance at a lower cost-per-specvirt_sc2010 than VMware The addition of vram pricing to VMware vsphere 5 increases the overall cost of VMware performance, the variability of that cost, and may require additional optimization to minimize cost beyond what is required to maximize performance OVERVIEW Red Hat Enterprise Virtualization (RHEV) is the strategic virtualization alternative for organizations looking for better total cost of ownership, faster return on investment, accelerated break-even, and freedom from vendor lock-in. RHEV consists of both a hypervisor technology and an enterprise virtualization manager. The RHEV hypervisor, based on the Red Hat Enterprise Linux kernel and the Kernel-based Virtual Machine (KVM) hypervisor technology, offers the high performance and scalability that Red Hat is known for. Hypervisor performance is a key factor in deciding which virtualization platform to implement, and Red Hat Enterprise Virtualization performance, as measured by the SPECvirt_sc2010 benchmark, leads the industry both in terms of highest overall performance and highest number of performant virtual machines on a single server. WHAT IS SPECvirt_sc2010? SPECvirt_sc2010 is the first vendor-neutral benchmark designed to measure the performance of datacenter servers that are used for server virtualization. The benchmark was developed by the non-profit Standard Performance Evaluation Corporation (SPEC) virtualization subcommittee, whose members and contributors include AMD, Dell, Fujitsu, HP, IBM, Intel, Oracle, Red Hat, Unisys and VMware. SPECvirt_sc2010 uses realistic workloads and SPEC s proven performance measurement methodologies to enable vendors, users and researchers to compare system performance across multiple hardware, virtualization platforms, and applications. The benchmark utilizes several SPEC workloads representing applications that are common targets of virtualization and server consolidation. SPEC modified each of these standard workloads to match a typical server consolidation scenario of CPU resource requirements, memory, disk I/O, and network utilization for each workload. These workloads are modified versions of SPECweb2005, SPECjAppServer2004, and SPECmail2008. Scaling is achieved by running additional sets of virtual machines (VMs), called tiles, until overall throughput reaches a peak. All VMs must continue to meet required quality of service (QoS) criteria. An example of a SPECvirt_sc2010 tile is shown below in Figure 1:
Figure 1 Scaling the workload on the tested system consists of running an increasing number of tiles. Peak performance is the point at which the addition of another tile (or fraction) either fails the QoS criteria or fails to improve the overall metric. HOW DOES RHEV PERFORM ON SPECvirt_sc2010? As of Jan. 1, 2012, 21 SPECvirt_sc2010 benchmark results have been published by organizations testing both Red Hat Enterprise Virtualization and VMware ESXi hypervisor platforms. All SPECvirt_sc2010 scores below are measured as XXXX@YYY where XXXX is the raw performance score and YYY is the number of VMs that yielded that score. Performance per core and per CPU Below in Table 1, the best SPECvirt_sc2010 score for each category of core count is compared. RHEV leads VMware in the 12, 20, and 40 core categories and has the only published 64 and 80 core scores. # of cores Best VMware SPECvirt_sc2010 score Best RHEV SPECvirt_sc2010 score 12 1221@78 1367@84 16 1570@102* 1369@84 20 1878@120 2144@132 32 2742@168 N/A 40 3824@234 4603@282 64 N/A 5466@336 80 N/A 8956@552 Table 1 * The best 16 core score for VMware is on a dual-socket AMD Opteron 6282 SE with 8 cores and 16 integer units per CPU, compared with the 8 cores and 8 integer units per CPU with the dual-socket Intel X7560 used for the best Red Hat score.
As shown in Table 2, RHEV also leads VMware in the two- and four-socket server category and has the only published scores for eight-socket servers, including the single server with the highest overall performance (8956) and the highest number of performant virtual machines (552 VMs). # of CPU Best VMware SPECvirt_sc2010 score Best RHEV SPECvirt_sc2010 score 2 1878@120 2144@132 4 3824@234 4603@282 8 N/A 8956@552 Table 2 CONSIDERING COST/PERFORMANCE When deciding which virtualization solution to deploy, it is important to consider not only the raw performance of the platform but also how much it will cost to achieve that performance. Like all Red Hat products, RHEV is sold on a subscription basis. Unlike proprietary software, there is no high upfront license fee. Instead, you pay per socket per year for the software you use. RHEV has always been less expensive than VMware on a one year and three year basis. (See the separate Red Hat Enterprise Virtualization for Servers: Competitive Pricing Guide for more details.) With the release of vsphere 5 in July 2011, VMware introduced the concept of licensing virtual RAM (vram). With vsphere 5, you now need to license the greater of either (a) the number of socket licenses or (b) the number of licenses needed for the vram on running virtual machines. For this paper, the three-year costs for the 21 submitted SPECvirt_sc2010 scores that were published as of Jan. 1, 2012 were compared. Each SPECvirt_sc2010 submission contains detailed information on the number of sockets on each tested system as well as the vram configuration of each virtual machine. This information was used to price RHEV, vsphere 4 (which was licensed only per-socket), and vsphere 5 (which requires the greater of the persocket or per-vram licenses). The cost to license the 21 listed environments for three years was calculated, and then the calculated license cost was divided by the performance value of the SPECvirt_sc2010 score to calculate dollars per SPECvirt_sc2010 ($/SPECvirt_sc2010 performance). For example, in the first score listed in the results page at www.spec.org/osg/virt_sc2010/results/specvirt_sc2010_perf.html, the SPECvirt_sc2010 score 2721@168 was achieved on a 4 socket system (per the Physical Configuration section in the Full Result Disclosure) running 476GB of vram (calculated by summing the Memory
defined for each VM in the tile as shown in the Virtual Configuration section, and multiplying by the number of tiles shown in the Performance Summary). RHEV pricing is based on socket count only, so the cost to license RHEV with 24x7 support for this environment is: $749/socket/year X 4 sockets X 3 years = $8998 The calculated value of $/SPECvirt_sc2010 performance over three years for RHEV for this configuration is therefore: $8998 / 2721 = $3.30/SPECvirt_sc2010 RHEV cost is lowest per SPECvirt_sc2010 performance As you can see in Figure 2, the cost to license RHEV for three years for each of the systems tested in the SPECvirt_sc2010 benchmark is the lowest whether you are looking at 2, 4, or 8 socket systems. The average cost is $2.72/SPECvirt_sc2010. In addition, the costs are clustered in a narrow range of $1.95 $3.84/SPECvirt_sc2010. Figure 2 vsphere 4 cost per performance is higher VMware vsphere 4 is more expensive than RHEV over the three year period, averaging $6.28/SPECvirt_sc2010. That is roughly 2.28 times the cost per SPECvirt_sc2010 of RHEV. This is due to the upfront license fee and the Support and Subscription (SnS). Again, because vsphere 4 pricing only considered sockets used, the cost range is fairly narrow at $4.45 $8.76/SPECvirt_sc2010.
vsphere 5 cost is highest, more variable, pushes Enterprise Plus With the addition of vram licensing, vsphere 5 pricing is now the highest, averaging $17.38/SPECvirt_sc2010, which is 6.38 times the average cost per SPECvirt_sc2010 of RHEV, and an increase of 2.8 times the cost of licensing the previous version vsphere 4. This represents a substantial increase in the cost of the hypervisor. The costs for vsphere 5 are also highly variable, with a low of $10.19/SPECvirt_sc2010 to a high of $35.29/SPECvirt_sc2010. The reason for this increased cost is that you must license the greater of the per-socket cost or the per-vram cost. High performance and utilization of virtualization involves many virtual machines with a large amount of vram; and in each of these cases the vram cost is what needs to be used to determine the cost of the vsphere licensing. In all of these cases, the cost was calculated using the Enterprise Plus Edition of VSphere 5. Why? Although Enterprise Edition is less expensive per socket, it only comes with 64 GB vram per license. Enterprise Plus comes with 96 GB of vram per license. In these cost calculations, it was actually less expensive to upgrade to Enterprise Plus solely to license the vram. The range of costs for vsphere 5 is also more variable than with the socket-based licensing of vsphere 4 and RHEV, because for each of the test systems, virtual machines were tuned for performance, not for vram utilization. When only socket count needs to be considered, one can be conservative and give the virtual machines more RAM and allow the hypervisor to manage the memory. Because VMware users now have to pay for vram, an additional optimization to minimize the vram in each VM may be necessary to lower costs. Time and effort spent on optimizing vram may add a hidden cost to your virtualization environment in addition to the higher overall cost of licensing vram, and may require trade-offs in either performance or cost. SUMMARY In conclusion, Red Hat Enterprise Virtualization not only offers great performance on the only independent virtualization benchmark, it does so at a great cost/performance ratio especially when you take the vram licensing of vsphere 5 into account. WHAT S NEXT For more information, please go to /rhev or contact your Red Hat authorized reseller or local Red Hat sales office.
DISCLAIMER Red Hat product offerings, pricing, and terms and conditions are accurate as of January 2012 and are subject to change and local pricing. Pricing for non-red Hat virtualization platforms and guest operating systems are based on published retail or volume license pricing in the United States as of December 2011 and is subject to change and local pricing. SPEC, SPECvirt, and SPECvirt_sc are trademarks or registered trademarks of the Standard Performance Evaluation Corp. (SPEC). Competitive numbers shown reflect results published on www.spec.org as of January 1, 2012. The comparison presented is based on core count versus SPECvirt_sc2010 score (Table 1), CPU count versus SPECvirt_sc2010 score (Table 2), and license cost divided by SPECvirt_sc2010 performance score for each published system under test (SUT) based on (a) 1 Red Hat Enterprise Virtualization Premium subscription per SUT socket for three years, (b) vsphere Enterprise Edition license per SUT socket plus three years of Enterprise Edition Platinum SnS per SUT socket, and (c) vsphere Enterprise Plus edition and three years of Enterprise Plus Edition Platinum SnS to license the amount of vram used in the virtual machines specified in the Virtual Configuration section for each SUT per the published VMware rules for vram licensing. For the latest SPECvirt_sc2010 results visit www.spec.org/osg/virt_sc2010. ABOUT RED HAT Red Hat, the world s leading provider of open source solutions and an S&P 500 company, is headquartered in Raleigh, NC with more than 70 offices spanning the globe. Red Hat provides highquality, affordable technology with its operating system platform, Red Hat Enterprise Linux, together with cloud, virtualization, applications, management, storage and service-oriented architecture (SOA) solutions, including Red Hat Enterprise Virtualization and JBoss Enterprise Middleware. Red Hat also offers support, training and consulting services to its customers worldwide. SALES AND INQUIRIES NORTH AMERICA 1 888 REDHAT1 EUROPE, MIDDLE EAST AND AFRICA 00800 7334 2835 www.europe.redhat.com europe@redhat.com ASIA PACIFIC +65 6490 4200 www.apac.redhat.com apac@redhat.com LATIN AMERICA +54 11 4329 7300 www.latam.redhat.com info-latam@redhat.com Copyright 2012 Red Hat, Inc. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, and RHCE are trademarks of Red Hat, Inc., registered in the U.S. and other countries. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. Jan 2012 / DOC63160