Server Virtualization: Avoiding the I/O Trap How flash memory arrays and NFS caching helps balance increasing I/O loads of virtualized servers November 2010
2 Introduction Many companies see dramatic improvements with server virtualization by streamlining application deployments, and increasing overall server utilization. But the rush to capitalize on these rewards can leave some installations with looming performance impacts. This occurs when consolidation driven by server virtualization does not include plans for a corresponding storage I/O performance adjustment. This paper examines how server virtualization impacts overall IT performance and how centralized storage caching protects the performance levels of the entire infrastructure, including storage....i think that over the past few years so much emphasis had been put on server consolidation that much of the [virtualization] community has ignored the disk I/O discussion. I don t think this was intentional but the value prop was so impressive around consolidation and test/dev that the I/O discussion was not a primary concern, the target audience has often also been server engineering teams and not storage engineering. Richard Bocchinfuso, Chief Technology Officer, MTI Technology 1 1 Source: http://www.gotitsolutions.org/2006/11/08/disk-io-a-virtualization-focus-area.html
Before Server Virtualization The Traditional Model Before server virtualization, most IT installations had a straightforward architecture with a limited number of servers connected to a storage system. Each server typically ran one application and the overall amount of I/O activity was manageable for the storage system, as shown in Figure 1. Figure 1: Typical configuration before server virtualization The practice of matching one application to one server allowed administrators to easily deploy or upgrade servers and applications without disrupting other server operations. However, the need to provision servers to meet peak workload demands resulted in over-provisioning and dramatically low server utilization rates.
4 Why Virtualization? A study by the Yankee Group points to the following reasons for virtualization deployment. 43% Reduce infrastructure costs 18% Easier, more flexible application deployment 15% Improve server utilization rates 12% Consolidate physical space 10% Faster server and application provisioning times Source: http://vmblog.com/archive/2006/07/04/942.aspx As indicated from these survey results, IT departments are primarily concerned with cost savings and consolidation. These important and worthwhile goals cannot be achieved without a comprehensive approach. Unfortunately, many projects to implement server virtualization and consolidation get underway without full consideration of the looming impact to storage performance.
5 Unintended Consequences: I/O Bottlenecks While server virtualization solutions have reached broad market adoption, the corresponding products and tools on the storage side have not kept up. As a result, a dramatic increase in the number of servers (or virtual servers) accessing a single storage system leads to the virtual I/O trap. The I/O contention and bottlenecks are due to the fact that while it is relatively easy to add another server to a given configuration, it is far more difficult to add corresponding storage horsepower. Consolidated [virtualized] servers can drive very dense I/O profiles that are very different than your garden- variety file serving... Chuck Hollis, Vice President Technology Alliances, EMC 2 Figure 2: Typical configuration after server virtualization 2 Source: http://chucksblog.typepad.com/chucks_blog/2006/12/vmware_virtual_.html
6 Understanding Capacity And Performance Storage solutions can be characterized on two general traits: capacity and performance. Frequently these areas overlap, but they represent distinct focus points when architecting storage deployments. Certainly there are other traits such as functionality and ease of use, but these represent higher layers than the basics of how much can be stored, and how quickly can applications get information in and out of available storage capacity. While it is relatively easy to add additional capacity to many storage solutions, it is far more difficult to add performance, specifically higher throughput and lower response times. There are few storage solutions that can expand their number of ports (a rough measure of throughput potential) as quickly as they can add capacity. And there are even fewer solutions which enable users to significantly reduce the access time, a result largely due to the nature of slower, mechanical disks. The end result of adding virtual server workloads to a single storage system without a corresponding boost to the storage performance is the server-storage performance gap. 3 Virtual Pitfalls But there are pitfalls to creating virtual machines. One is the added temptation to create more and more virtual machines......while CPU clock cycles, memory capacity, and storage are available in prolific quantity, I/O bottlenecks still exist. In fact, bottlenecks can be exacerbated if you create too many virtual machines on one physical machine. The risk is that all the virtual machines will attempt to access memory or storage simultaneously. If this happens, virtualization can actually reduce the server's performance. David Gilbert, Dr. Dobb s Journal, Jan 22, 2007 4 3 For more information, see The Server-Storage Performance Gap, a Violin Memory whitepaper. 4 Source: http://www.ddj.com/dept/64bit/196902422
7 Workload I/O Impact On Disk Performance Increasing the workload on any storage system will impact the overall performance, but this effect can be exacerbated when many servers with unique applications are directed towards a single storage system. Figure 3 demonstrates the effects of adding multiple application workloads to a single storage system. As indicated, a single application has a moderate impact on disk I/O, and the mechanical spindles can still deliver a reasonable storage system response time keeping the I/O queue in check. As the number of applications increases, and the randomness of the I/O pattern intensifies, the storage system ultimately reaches a debilitated state in which it can no longer deliver sufficient response time, as indicated by an excessive I/O queue. Figure 3: Application and virtual machine effects on I/O queues and storage response times
8 Sustaining Virtual I/O With Flash Memory Arrays Fortunately, there are ways to enjoy the benefits of server virtualization and maintain the performance needed for today s real time applications. Since the dense I/O profiles that accompany virtualized environments drive heavy loads of random storage access, the mechanical disks in the storage system have trouble keeping up. This imbalance is a result of increasing power at the server and CPU layer, with only nominal increases in performance at the disk layer. The solution to sustainable Virtual I/O is the use of flash memory. Violin s memory arrays are built to handle sustained writes and reads with spike-free latency. These attributes enable hundreds or thousands of virtual servers to share a common storage infrastructure. The infrastructure can be based on Fibre Channel, iscsi or NFS protocols. Where NFS is used, NFS caching makes use of high-capacity, high-speed cache memory that is shared as a network resource, serving I/O intensive requests directly from cache. This process accelerates I/O operations by orders of magnitude delivering real time response and significantly higher I/O operations per second (IOPS). NFS caching can be implemented with products such as the Violin Memory scalable caching technology, vcache. A vcache system can be attached to an existing NFS network storage environment and provide an immediate acceleration boost to applications. Unlike many other methods to scale NAS performance such as changing to parallel file systems, or replacing existing equipment with new storage systems, Violin Memory solutions work within existing environments, allow IT managers to retain all current storage systems and OSes, and do not require any change to storage management software. Unlike embedded caching solutions, the vcache technology scales to 15 TB or more at a much lower cost per GB. This allows the entire active dataset to be cached. Each 3U Violin Memory Array provides 7.5TB of SLC flash storage capacity and over 200K sustained write IOPS. This approach significantly reduces risk and provides immediate returns to productivity and data center efficiency. Over-provisioning storage becomes a thing of the past, as caching provides the IOPS and throughput needed to satisfy the most demanding server virtualization environments. A sample centralized storage caching deployment with vcache is shown in Figure 4.
9 Figure 4: Restoring storage health in virtualized environments with vcache I think that, in the long term, we'll find high- end NAS much more friendly for high- end [virtual migration / distributed resource scheduling] farms than today's SANs. Chuck Hollis, Vice President Technology Alliances, EMC 5 The same Violin Memory Arrays can also be used in iscsi and Fibre Channel modes. In this configuration, specific high performance LUNs can be assigned to the flash memory arrays for significant read and write acceleration. Long-term archival of data, software and machine states can be stored on traditional SATA storage. This two tier architecture is also very cost effective. Conclusions Server virtualization offers tremendous benefits to enterprise customers, particularly the ability to reduce overall infrastructure costs and increase flexibility. But the sole focus on benefits at the server level leave looming consequences for a virtual I/O trap. IT managers must keep a complete view in mind when architecting virtualization solutions and closely monitor the performance requirements. When I/O bottlenecks arise, flash memory arrays act as a perfect complement to server virtualization and can restore the proper balance to the overall computing and storage configuration. Violin enables server virtualization storage solutions through NFS caching, Fibre Channel (SAN) or iscsi storage. 5 Source: http://chucksblog.typepad.com/chucks_blog/2006/12/vmware_virtual_.htm
10 Violin Memory accelerates storage and delivers real time application performance with vcache NFS caching. Deployed in the data center, Violin Memory scalable vcache systems provide scalable and transparent acceleration for existing storage infrastructures to speed up applications, eliminate peak load disruptions, and simplify enterprise configurations. 2010 Violin Memory. All rights reserved. All other trademarks and copyrights are property of their respective owners. Information provided in this paper may be subject to change. For more information, visit www.violin-memory.com Contact Violin Violin Memory, Inc. USA 2700 Garcia Ave, Suite 100, Mountain View, CA 94043 33 Wood Ave South, 3rd Floor, Iselin, NJ 08830 888) 9- VIOLIN Ext 10 or (888) 984-6546 Ext 10 Email: sales@violin- memory.com www.violin- memory.com