Content-Based Caching by Infinio Maximizes I/O Workloads on vsphere NFS Datastores

Transcription

1 openbench Labs Jack Fegreus April 17, 2014 Benchmark Analysis Content-Based Caching by Infinio Maximizes I/O Workloads on vsphere NFS Datastores Storage Optimization in a Virtual Infrastructure 01

2 Executive Overview What s In This Document? openbench Labs tested Infinio Accelerator in our laboratory test bed to assess the affects of Infinio s content-based caching on I/O workloads of virtual machines in a vsphere 5.5 environment. We established our baselines and tested Infinio using three I/O workloads: Direct synthetic IO, using oblload v5 and oblfilegen Database-driven OLTP, using the TPC-E benchmark on SQL Server Enterprise-scale , using the LoadGen benchmark for 2,000 Exchange users Test Our tests showed the global shared cache created by Infinio Accelerators improved the performance of our test workloads both individually and when running concurrently including vmotion and failover scenarios. Furthermore, Infinio s data deduplication process allows the shared cache to have a much larger effective size than provided by the physical resources provisioned for it. Snapshot of Findings Synthetic IO workload TPC-E OLTP workload with SQL Server workload with Exchange Simultaneous mixed workload with Exchange and SQL Server Infinio Accelerator Workload Performance Test Results Results 90% of read IO offloaded from storage array Response time improved 1.8x 107GB effective cache size (16GB actual size) 37% of read IO offloaded from storage array Response time improved 1.7x 25.41GB effective cache size (16GB actual size) 50% read IO offloaded from storage array Response time improved 2.3x 50GB effective cache size (16GB actual size) 50% read IO offloaded from storage array Response time improved 2x 50GB effective cache size (16GB actual size) Our tests results were achieved running Infinio Accelerator on just two servers, which limited the total cache size to 16GB of physical RAM. In an environment with more servers, Infinio performance results would continue to scale with respect to a the growing physical cache base. 02

3 Content-Based Caching by Infinio Maximizes I/O Workloads on vsphere NFS Datastores Storage Optimization in a Virtual Infrastructure Minimizing VM Host Infrastructure Cost The mission of IT is to get the right information to the right people in time to create value or mitigate risk. Today, demands on IT continue to focus on support for activities with a distinct potential for revenue generation. Typically these projects involve the use of large database driven applications in an on-line transaction processing (OLTP) scenario, which is often augmented for multi-dimensional on-line analytical processing (OLAP). As a result, applications continue to drive double-digit growth in stored data, which puts data storage cost Infinio Accelerator 1.2 for NFS Datastore Performance Enhancement 1) Reduces IT Operations Complexity One management VM is installed for each vcenter Server. containment at the forefront of IT strategic planning. To optimize physical resource From the management Console, one Accelerator virtual appliance is installed on each utilization in host with NFS shared datastores for I/O acceleration. an expanding 2) Reduces Capital and Operational Costs applications Existing direct attached storage can be shared using NFS over TCP to provide vsphere environment, IT hosts with shared storage to support vstorage vmotion in a VI environment. executives have Accelerator VMs form a pooled cache from server memory rather than expensive Flash turned to softwaredefined virtual Memory devices 3) Hashes Disk Cache based on Data Content infrastructure (VI) Content-based caching creates a single-instance hash. technology, such as Content-based caching leverages common content to support increased VM density, VMware vsphere. offload NFS disk I/O, reduce NFS network traffic, and cut average access time. Nonetheless a 4) 5) Content-based caching is entirely independent of VM locality within a datacenter. Seamless I/O Workload Performance with vmotion and Disaster Recovery Operations Content-based, location-independent caching prevents I/O degradation when a VM is migrated to a new host or datastore for load balancing. Replica standby VMs can be booted in a disaster recovery scenario with no cache recovery required to maintain I/O performance. Maximizes Effective Cache Size Data deduplication expands single-instance cache capacity through pointers. Multiple Accelerator VMs automatically federate local caches into a unified global cache. VI introduces a new duality between physical hosts running a hypervisor and virtual machines (VMs) running applications, which increases Federated cache spans the datacenter, scales in proportion to the infrastructure, and the complexity of maximizes cache offload effectiveness via datacenter-wide deduplication of content. IT operations for administrators when optimizing application performance, minimizing resource contention, and protecting data. 03

4 To enhance read performance of VM workloads at small to medium enterprise (SME) sites, version 1.2 of Infinio Accelerator implements a very cost-effective, location-independent, data caching scheme for NFS-based datastores. Infinio Accelerator does not require a substantial capital investment in NAND Flash Memory devices, such as solid state drives (SSDs) and high-performance PCIe flash cards. Infinio provides IT with a pure software solution for accelerating reads in I/O workloads on VMs, which is precisely the business issue that IT needs to address. From the perspective of a Line of Business (LoB) executive, business applications running on VMs are the items that need to be addressed in any Service Level Agreement (SLA) entered into with IT. For IT, this LoB focus limits the scope of SLA issues and often makes solutions far more tractable. Caching Evolves from Location to Content Long before VI constructs became common parlance among IT administrators, I/O workflows were being accelerated on physical systems using disk controllers that had a firmware-based caching scheme based on disk block addresses designated by logical block numbers (LBNs). With dedicated disks on a single physical system, a small fixed cache of logical disk blocks populated using LBN request frequency reduced average data access time, while a system executed a given number of I/O operations per second (IOPS). DAS I/O Performance oblload: I/O Transaction Latency Average Access Time (ms) OPENBENCH LABS Dell R710 SAS RAID I/O Operations per Second (IOPS) We used our oblload benchmark to test well-controlled I/O workloads. We provisioned a 100GB disk with 25 percent unique (75 percent redundant) data. We ran our initial test on the RAID-5 SAS array used to provision NFS volumes. The tipping point between IOPS and average access time occurred around 13,000 IOPS. At that point, a 13% increase in I/O activity to 15,000 IOPS resulted in a 40% increase in average access time. Every time a data block is accessed by an electromagnetic disk drive, two distinct overhead delays occur: a seek time delay the time needed to position the read/write head laterally along the radius of the disk and a rotational latency delay the time spent waiting for the target data block to rotate into position under the head. Today s high-speed servo motors minimize those mechanical delays until a large queue of outstanding I/O requests builds up. At that tipping point, the relationship between average access time and IOPS execution becomes distinctly nonlinear. Data access time takes much longer with a small increase in the IOPS rate. As a result, a data block cache only needs to reduce a small number of the read requests from the drive to measurably lower average I/O access time. In theory, this read optimization scenario is equally true for a VM host; however, in practice read optimization of multiple independent workflows is significantly more complex. To minimize the overhead needed to enable a host to run multiple VMs simultaneously, the VMware ESXi hypervisor grants each VM exclusive access to its virtual disk files in a host datastore, which makes VM access to virtual disk files on a host datastore analogous to physical server access of logical disks on a local storage subsystem. 04

5 A n Infinio Accelerator VM is able to use its content-based hash to uniquely represent data common to multiple VMs whether from an OS, an application, or a user in order to minimize physical reads on multiple VM disks on multiple host datastores. With each VM in full control of its logical disks, light-weight disk caching software based on an LBN hash can be very effective, when run within an individual VM. Nonetheless, the independence of VM virtual disks makes the scaling of an LBN-based cache for a single VM into a solution supporting multiple VMs a complex problem that is often resolved by growing the size of provisioned cache geometrically with the number of VMs. From an LBN perspective, the blocks requested by two different VMs are mutually exclusive: There is no location commonality, even when two VMs run the same application. As a result, host caching solutions that rely on data block address patterns require a significant capital investment in SSDs or PCIe flash cards simply to provision a cache with the same effective size as a content-based cache using server RAM. Content-based Caching In Real Time To optimize reads in I/O workloads for multiple VMs, Infinio takes a radically different approach to caching. Rather than classify data blocks by LBN addresses, Infinio deploys one Accelerator VM appliance per datacenter host to implement a global singleinstance cache with a hashing scheme based on data content. Each host s Accelerator handles all of the VMs utilizing NFS datastores imported by the host. In addition, a single Infinio Management VM provides a Web Console to monitor and control the read acceleration activities for all of the NFS datastores imported into a vsphere datacenter. The Web Console presents a list of all NFS datastores present in the vcenter datacenter and automates installation of an Accelerator VM on any host that shares a datastore, but does not have an appliance present. What s more, the Infinio Web Console automatically discovers new NFS-shared volume imported on a host. As data blocks are ingested by a VM, the Infinio Accelerator associated with the VM s host creates a hash of the block content and deduplicates redundant content already in the cache. By classifying and deduplicating the content of data blocks in real time, Infinio extends critical data caching constructs in a way that caching schemes based on LBNs do not permit. A content-based hash does not restrict block data to a particular VM. An Infinio Accelerator VM is able to use its content-based hash to uniquely represent data common to multiple VMs whether from an OS, an application, or a user in order to minimize physical reads on multiple VM disks on multiple host datastores. With a hash that can point to data blocks across VMs, Infinio has the key to creating a global single-instance cache, which adds a new dimension to I/O workload performance. Furthermore, by deduplicating the data, an Infinio Accelerator directly increases the effective size of its cache with respect to the amount of data stored. Depending on data commonality and the number of VMs, pointers to duplicate data can extend cache effectiveness for a single appliance by an order of magnitude beyond provisioned memory. 05

6 Infinio Accelerator Cache Federation The current version of Infinio implements its caching scheme exclusively on VMs with virtual disks resident on NFS-shared datastores. To implement its radical, content-based, read-caching scheme, Infinio creates a very sophisticated federated infrastructure using VM appliances. While each Infinio Accelerator VM maintains exclusive control over the read I/O caching process for each VM running on the appliance s assigned host, multiple Accelerator VMs in a datacenter federate their local cache into a global singleinstance cache. When IT creates an NFS domain and shares each NFS volume with all of the hosts in a vsphere datacenter, Infinio leverages cache federation to make the base physical cache for each Accelerator equal to the sum of the cache on each Accelerator. As Infinio s federated cache spans to encompass the entire ESXi datacenter, Infinio simultaneously extends the effective size of the extended base cache via data deduplication of content. As a result, Infinio provides an I/O offload capability that scales in proportion to the IT infrastructure. On installation, Infinio creates a public port connected to the Accelerator VM to intercept NFS traffic. During the installation process, Infinio mirrors the NIC teaming policy on the host s VMkernel port used for NFS traffic on the public Accelerator port. Next, Infinio reconnects the physical NICs from the VMkernel port to the public Accelerator port. Infinio completes the NFS traffic path by creating a private Accelerator port and a virtual NIC to connect to the host s VMkernel port. To federate Accelerator VM caches, Infinio creates a point to point (P2P) port group that mirrors the vmotion VLAN to pass cached data traffic across VM host boundaries. Running a synthetic benchmark on one host, with Infinio enabled, physical I/Os on the logical disk were reduced by 93 percent and inline data deduplication of data content increased effective cache size from 8GB to 51.2 GB. When the benchmark was run on two hosts, each host s Accelerator VM shared a single federated cache of 16GB with an effective size of 107.1GB, which enabled 99% of read requests to be offloaded NFS Test Bed Infrastructure Our goal was to assess how effectively Infinio is able to optimize I/O workloads of VMs using logical disk files on NFS-shared datastores in a vsphere 5.5 environment. 06

7 To accomplish this task, we set up a regime of tests that began with highly constrained synthetic benchmarks to establish a baseline for read performance. We then expanded our focus to application benchmarks for SQL Server and Exchange, which were performed in isolation and in tandem to simulate a fully functioning SME production environment. We built our test infrastructure on three servers: Two formed a vsphere 5.5 datacenter, while the third ran Windows Server 2012 with Hyper-V. On the Windows server, we configured four 1GbE NICs as two network teams with two NICs per team. We set up one team for standard LAN traffic and configured the other on a separate subnet for exporting NFS volumes to vsphere datacenter hosts. For storage, we provisioned an internal caching RAID controller with 15K SAS drives to create a 4TB direct attached storage (DAS) pool. To manage our NFS storage domain resources, which were shared among all hosts in our vsphere datacenter, we set up a Hyper-V VM running opensuse Linux We provisioned that VM with two virtual NICs one for each NIC team and three 1TB volumes from the RAID-5 DAS pool. We set up a second VM running vcenter Server on Windows Server 2008 R2 to manage the vsphere VI. 07

8 I T best practices minimize systems management costs in a VI by relying on instituting a high degree of commonality in VM deployment. To assess Infinio effectiveness with both new hosts using Intel s Data Direct I/O and older hosts using bus-based I/O, we set up a Dell R710 along with a legacy Dell R2950 for our datacenter. On each host, we configured a standard vswitch with four network ports explicitly allocated to host management, VM networking, NFS traffic, and vmotion. To utilize vmotion with NFS datastores, we exported all NFS volumes to every host in the datacenter. Testing Dynamic Workflow Scenarios To test the optimization capabilities of Infinio, we created thirteen VMs, with nine VMs located on NFS-shared datastores. Those nine VMs were used to generate three distinct workflows for direct synthetic I/O, database-driven OLTP, and enterprise . We drove our initial I/O workload using synthetic benchmarks in a tightly controlled environment. In these tests, our goal was to discover boundary conditions for VM I/O scalability in our test environment. We used two VM servers running Windows Server 2008 R2 to run our oblload benchmark and oblfilegen, a collection of ActivePython scripts that generate files with a specific level of data redundancy for evaluating data deduplication. The oblload benchmark generates I/O requests and measures average access time exclusive of any CPU overhead. The master oblload benchmark process creates a series of test iterations. On each iteration, the master launches a fixed number of background daemon processes, which generate 1,000 random LBN addresses for I/O requests defined by a predefined profile. The oblload I/O profile describes the size and distribution pattern of I/O requests for a set of designated test volumes. To simulate VM I/O on a datastore, we used a random LBN addresses pattern with oblload. While multiple database processes combine to scale a single I/O profile, multiple VMs randomize LBN address access, as each VM has a unique local I/O profile. As a result, an LBN-based caching scheme is much more difficult to implement with VMs. Rather than use a massive physical cache to collect randomized LBNs, Infinio uses smart algorithms to leverage the underlying commonality in VM data content to accelerate I/O workloads. IT best practices minimize systems management costs in a VI by relying on instituting a high degree of commonality in VM deployment. That s why we needed to explicitly characterize the content of the data blocks oblload daemons would access. For this task we utilized oblfilegen to populate each test volume with data that could be deduplicated in a 4 to 1 ratio. For our synthetic benchmark testing, we populated the 100GB work volumes of all test VMs with identical sets of files that were created with a 4 to 1 data deduplication ratio using oblfilegen. While I/O benchmark software typically uses a random number generator to create LBN addresses for I/O requests with no regard to the content of data blocks being accessed, Infinio leverages commonality in data content and applies inline data deduplication to that content to create a hash that maximizes caching effectiveness. As a result, working with purely random data, will not provide a realistic scenario for assessing Infinio performance. 08

9 T he degree to which the results of OLTP queries contain common information, such as account numbers, product codes, and transaction classifications, is a good predictor of how well an Infinio Accelerator VM will be able to offload physical disk read I/O, and improve database performance. After completing our baseline synthetic benchmark tests, we ran a series of application tests that involved a SQL Server VM modeling stock trading at a brokerage firm, and an Outlookbased processing test for 2,000 users with accounts on an Exchange VM. While the SQL Server and Exchange transactions were more complex than the direct I/O requests created by our synthetic benchmark, results from oblload still proved useful in predicting performance; however, we needed to accurately correlate predictions with content redundancy rather than I/O processing patterns. Testing TP Workloads with SQL Server To model an OLTP application, we configured a 20GB instance of the Transaction Processing Performance Council s TPC-E database, which simulates business at a brokerage firm. To support the benchmark database, we provisioned a quad-processor VM with 8GB RAM to run Windows 2012 Server and SQL Server In the TPC-E OLTP scenario, customers of a brokerage firm generate transactions to research market activity, check their account, and make stock trades. In turn, the company makes market transactions to execute customer orders and updates customer accounts. To model this brokerage activity, ten template transactions are weighted and customized using a random number generator. The customized transactions are executed on three sets of tables, which contain market, customer, and broker data. Audited TPC-E tests use only the number of trade order result transactions; however, to assess I/O workflow acceleration, we measured all transactions executed and calculated an overall average transaction response time. More importantly for Infinio workload optimization, the TPC-E database maintains a set of dimension and fact tables that would typically be the foundation for OLAP discovery on business history data. The TPC-E benchmark template set, however, generates a strict OLTP query collection. The data found in the TPC-E database dimension tables represent precisely the kind of common data that an Infinio Accelerator VM will cache as redundant content. The degree to which the results of OLTP queries contain common information, such as account numbers, product codes, and other transaction classifications, is an excellent predictor of how well an Infinio Accelerator VM will be able to offload physical disk accesses, lower the network bandwidth utilization of NFS traffic, and improve database performance. Each virtual user in our TPC-E tests continuously executed transactions created from the ten TPC-E templates, which were weighted in frequency by the TPC guideline. As with oblload, we launched users in groups and ran each group for a fixed period. We then examined the performance relationship between the total TPS rate and average response time with and without Infinio enabled. 09

10 F rom a workflow testing perspective, an Exchange configuration with 2,000 users running in parallel with our TPC-E benchmark provided the means to test the ability of the Infinio infrastructure to cache reads from multiple unique applications. Next, we examined the ability of a federated Infinio Accelerator infrastructure to support vmotion load balancing of host servers and Disaster Recover (DR) solutions for our SQL Server VM. Load balancing and DR operations create a very knotty problem for LBN-based caching schemes, as the movement of VMs within a VI significantly complicates the notion of locality of reference for data blocks. For load balancing, we enabled vcenter Server to automate host migration for the SQL Server VM using vmotion. For DR support, we used Dell AppAssure to set up a warm standby VM on a separate NFS datastore. We then examined the ability of the Infinio infrastructure to maintain the I/O performance of our TPC-E benchmark workload during a vmotion host migration and a DR fail-over operation. Testing Workloads with Exchange is another high profile corporate application being moved to a VI. Deeply embedded in IT infrastructure, broadly impacts corporate productivity, which makes reliability critically important for IT and LoB divisions alike. From a workflow testing perspective, an Exchange configuration with 2,000 users running in parallel with our TPC-E benchmark provided the means to test the ability of the Infinio infrastructure to cache reads from multiple unique applications. We configured three VMs running Windows Server 2008 R2 and one VM running Windows Server 2012 to support our I/O workload. Two VMs provided direct services. A quad-processor VM with 8GB of RAM and two 300GB drives ran Exchange 2010 sp3 and a Windows Server 2012 VM acted as the primary domain controller (PDC). The two additional VMs ran the LoadGen benchmark to set up an workload that created 2,000 user accounts in the PDC and leveraged their mailboxes on the Exchange server. Over an 8-hour work day, 131 Outlook-based transactions were generated for each user. A majority, 87 percent, of the transactions fell into three major categories: 61 percent received and read messages, 10 percent composed and sent messages, and 16 percent browsed calendars and contacts. This workload issued 9 Outlook TPS to the Exchange and PDC VMs and changed 6GB of data in mailbox databases and logs every hour. Baseline Benchmarking We began our synthetic benchmark testing using one VM. We then expanded the scope of the configuration with a second VM on the same host running on either the same NFS-shared datastore or a different NFS-shared datastore. Finally, we followed that configuration with the two VMs on different hosts with either the same or different datastores. For the final configuration, we used with two datastores on two different hosts. In these tests, the test VMs running oblload were maximally separated from a vsphere 10

11 topology perspective; however, from an NFS storage domain perspective, both VMs were running on the same RAID-5 volume used to create all of the logical volumes being exported from our Hyper-V VM as datastores in the vsphere datacenter. As a result, reads that were no cached had the same physical disk contention issues no matter whether the reads were directed to the same or different datastores. With the Infinio Accelerator VM active only on the vsphere host running on the Dell R710 PowerEdge host with DDIO, we measured distinct performance improvements following the launch of just five oblload daemon processes. By the time oblload launched 100 daemons, the Infinio Accelerator was fulfilling 90 percent of reads from cache and had cut average access time for reads in half. In particular, the Accelerator was sending an average of 42MB of data per second to the VM running oblload to satisfy benchmark read requests. At the same time, the Accelerator while it was receiving just 4.5MB of data per second from the NFS volumes. Moreover, via data deduplication the Accelerator cache represented over 51GB of data 5.4 times more data than the provisioned cache capacity. We completed testing by installing an Infinio Accelerator on the vsphere host running on the Dell PowerEdge R2950. Immediately upon launching the second Accelerator, Infinio automatically federated the two local 8GB caches into a single 16GB cache. When we then launched a second test VM running oblload on a cloned work disk on the Dell PowerEdge R2950 host, the Infinio Accelerators worked together to increase the effective size of the federated cache to over 116GB. oblload Accelerator Performance We ran oblload on two VMs on separate hosts and separate datastores, but that had common OS, application, and work data. The Infinio Accelerators on each host leveraged the high level of data redundancy to extend the effectiveness of the federated cache and offload over 99 percent of the I/O from each datastore. Using the vcenter Web client, we verified that each Accelerator was receiving about 1MB of data per second from the NFS storage server to support the test VM that was accessing data at an average of 40 to 50 times that rate. What s more, an average of 5,000 IOPS were being cached for each VM running oblload. 11

12 oblload Synthetic Benchmark Acceleration Average Response Time (ms) OPENBENCH LABS oblload v5: I/O Transaction Latency One NFS Datastore 1VM With Infinio 1VM Without Infinio I/O Operations per Second (IOPS) oblload v5: Workload Scalability IOPS=4067 IOPS=548 From the perspective of an LoB user, improved IT processing, such as lowered NFS overhead, has little intrinsic value. IT infrastructure constructs need to be translated directly into application performance advantages. For users of oblload, the Infinio Accelerator created an environment that provided measurably higher I/O throughput. The value of this benefit for users was manifested in higher IOPS rates and quicker average access times. In addition, more workloads and applications were serviced with the same hardware, thereby providing a key economic benefit that is a critical success factor for every CIO and an LoB executive alike. With a single Infinio Accelerator VM offloading read requests from the NFS storage subsystem to the Accelerator s cache, the benchmark application sustained a read request rate of 6,000 IOPS while maintaining an average access time that was under 1 ms. As a result, Infinio s content-based caching scheme plays a key role in enabling oblload to perform more useful work, which for our oblload benchmark can be construed as the delivery of data. By enhancing I/O workload scalability, the Infinio appliance provides a very cost effective way for IT to increase resource utilization by increasing the density of VMs running within a datacenter. In particular, we used the same NFS datastore to support two VMs on different hosts with no degradation in I/O workload. Running oblload on two VMs on separate hosts, the federated Infinio cache satisfied over 99 percent of disk reads both for both VMs. In OPENBENCH LABS One NFS Datastore with Two Hosts 2VMs, 2 Hosts With Infinio 1VM With Infinio 1VM Without Infinio I/O Operations per Second (IOPS) Number of Simultaneous Daemon Processes With a single Infinio Accelerator offloading I/O requests, oblload rapidly processed more I/O requests with very small increases in average access time 0.5 ms or less. Only when I/O processing exceeded 7,000 IOPS did the relationship between IOPS and access time begin to degrade and culminate with access time increasing asymptotically as oblload approached an 8,500 IOPS rate. While oblload was able to process up to 14,000 IOPS without Infinio, it was necessary to launch more than 100 oblload daemon users to generate that workload. More importantly, accelerated scaling of IOPS with access time is linked to the efficient scaling of I/O workloads as users are added. As the number of oblload daemons increased from 2 to 15, the IOPS rate increased by 29 percent without Infinio caching. The same increase in daemon users with Infinio caching resulted in a 149 percent increase. In particular, 15 daemons were able to retrieve 185 percent more data with Infinio caching support than they were able to retrieve without Infinio. More importantly, when we used the same NFS datastore to support two VMs on different hosts, we measured no degradation in the I/O workload of either VM. Moreover, the federated Infinio cache satisfied over 99 percent of the reads made by both VMs. 12

13 particular, the two Infinio Accelerators were able to offload an average of 10,000 IOPS. As a result, the two VMs processed upwards of 14,000 IOPS from the perspective of an LoB user. In terms of user data throughput, the VM on the Dell R710 with DDIO streamed reads at an average of 50MB per second, while the VM on the Dell R2950 streamed reads at an average of 40MB per second. Meanwhile, the total I/O load on the physical RAID-5 array utilized by the NFS storage domain remained under 2MB per second. From an I/O scalability viewpoint, the base storage resource showed little physical utilization. Accelerating SQL TP Applications The constructs of oblload that were exploited by the Infinio Accelerator VM to optimize the benchmark s read workload have analogous constructs in the more complex transaction processing applications deployed to support business critical processes. Complicated SQL and Outlook application transactions involve multiple I/O accesses and CPU-intensive activities; however, the plethora of interactions result in a limited set of outcomes. Moreover, these outcomes often become the defining metrics of an SLA between IT and an LoB organization. In our TPC-E benchmark, each virtual user executed SQL queries that simulated a stock trading business scenario. The database supporting this scenario had 33 tables populated with 18GB of data. All queries were generated using a set of ten weighted templates customized using a random number generator. In addition to containing all of the unique active trading information of customers and a detailed trading history of each customer, the database also contained a set of fact and dimension tables with common data, including company names, stock exchanges on which companies traded, client identification, broker identification, and tax codes. OLTP SQL Server Accelerator Performance The presence of dimension and fact tables in the TPC-E database is a key indicator of potential content redundancy in database query results. In practice, there was sufficient redundant data returned by TPC-E queries for the Infinio Accelerator to utilize content-based caching to offload about 37 percent of the database s physical disk reads. While running the TPC-E query workload, the Infinio Accelerator leveraged common query data, to offload an average of 37 percent of physical reads from the NFS-shared datastore supporting the SQL Server VM. The vsphere Web Client reflected Accelerator caching with a 35 percent difference in the throughput rates for network data transmitted from the Accelerator to the host and network data received by the Accelerator from the NFS storage domain server. While scaling out TPC-E query processing for a large number of users, logical database reads, which are serviced by data pages resident 13

14 in the SQL Server buffer cache, represented over 98 percent of all read operations. Nonetheless, the impact of disk-block caching by the Infinio Accelerator was greater processing the TPC-E benchmark workload, than it was when processing the oblload synthetic workload. Unshackling SQL Server Synergies The oblload benchmark was explicitly designed to isolate and independently stress a disk subsystem. On the other hand, the TPC-E benchmark stresses CPU, memory and I/O resources. As a result, the TPC-E benchmark highlights resource interrelationships that complicate bottleneck conditions. In particular, as users were added by the TPC-E test script, CPU bottlenecks were manifested in dramatically longer maximum I/O wait times and a growing number of blocked processes. With physical reads representing less than two percent of all reads and only a portion of the physical reads taking an excessive amount of time, the response times of queries retrieving cached data dominated the average query response calculations. Only when long I/O waits dramatically skewed the tail points in response-time collections did the average increase rapidly. For example, with 102 users and no read caching, maximum response time of physical reads reached more than three seconds. Nonetheless, the average response time was a far less dramatic 210 ms as logical reads, dominated I/O requests. TPC-E Performance Improvement The relative paucity of physical reads did not negatively impact the ability of an Infinio Accelerator to offload those reads. Infinio caching depends on data redundancy and not I/O activity. In particular, redundancy in the data returned by the TPC-E queries, enabled Infinio to offload 37 percent of the SQL Server physical reads to cache. In so doing, Infinio accelerated about 37 percent of the long running I/O operations that were creating the CPU wait states constraining SQL Server processing. In particular, the peak response time of tail points in response time collections was reduced from 3,110 ms to 486 ms and average response time dropped 50% to 105 ms with Infinio caching. Scaling out to 104 user processes, 1.6 percent of reads 194 IOPS were physical u. Physical reads remained a key workflow factor, however, as maximum I/O wait time v extended upwards of 3,000 ms, and blocked processes w reduced the level of CPU utilization x by SQL Server to less than 25 percent. 14

15 Nonetheless, there is an alternative to improving database performance by caching SQL Server physical reads with an Infinio Accelerator. We could lower and possibly eliminate physical reads by expanding the SQL Server buffer cache by adding memory to the VM running SQL Server or by assigning a buffer file resident on a flash memory device such as a solid state disk (SSD) a recommended practice for SQL Server To assess this option, we provisioned ran the SQL Server VM with an additional 4GB and 8GB of RAM, which brought total VM memory up to 12GB and 16GB respectively. SQL Server TPC-E Acceleration Average Transaction Response Time (ms) OPENBENCH LABS TPC-E: Average Transaction Latency With Infinio Without Infinio Without Infinio Plus 4GB RAM Without Infinio Plus 8GB RAM Total TPC-E SQL Transactions per Second (TPS) Data redundancy within the results of our TPC-E queries enabled the Infinio Accelerator VM to offload 30 to 35 percent of physical reads and cut average access time nearly in half. More importantly, Infinio caching reduced the average transaction response time for all TPS-E transaction loads, as well as the transaction execution rate for a given response time. We were also able to extend query processing by increasing VM RAM by 4GB and 8GB increments. While increasing the amount of RAM by 8GB decreased physical I/O by about 30 percent a decrease in physical I/O comparable to the level of I/O offloaded by Infinio average access time was not reduced for all TPS loads. For LoB users, the value proposition of Infinio caching is centered in the ability to run greater database transaction workloads. By offloading 1 out of 3 physical reads from the NFS datastore to the distributed cache, Infinio lowered the probability that long running read requests would increase CPU resource waits and stall SQL Server process queues. In this way, the Infinio Accelerator was able to free CPU cycles to process more queries. More importantly, LoB executives will often insist on an SLA linking a transaction performance level with the response time of a business critical OLTP application. In our NFS-based test scenario, for an average response time of 20 ms, we were able to deliver a 30 percent higher transaction rate 130 TPS with Infinio compared to 100 TPS without Infinio acceleration. Moreover, we were able to meet a 100 TPS rate with a 4 ms average response time running with Infinio caching as compared to 20 ms without Infinio caching. While we were able to consistently increase the transaction load by increasing the SQL Server buffer cache, we did not measure decreased average access times for all transaction loads, as we did with Infinio caching. 15

16 Caching Continuity For business critical applications, many LoB executives rank the issue of business continuity even higher than performance. To prevent server downtime from disrupting guest VMs and enable dynamic VM load balancing across physical host, vsphere vmotion enables live migration of VMs from one host to another without impacting application availability on guest VMs. SQL Server vmotion Host Migration We tested vmotion host migration with the SQL Server VM, while running the TPC-E workload. As vcenter changed the VM s host from the Dell R2950 to the more powerful Dell R710, the Infinio Accelerator on the Dell R2950 simultaneously passed the VM s I/O handling to the Accelerator on the Dell R710. To make the vmotion process transparent to the TPC-E workload, the Infinio Accelerator VMs leveraged the federated cache to maintain a 34 percent I/O offload rate, which enabled the TPC-E benchmark workload to maintain an average query processing rate of 70 TPS. The movement of VMs within a VI, however, complicates the notion of data locality in any LBN-based caching scheme. To test vmotion in an Infinio content-based caching infrastructure, we migrated the SQL Server VM from the Dell R2950 host to the Dell R710 host. This process also caused control of VM acceleration to switch from the Infinio Accelerator on the Dell 2950 host to the Accelerator on the Dell R710. The process was entirely transparent for the perspective of the Infinio Management console; however, we 16

17 were able to use the vcenter Web console to monitor the hand over of VM caching, which proceeded with no statistically significant changes in I/O workload. Third-party data protection software also plays a SQL Server VM DR Failover major role in ensuring business continuity for business critical applications. With respect to implementing a DR plan, the ISO standard for a Service Level Agreement (SLA) on business continuity reflects this concern by defining a Recovery Time Objective (RTO), which sets a limit on the length of time taken to recover a system from a downtime event and a Recovery Point Objective (RPO) which limits the acceptable amount of data lost.. To test the ability of the Infinio infrastructure to handle a DR plan, we created a warm standby VM on a different NFS-shared datastore from the datastore used for our SQL Server VM. To minimize data loss, the warm standby VM was kept up to date using VM snapshots created after every incremental backup of the active SQL Server VM. In the event of a system failure on the master VM, our DR plan called for booting the fully functional warm standby, which represented the SQL Server VM at the time of its last incremental backup. While running the TPC-E query workload, we tested a DR scenario in which a standby VM on a different datastore was booted into the VI environment with the same credentials as the master VM. In this DT scenario, the Infinio Accelerator VM leveraged common query data and immediately provided identical I/O workload acceleration for the standby VM. In particular the standby VM immediately benefited form a 37 percent I/O offload rate while it processed queries at a rate averaging around 100 TPS. With respect to Infinio, the only issue was how would the Infinio Accelerator VM deal with a new instance of a system running the TPC-E benchmark workload on a completely different datastore. In theory, the structure and content of the queries used to retrieve data on the standby VM were identical to the queries used on the master. We therefore expected to encounter no flushing of cache on the Infinio Accelerator with the change in server. In practice, real world performance behaved precisely as the theory projected. Immediately following VM boot up, the warm standby VM began processing an I/O workload that was identical to that of the master VM. For both servers, the Infinio Accelerator offloaded 37 percent of the physical database I/Os and both VMs exhibited query TPS rates that were statistically identical. Accelerating Exchange Workloads In our final test series, we added two VMs to our NFS datastore to provide an service for 2,000 users. To run Exchange 2010 sp3, we used a quad-processor VM with 8GB of RAM and two 300GB volumes for two mailbox databases. The second VM was configured as a primary domain controller supporting the 2,000 user accounts. We used two additional VMs to create an workload using the LoadGen benchmark. 17

18 To exercise the Exchange configuration, our test workload generated nine Outlook 2007 transactions per second. Over an 8-hour work day, this amounted to 131 Exchange transactions for each of the 2,000 users. For each user, 77 percent of their Exchange transactions were read-centric and fell into three primary categories: 61 percent received and read messages, 9 percent browsed calendars, and 7 percent browsed contacts. Every hour, this transaction load changed 6GB of mailbox database and log data on the Exchange server VM. In addition, transactions were validated at the PDC before being processed on the Exchange VM. We first ran Exchange for several hours with no other workloads active. We then ended with a test running the Outlook workflow in parallel with the TPC-E benchmark. Our goal was to stress the Accelerator VM s ability to cache and deduplicate data from multiple application streams. After several hours of running the constant Outlook workload with variable TPC-E workloads, The Infinio Accelerator VM was still offloading 50% of datastore reads and maintaining an effective cache size of 50.4 GB, while using only 16GB of actual RAM. Exchange LoadGen Acceleration While running the LoadGen test for two hours, the Infinio Accelerator VM maintained a very consistent caching rate that offloaded two out of every three I/Os. As a result, the average access time was reduced by 57 percent. We then ran our TPC-E benchmark simultaneously with the Exchange workflow on the same NFS-shared datastore. The Infinio Accelerator VM was able to offload 50 percent of both independent workloads. Customer Value For many CIOs, a critical top-of-mind issue is how to reduce both the operating and capital costs of IT operations. With procuring, provisioning, and managing high-end storage arrays now the biggest cost driver at most IT sites, the need to optimize hardware utilization is of particular importance. In theory, the easiest way to solve the hardware utilization issue is by maximizing the density of VM servers, whether deploying tens or even hundreds of VMs in a VI environment. In practice, however, the scalability of additional VMs is very limited using existing storage without some form of augmentation to meet intensifying I/O demands. 18

19 Infinio Accelerator Feature Benefits 1) Infinio Data Caching Utilizes a Content-Based Hash: Infinio maintains a contentbased hash for cached data, which enables the implementation of cross VM caching of common content and data deduplication to extend the effective storage capacity of the physical cache by upwards on an order of magnitude. 2) Infinio Federates Caches of Multiple VM Appliances: Multiple Accelerator VMs utilize vmotion network ports to federate the local single-instance caches into a unified global single instance cache that works across vsphere hosts. 3) Infinio Caching Enhances VI scalability: By reducing average data access time, offloading local disk I/Os, and reducing network traffic, Infinio enables IT to deploy more VMs on hosts and run larger, more active workloads on VMs. 4) Infinio Content-Based Caching supports vmotion Migration and 3 rd Party Failover: By hashing data based on content and federating local caches, Infinio creates a datacenter-wide caching infrastructure that is entirely independent of VM location. To enhance read performance of VM workloads, version 1.2 of Infinio Accelerator implements a costeffective, locationindependent, data caching scheme for NFS-based datastores. Infinio provides IT with a pure software solution for accelerating I/O workloads to address operating costs. Using Infinio, there is no need to augment a high-performing storage system with a large expensive flash system using traditional data caching technologies. By using a content-based hash for data, Infinio is able to leverage the common data contained in VMs. While multiple instances of common content exacerbates cache capacity in an LBN-based cache, Infinio uses data deduplication to increase effective cache capacity by maintaining a global, shared, single-instance cache with pointers to duplicate blocks. As a result, common content, within and across VMs is the key to Infinio s effectiveness. More importantly, Infinio allows IT to easily transition into an application centric world, in which IT must meet SLAs crafted by LoB executives. To this end, Infinio provides I/O workflow acceleration along multiple dimensions of data throughput including both average access time and IOPS volume. Through Infinio s unique capability to leverage common VM content, we were able to scale an OLTP application on a VM running SQL Server using multiple metrics in a more robust manner than was possible when the scaling via the SQL Server buffer cache. Moreover, through the federation of multiple Infinio Accelerator VMs on multiple hosts in a vsphere datacenter, Infinio was able to maintain a global single-instance cache that supported multiple shared NFS datastores and multiple high-activity applications. Jack Fegreus is Managing Director of openbench Labs; consults through Ridgetop Research; and contributes to InfoStor and Open Magazine. Previously he was Editor in Chief of Open Magazine, Data Storage, BackOffice CTO, Client/Server Today, and Digital Review. Jack also served as CTO of Strategic Communications, a consultant to Demax Software, and IT Director at Riley Stoker Corp. Jack holds a Ph.D. in Mathematics and worked on Lie Algebras the application of computers to symbolic logic. 19