REFERENCE ARCHITECTURE. PernixData FVP Software and Microsoft SQL Server

Transcription

1 REFERENCE ARCHITECTURE PernixData FVP Software and Microsoft SQL Server 1

2 Table of Contents Executive Summary Solution Overview Software Components PernixData FVPTM Software Microsoft SQL Server and VMware vsphere... 4 Hardware Components Server and Network... 5 Storage Capacity Storage Performance Test Results HammerDB TPC-C (OLTP) HammerDB TPC-H (DSS) System Details and Configurations FVP Software Microsoft SQL Server Server and Network Configuration Storage Configuration vsphere Configuration Infrastructure and Management Configuration Conclusion Appendix A: HammerDB TPC-C Test Methodology and Detailed Results Appendix B: HammerDB TPC-H Test Methodology and Detailed Results Appendix C: Bill of Materials References

3 Executive Summary Like many other applications, there are numerous benefits to virtualizing Microsoft SQL Server. Often, the simplified operations and lowered costs enabled through physical server consolidation can be reason alone to virtualize. Add the benefits of hardware independence and flexibility and the ability to easily scale-up (or down) by allocating more (or less) resources and the decision to virtualize SQL Server becomes an easy one. However, misconfigured or mismanaged virtualized environments, more often than not, lead to poor database performance. When many virtual machines (VMs) access the same shared storage, contention for resources (disk and/or CPU) can arise in the storage devices and fabric. This creates storage I/O bottlenecks, which result in poor database performance (regardless of the database platform). This paper includes the results of a series of virtualized SQL Server database performance tests. These performance tests were completed using the HammerDB performance benchmark. For some of the tests, FVP software was deployed with the VMware vsphere stack running the SQL Server virtual machines, with no changes to operating system or SQL Server configurations. In the design detailed here, FVP software uses server-deployed PCIe solid-state disk (SSD) to increase Online Transaction Processing (OLTP) database transactions-per-minute (TPM) and Decision Support System (DSS) database query performance. To provide optimal performance for all tests, the storage array providing capacity was configured according to typical virtualized database server best practices. Test results revealed that PernixData FVP software eliminated storage I/O bottlenecks and significantly decreased I/O latency, allowing full CPU utilization and resulting in more than 2x OLTP TPM and 10x faster DSS query performance. Results show that with FVP software installed, write bursts associated with OLTP database transactions are absorbed into low latency server-side flash or RAM. This leads to substantially higher database transaction commit rates. FVP also allows SQL Server to leverage the throughput delivered by server-side resources, leading to a greater degree of parallelism, increased processor utilization, and faster DSS database query performance. 500k 400k OLTP Transactions (Higher is better) 50k 40k DSS Query Total Completion Time (seconds) (Shorter is better) 300k 200k More than 2x Increase 30k 20k 10x Faster 100k 10k 0 WITHOUT PERNIXDATA FVP WITH PERNIXDATA FVP 0 WITHOUT PERNIXDATA FVP WITH PERNIXDATA FVP Figure 1. OLTP Transactions Per Minute Figure 2. DSS Query Total Completion Time 3

4 Solution Overview The Microsoft SQL Server solution with FVP software combines one of the most widely deployed database platforms in the world with the industry s premier platform for storage intelligence. By eliminating storage bottlenecks and minimizing noisy neighbor effects, the FVP-enabled solution allows for a high performance SQL Server deployment on a single shared VMware vsphere hosting platform. In addition, as requirements change, SQL Server performance and capacity can be discretely scaled using a choice of many server and storage systems. Software Components PernixData FVP Software PernixData FVP software puts storage intelligence into server flash and RAM to accelerate the performance of virtualized applications. By putting storage intelligence into high speed server media, PernixData FVP software delivers the following benefits: Faster SQL response times. Latency of database queries is reduced by 10x on average when data is served from the local host, resulting in more transactions in any time period. Performance that scales with demand. FVP software ensures databases get the I/O resources they need, even when demand spikes. This means your SQL Server performance always remains consistent even during periods of peak usage. Cost-effective I/O acceleration. Instead of buying expensive storage capacity to get better SQL performance, you can add performance when and where you need it most. An FVP solution can save up to 95% when compared to alternative storage upgrades. Microsoft SQL Server and VMware vsphere Microsoft SQL Server database installations can range from large data warehouses, to small, highly specialized application databases. Whether unlocking insights with pervasive data discovery or recording online transactions, each application has its own set of requirements for the database layer, resulting in multiple versions, patch levels, and maintenance processes. To meet these requirements, administrators often deploy SQL Server installations dedicated to each application. Unfortunately, application isolation can create significant waste of server resources. This waste becomes even more costly when requirements call for high availability. Fortunately, virtualizing SQL Server with VMware vsphere can allow the best of both worlds by simultaneously optimizing compute resources through server consolidation and maintaining application flexibility through role isolation. For smaller, specialized databases, vsphere offers high consolidation ratios and advanced resource scheduling features that allow SQL Server VMs to easily scale-up (or down) based on demand. This gives application owners the flexibility and performance they need while simplifying and lowering costs for the enterprise. vsphere has also shown the ability to run the most challenging SQL Server workloads with near native performance when given the proper resources. 4

5 Hardware Components Server and Network Cisco Unified Computing System is the first converged data center platform that combines industry-standard, x86-architecture servers with networking and storage access into a single converged system. The system is entirely programmable using unified, model-based management to simplify and speed deployment of enterprise-class applications and services running in bare-metal, virtualized, and cloud computing environments. The computing platform with Cisco UCS used for these tests includes: Cisco UCS C220 M3 Servers Cisco UCS 6200 Series Fabric Interconnects Cisco UCS 2000 Series Fabric Extenders Storage Capacity EMC VNXe Series is an affordable unified storage platform with solution-focused software that s easy to manage, provision, and protect. The VNXe series allows you to easily deploy private clouds with deep VMware integration. It s built for trademark ease-of-use across the entire storage lifecycle - from setup to management to support. It not only provides a single platform for file and block data services with Ethernet and Fibre Channel connectivity, but data efficiency services reduce your capacity requirements up to 50 percent. The storage capacity configuration used for these tests includes: EMC VNXe3200 Storage Array 8 x 200 GB MLC SSDs 16 x 600 GB 10K RPM SAS Disks 12 x 4 TB 7.2K RPM NL-SAS Disks Storage Performance The P420m* SSD is Micron s mainstream PCIe workhorse. The P420m* is like having a turbo button for your workload. It was designed to use Micron s RAIN and DataSafe technology to deliver constant, steady performance ideal for caching, cloud, and data center applications as well as blade enterprise servers. Micron offers HHHL card and hot-swappable 2.5-inch drive form factors with up to 1.4 TB of capacity. The storage performance configuration used for these tests includes: 2 x 700 GB Micron P420m* (one per host) *Since original publication, PernixData no longer recommends deployment of the Micron P420m in production environments. For further details, see PernixData Support KB or consult your PernixData account team. 5

6 Test Results Test results are summarized in the following section. For further details, see Appendices A and B. HammerDB TPC-C (OLTP) HammerDB TPC-C testing showed that FVP software significantly decreased storage latency, allowing full CPU utilization and resulting in more than 2x OLTP average transactions-per-minute (TPM). Two identical standard HammerDB TPC-C benchmark tests were run to simulate a typical OLTP database workload and compare performance with and without FVP software. Test 1 ran with FVP software enabled using PCIe SSD as an acceleration resource. Test 2 was run to measure performance without FVP software. In Figure 3 below, you can see uniform ~400k average TPM in Test 1 on the left half and inconsistent ~200k average TPM in Test 2 on the right half. Figure 3. HammerDB Transaction Counter - Test 1 (with FVP) on left half of chart; Test 2 (without FVP) on right half of chart HammerDB TPC-H (DSS) HammerDB TPC-H testing showed that the increased throughput and reduced latency provided by server-side flash and FVP software resulted in an average of 10x faster DSS query performance. Two identical standard HammerDB TPC-H benchmark tests were run to simulate a typical DSS database workload and compare performance with and without FVP software. Test 1 ran with FVP software enabled using PCIe SSD as an acceleration resource. Test 2 was run to measure performance without FVP software. In Figure 4 below, you can compare average (5 users) query completion time of all 22 DSS queries. 5k 4k Average DSS Query Completion Times (Shorter is better) WITHOUT PERNIXDATA FVP WITH PERNIXDATA FVP 3k 2k 1k Figure 4. Average DSS query completion times (seconds, lower is better) 6

7 System Details and Configurations FVP Software In this configuration FVP software virtualizes server-side flash across two VMware ESXi hypervisor nodes in a compute cluster and hooks these high-speed server-side resources into existing VM I/O paths to transparently reduce the IOPS burden on an existing storage system. The FVP software installation integrates FVP-enabled VMware ESXi hosts and the PernixData FVP Management Server within the VMware vsphere platform. The installation includes three components: The PernixData FVP Management Server software, installed on a Windows server. The Management Server coordinates operations between FVP and the ESXi host computers. The PernixData FVP UI Plugin, installed on servers where the vsphere Client is installed or accessible from within the VMware Web Client. The plug-in is the primary tool used to manage FVP. The PernixData FVP Host Extension software on ESXi hosts. For a virtual machine to take advantage of FVP s acceleration capabilities that VM must be running on a host computer where the FVP Host Extension module has been installed. Using the UI Plugin, a single FVP Cluster was created and vsphere infrastructure VMs were all accelerated using a single datastore policy. SQL Server virtual machines were accelerated using VM policies. Using this configuration method allowed all infrastructure VMs to be accelerated using a single policy, but also allowed for fine tuning SQL Server policies between different types of tests. All virtual machines were accelerated in Write Back mode using 1.4 TB of clustered PCIe SSD based flash. The following FVP configuration was used in this deployment: One Flash FVP Cluster All Virtual Machines 700 GB SSD flash per host Write Back (Local + 1 Peer) vsphere infrastructure VMs Write Back (Local Host) SQL Server VMs Note: Local host acceleration was chosen for SQL Server VMs because they are protected against host and flash device failure at the application layer by a SQL Server AlwaysOn Availability Group (see Microsoft SQL Server section). FVP Cluster Assignments The datastores were added to FVP Clusters as summarized in Table 1: DATASTORE/VM NAME FVP CLUSTER NAME RESOURCE TYPE RESOURCE SIZE vnxe_7k_lun00 (Datastore) Infrastructure PCIe PCIe SSD (NAND Flash) 1.4 TB mssql01 (VM) Infrastructure PCIe PCIe SSD (NAND Flash) 1.4 TB mssql02 (VM) Infrastructure PCIe PCIe SSD (NAND Flash) 1.4 TB Table 1. FVP Cluster Assignments 7

8 Microsoft SQL Server In this design Microsoft SQL Server 2014 Enterprise Edition was configured in a highly available manner to demonstrate the full capabilities of FVP software with SQL Server, but the same principles and performance aspects apply to single-vm configurations. With the release of SQL Server 2012, Microsoft introduced AlwaysOn Availability Group technology to SQL Server. This high-availability and disaster-recovery solution provides an enterprise-level alternative to database mirroring and other previous availability methods that relied on Microsoft Windows Server Failover Clustering (WSFC) and complex Raw Disk Mapping (RDM) configuration when used with VMware vsphere. When configured using Availability Groups (AGs), traditional WSFC and RDMs are no longer required and FVP software acceleration can be configured using the lowest latency local host write-back acceleration. An Availability Group supports a failover environment for a discrete set of user databases that failover together. An availability group supports a set of read-write primary databases and up to eight sets of corresponding replica databases in SQL Server For this configuration two Windows Server 2012 R2 Standard VMs (one per physical vsphere host) were used to host a single AG. This AG hosted two databases, one for OLTP testing and another for DSS testing, and both were configured for synchronous replication. This SQL Server installation included the following systems: Two Windows Server 2012 R2 Standard Servers with SQL Server 2014 Enterprise One Windows Server 2012 R2 Standard Server (Quorum File Share Witness) Note: An additional server was required to host a quorum file share witness to avoid a split-brain condition where both nodes believe they are the master. For this design, the Microsoft Active Directory server was used for this purpose. SQL Server VM Configuration An eight virtual CPU configuration was selected to fit within the physical host server s 8-core per socket NUMA boundary. Since datasets will be cached in memory by SQL Server during OLTP tests, only 16 GB of memory was allocated to the virtual machine to represent the most typical configuration where database size is larger than available memory. SQL Server binaries, Temp DB, log files, and data files were all placed on separate virtual disks that mapped to individual VMFS datastores, storage array LUNs, and storage pools for maximum performance. The datastore to storage array LUN mapping is summarized in Table 2: SQL FILE TYPE DATASTORE NAME STORAGE POOL RESOURCE TYPE Binaries vnxe_7k_lun00 NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk Temp DB vnxe_ssd_lun01, vnxe_ssd_lun02 SSD-Pool (RAID 5, 4+1) MLC SSD Logs vnxe_10k_lun03, vnxe_10k_lun04 SAS-Pool (RAID 6, 10+2) 10k RPM SAS Disk Data vnxe_7k_lun05, vnxe_7k_lun06 NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk Table 2. SQL VM datastore to storage array LUN mapping. 8

9 The Windows Server 2012 R2 VMs were additionally optimized according to VMware Best Practices for running large-scale workloads with intensive I/O patterns. These optimizations included the following: Multiple Paravirtual SCSI disk controllers and default queue depth increase (VMware KB ) Eager Zeroed Thick Disk Provisioning (SQL Server on VMware Best Practices Guide) 64K File Allocation Unit Size (Microsoft TechNet Reference) Virtual hardware of the SQL Server virtual machines was configured and optimized as defined in Table 3: ATTRIBUTE Server OS SPECIFICATION Microsoft Windows Server 2012 R2 (64-bit) VMware virtual hardware Version 10 VMware Tools version Virtual CPU Virtual memory vnics Virtual network adapter 1 Virtual network adapter 2 Virtual SCSI controller 0 Virtual SCSI controller 1 Virtual SCSI controller 2 Virtual SCSI controller (up to date) 8 virtual sockets 1 virtual core per socket 16 GB 2 (VMXNET3) VMXNet3 Adapter (Primary) VMXNet3 Adapter (Cluster Link) LSI Logic SAS Paravirtual (PVSCSI) Paravirtual (PVSCSI) Paravirtual (PVSCSI) Virtual Disk OS - VMDK 40 GB (Thin, vscsi 0:0) Virtual Disk Binaries - VMDK 40 GB (Thin, vscsi 0:1) Virtual Disk SQL Temp DB - VMDK 200 GB (Eager Zeroed Thick, vscsi 1:0) Virtual Disk SQL Logs - VMDK 2 TB (Eager Zeroed Thick, vscsi 2:0) Virtual Disk SQL Data - VMDK 2 TB (Eager Zeroed Thick, vscsi 3:0) Virtual CD/DVD Drive 1 Removed Table 3. SQL Server VM Configuration 9

10 Server and Network Configuration Tests were performed on Cisco UCS C220 M3 rack-mount servers designed for performance and density over a wide range of business workloads from web serving to distributed database. One vsphere cluster containing both host servers was deployed and contained all SQL Server and vsphere management virtual machines. To ensure high availability and provide the best possible test performance, vsphere DRS affinity rules were used to keep the two SQL Server VMs on separate vsphere hosts. Each Cisco UCS C220 M3 blade was configured as follows: Two Intel Xeon E GHz processors (16 cores total) 256 GB of RAM One VIC 1225 Dual Port 10 Gb SFP+ CNA One Micron P420m* 700 GB PCIe SSD The Cisco UCS 6248UP Fabric Interconnects used in this architecture provide both network connectivity and management capabilities for the system. They offer line-rate, low-latency, lossless 10 Gigabit Ethernet and Fibre Channel over Ethernet (FCoE) functions, and were built to consolidate LAN and SAN traffic onto a single unified fabric. Each 6248UP was additionally connected to a single Cisco Nexus 2232TM Fabric Extender to facilitate direct connection of the VNXe32000 s 10GBase-T interfaces to the unified fabric. Storage Configuration Capacity The EMC VNXe3200 unified hybrid storage system used in this architecture brings the power of EMC s VNX to the IT generalist. It retains the affordability, simplicity, and efficiency of previous generation VNXe systems and adds support for MCx multicore optimization, FAST Cache SSD caching, FAST VP auto-tiering, and Fibre Channel host connectivity. These enterprise-class features were previously reserved for higher-end storage systems. A single, redundant, VNXe3200 array was used for testing. The system was connected to the storage fabric by four 10GbE (one 2-port LACP bond per SP) connections utilizing iscsi. The following components made up the VNXe3200 deployment: One RAID 6 (8+2) Storage Pool (4 TB, 7k RPM NL-SAS disks) One RAID 6 (10+2) Storage Pool (600 GB, 10k RPM SAS disks) One RAID 5 (4+1) Storage Pool (200 GB, MLC SSD disks) *Since original publication, PernixData no longer recommends deployment of the Micron P420m in production environments. For further details, see PernixData Support KB or consult your PernixData account team. 10

11 Both hosts were zoned and masked to allow access to all 7 VMFS (block) iscsi datastores in Table 4: DATASTORE NAME DATASTORE SIZE STORAGE POOL RESOURCE TYPE vnxe_7k_lun00 2 TB NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk vnxe_ssd_lun GB SSD-Pool (RAID 5, 4+1) MLC SSD vnxe_ssd_lun GB SSD-Pool (RAID 5, 4+1) MLC SSD vnxe_10k_lun03 2 TB SAS-Pool (RAID 6, 10+2) 10k RPM SAS Disk vnxe_10k_lun04 2 TB SAS-Pool (RAID 6, 10+2) 10k RPM SAS Disk vnxe_7k_lun05 2 TB NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk vnxe_7k_lun06 2 TB NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk Table 4. Storage Capacity VMFS Datastores Performance PernixData FVP Software was used in this architecture to serve reads and writes locally on host servers and eliminate storage I/O bottlenecks. In this design, FVP software uses Micron P420m* 700 GB PCIe SSDs to accelerate all virtual machines. The following FVP configuration was used in this deployment: One Flash FVP Cluster All Virtual Machines 700 GB SSD flash per host Write Back (Local + 1 Peer) vsphere infrastructure VMs Write Back (Local Host) SQL Server VMs The smallest P420m* PCIe SSDs available were chosen for this reference architecture because on-disk database size for all HammerDB SQL databases was ~400 GB, leaving ample flash area for the frequently accessed data of other VMs in the environment. This may also be appropriate for many standard virtual deployments, but acceleration resource sizing will be ultimately determined by specific workload requirements. vsphere Configuration One virtual data center was deployed for all virtual machines. This is the standard, recommended configuration for most production deployments. Infrastructure and Management Configuration A single VMware vcenter Server instance was deployed for all virtual machines. For simplicity, all vcenter roles (inventory, SSO, vcenter) were consolidated onto a single VM. No resource contention was observed on any parts of the management infrastructure during tests. *Since original publication, PernixData no longer recommends deployment of the Micron P420m in production environments. For further details, see PernixData Support KB or consult your PernixData account team. 11

12 All server resources were sized according to the current best practices from VMware. They are listed in the following table: SERVER ROLE VCPU RAM (GB) STORAGE (GB) OS Domain Controller Windows Server bit R2 SQL Server (vcenter) Windows Server bit R2 vcenter Server Windows Server bit R2 Table 5. vsphere Server Resource Sizing Conclusion HammberDB tests clearly show that PernixData FVP software is a perfect complement for SQL Servers in VMware vsphere environments. Results demonstrate how an FVP-enabled architecture absorbs OLTP write bursts into low latency server-side flash or RAM for 2x transactions-per-minute and how decreased latency and increased storage throughput lead to 10x faster DSS database query performance. In addition, contrary to traditional storage capacity/performance coupled designs, FVP software also makes it possible to independently scale SQL Server performance while independently sizing underlying storage capacity. PernixData FVP software enables you to get all the operational benefits of virtualizing SQL Server with the best possible database performance. FVP s unique scale-out approach delivers SQL performance where and when its needed most, delivering more database transactions in less time for a fraction of the cost of other I/O acceleration solutions. When designing this PernixData FVP Software and Microsoft SQL Server reference architecture, specific hardware components were selected to accurately represent an existing, well-designed, virtual environment. However, it s important to note that although these components were used to test and validate SQL Server performance with FVP, they are not specifically required. Decoupling storage performance from capacity using FVP software allows for the use of many different hardware platforms designed to run VMware vsphere. The results of this document should be used alongside Microsoft SQL Server and VMware vsphere best practice documents to help choose hardware based on actual environmental requirements. 12

13 Appendix A: HammerDB TPC-C Test Methodology and Detailed Results As defined by HammerDB s Introduction to Transactional (TPC-C) Testing for all Databases, a transactional or OLTP (online transaction processing) workload is a workload typically identified by a database receiving both requests for data and multiple changes to this data from a number of users over time where these modifications are called transactions. The Transaction Processing Performance Council (TPC) is the industry body most widely recognized for defining benchmarks in the database industry and TPC-C is the benchmark published by the TPC for Online Transaction Processing. HammerDB includes an implementation of the TPC-C benchmark specification that can be run against Microsoft SQL Server and other supported database environments. Implementing the TPC-C specification ensures accurate, repeatable, and consistent results. HammerDB is designed to measure relative as opposed to absolute database performance between systems. This means that the test results published as part of this design shouldn t be compared to other published tests, but do serve as a good relative measure for comparing performance with and without FVP software. Note: HammerDB implements a workload based on the TPC-C specification. However it does not implement a full specification TPC-C benchmark and the transaction results from HammerDB cannot be compared with the official published TPC-C benchmarks in any manner. HammerDB workloads produce two statistics to compare configurations called transactions-per-minute (TPM) and new-orders-per-minute (NOPM), respectively. TPM is the specific database transactional measurement typically defined as the number of user commits plus the number of user rollbacks. TPM values cannot be compared between different database types, but for the purposes the tests in this design, work well since the same SQL Server database was used for all tests. Database Schema The TPC-C workload represents a system used to fulfill orders to supply products. The workload is defined by a mix of 5 transactions selected at random according to the balance of the percentage value shown below: 45% - New-order: receive a new order from a customer 43% - Payment: update the customers balance to record a payment 4% - Delivery: deliver orders asynchronously 4% - Order-status: retrieve the status of customer s most recent order 4% - Stock-level: return the status of the warehouse s inventory The database schema is depicted in Figure 5. 13

14 Figure 5. HammerDB TPC-C Schema Image Source: Introduction to Transactional (TPC-C) Testing for all Databases Configuration There are several HammerDB configuration options that can be tuned to provide the highest level of performance for any given hardware and software configuration. Generally speaking, the highest level of TPM performance is usually seen near maximum database server CPU utilization. Removing all other potential bottlenecks to maximize CPU utilization was a primary testing goal. As an example, to eliminate possible contention, 500 warehouses were generated as part of schema creation. All schema creation options are listed in the following table: OPTION SQL Server VALUE mssqlclus01.lab.pernixdata.com SQL Server Port 1433 SQL Server ODBC Driver SQL Server Native Client 11.0 Authentication SQL Server Database Schema Windows Authentication tpcc Updated Number of Warehouses 500 Virtual Users to Build Schema 32 Table 6. HammerDB TPC-C Schema Creation Options The Timed Test Driver Script was configured as part of the driver options to have HammerDB time the tests, measure the results and report on an average transaction rate for a period of time. A test duration of five minutes was selected with a rampup time of two minutes. Since these tests were not performed to compare to official TPC-C results, keying and thinking time was disabled. When configured this way, each virtual user executes tens of thousands of transactions a minute. 14

15 All driver options are listed in the following table: OPTION SQL Server VALUE mssqlclus01.lab.pernixdata.com SQL Server Port 1433 SQL Server ODBC Driver SQL Server Native Client 11.0 Authentication SQL Server Database TPC-C Driver Script Windows Authentication tpcc Timed Test Driver Script Total Transactions per User Exit on SQL Server Error Keying and Thinking Time Checkpoint when complete False False False Minutes of Rampup Time 2 Minutes for Test Duration 5 Table 7. HammerDB TPC-C Driver Options With keying and thinking time disabled, a limited number of virtual users and warehouses are required rather than hundreds or thousands of virtual users and warehouses. With each user executing such a large number of transactions, additional prerequisite testing was performed to determine the optimal number of virtual users that produced the highest TPM without exhibiting contention for data. This testing revealed the optimal number of virtual users to be 30. When running the Timed Test Driver Script, one additional user is required to perform timing and collect results. All other options were configured with their default settings. All virtual user options are listed in the following table: OPTION VALUE Virtual Users 31 User Delay(ms) 500 Repeat Delay(ms) 500 Iterations 1 Show Output Log Output to Temp Use Unique Log Name No Log Buffer True True True False Table 8. HammerDB TPC-C Virtual User Options Note: As per HammerDB best practice, a separate virtual machine with HammerDB software and the Microsoft SQL Server client installed was used to generate the workload for all tests. 15

16 Detailed HammerDB Results The results of the 5 minute HammerDB TPC-C tests are summarized in the table below: BENCHMARK TEST 1 (FVP) TEST 2 (WITHOUT FVP) Transactions Per Minute 400, ,464 New Orders Per Minute 87,036 39,465 Table 9. HammerDB TPC-C 5 minute TPM results In the following figures taken from the FVP Management Server user Interface, Read (blue) and Write (orange) lines depict the actual storage performance of one of the SQL Server virtual machines during tests. Virtual Machine Storage Latency In Figure 10 below, SQL Server virtual machine storage latency in Test 1 (with FVP) is shown on the left half and Test 2 (without FVP) on the right half. The significantly decreased latency in Test 1 allows for greater HammerDB TPM results. Figure 10. Virtual machine storage latency comparison between tests In Figure 11 below, SQL Server virtual machine storage latency (orange) is barely distinguishable along the bottom of the left half of the chart in Test 1 (with FVP). Datastore write latency for the same period (gold) can also be seen. This decrease in latency allows for greater HammerDB TPM results. Figure 11. Virtual machine to datastore latency comparison 16

17 In Figure 12 below, SQL Server virtual machine storage latency from an additional test with FVP is shown. Note the sub-millisecond scale on the left side of the chart. Figure 12. Virtual machine storage latency with FVP In Figure 13 below, SQL Server virtual machine storage latency from an additional test without FVP is shown. Figure 13. Virtual machine storage latency without FVP Virtual Machine Storage IOPS In Figure 14 below, SQL Server virtual machine storage IOPS in Test 1 (with FVP) is shown on the left half and Test 2 (without FVP) on the right half. The increased lops in Test 1 allows for greater HammerDB TPM results. Figure 14. Virtual machine storage IOPS comparison between tests 17

18 In Figure 15 below, SQL Server virtual machine storage IOPS from an additional test with FVP is shown. Figure 15. Virtual machine storage IOPS with FVP In Figure 16 below, SQL Server virtual machine storage IOPS from an additional test without FVP is shown. Figure 16. Virtual machine storage IOPS without FVP Virtual Machine Write Back Destaging Figure 17 below illustrates the amount of data (and estimated time to commit) that FVP is able to absorb into server-side flash during the write bursts of HammerDB TPC-C testing. This is data that the storage array is incapable of ingesting and becomes a bottleneck. Eliminating this bottleneck allows for greater HammerDB TPM results. Figure 17. Virtual machine write back destaging 18

19 Appendix B: HammerDB TPC-H Test Methodology and Detailed Results As explained by HammerDB s Introduction to Decision Support, Data Warehousing, Business Intelligence, and Analytical Load Testing for all Databases, the basis of Decision Support Systems is the ability to process complex ad-hoc queries on large volumes of data. As previously mentioned, the Transaction Processing Performance Council (TPC) is the industry body most widely recognized for defining benchmarks in the database industry. TPC-H is the benchmark published by the TPC for decision support systems (DSS) databases. HammerDB includes an implementation of the specification of the benchmark that can be run in a Microsoft SQL Server environment. Whereas TPC-C simulates an online ordering system, TPC-H represents the typical workload of business users inquiring about the performance of their business. In particular the focus is upon highly complex queries that require the processing of large volumes of data. Note: HammerDB implements a workload based on the TPC-H specification. However it does not implement a full specification TPC-H benchmark and the transaction results from HammerDB cannot be compared with the official published TPC-H benchmarks in any manner. A typical DSS performance profile is represented by the time it takes the system to process a query set from Query 1 to Query 22 (run in a pre-determined random order). As a simple benchmark you can compare the timings required for a single virtual user to complete the 22 queries across different tests. Since these tests were not performed for comparison to official TPC-H results, specific Power and Throughput tests were not performed to compile a Composite Query-per-Hour Performance Metric (QphH). Database Schema Similarly to TPC-C, the schema size for TPC-H is not fixed and is dependent upon a Scale Factor. The larger the schema, the more powerful the SQL Server system must be to process the increased data volume for queries. The database schema is depicted in Figure 18. Figure 18. HammerDB TPC-H schema Image source: Introduction to Decision Support, Data Warehousing, Business Intelligence, and Analytical Load Testing for all Databases 19

20 Configuration As with TPC-C, there are several HammerDB configuration options that can be tuned to provide the highest level of performance for any given hardware and software configuration. For DSS, processing large amounts of data within a single process or thread on a traditional row oriented database is time consuming. To increase performance, SQL Server employs Parallel Query Processing to optimize query execution and index operations for computers that have more than one CPU. With a chosen scale factor of 100 (~220 GB on disk), prerequisite testing revealed the optimal Maximum Degree of Parallelism (MAXDOP) to be 8. All schema creation options are listed in the following table: OPTION SQL Server VALUE mssqlclus01.lab.pernixdata.com SQL Server Port 1433 SQL Server ODBC Driver SQL Server Native Client 11.0 Authentication SQL Server Database Windows Authentication tpch MAXDOP 8 Scale Factor 100 Virtual Users to Build Schema 32 Table 10. HammerDB TPC-H Schema Creation Options Within the driver options a MAXDOP of 8 set the value to be used for query hints to set the Maximum Degree of Parallelism that a particular query will use. The Total number of query sets was set to one. This is the number of times after logging on that the virtual user completes an entire sequence of queries. The refresh function can be used to more accurately simulate official TPC-H testing, but for these tests was unused. Note: The queries do not run in sequence order from Query 1 to Query 22 and instead run according to a predefined random order depending on the number of virtual users. Since the same number of virtual users (5) was used for all tests, queries ran in the same order for both Test 1 and Test 2. All driver options are listed in the following table: OPTION SQL Server VALUE mssqlclus01.lab.pernixdata.com SQL Server Port 1433 SQL Server ODBC Driver SQL Server Native Client 11.0 Authentication SQL Server Database Windows Authentication tpch MAXDOP 8 Total Query Sets per User 1 Exit on SQL Server Error Verbose Output Refresh Function False False False Table 11. HammerDB TPC-H Driver Options 20

21 To generate significant levels of storage i/o for performance comparison, the query sets were run with 5 simultaneous user streams. This is similar to the number of streams required by the official Scale Factor 100 TPC-H Throughput test. All other options were configured with their default settings. All virtual user options are listed in the following table: OPTION VALUE Virtual Users 5 User Delay(ms) 500 Repeat Delay(ms) 500 Iterations 1 Show Output Log Output to Temp Use Unique Log Name No Log Buffer True True True False Table 12. HammerDB TPC-H Virtual User Options Detailed HammerDB Results The average total query completion times (5 users, 22 total queries each) recorded during HammerDB TPC-H tests are summarized in the table below: BENCHMARK TEST 1 (FVP) TEST 2 (WITHOUT FVP) Avg Total Query Completion Time 2h:14m;14s 11h:14m:38s Table 13. HammerDB TPC-H avg total query completion times 21

22 The average query completion times recorded during HammerDB TPC-H tests and the factor decrease seen in Test 1 (with FVP) are summarized in the table below: QUERY TEST 1 (FVP) TEST 2 (WITHOUT FVP) FACTOR DECREASE x x x x x x x x x x x x x x x x x x x x x x Table 14. HammerDB TPC-H query completion times (seconds, lower is better) In the following figures taken from the FVP Management Server user Interface, Read (blue) and Write (orange) lines depict the actual storage performance of one of the SQL Server virtual machines during tests. Note: The extended nature of TPC-H tests yields less granular datapoints than shorter duration TPC-C tests. The granularity of datapoints may also differ between Test 1 and Test 2. 22

23 Virtual Machine Storage Latency In Figure 19 below, SQL Server virtual machine storage latency from Test 1 (with FVP) is shown. The decreased latency in this test translates to faster HammerDB query completion times. Figure 19. Virtual machine storage latency during Test 1 (with FVP) In Figure 20 below, SQL Server virtual machine storage latency from Test 2 (without FVP) is shown. Figure 20. Virtual machine storage latency during Test 2 (without FVP) Virtual Machine Storage IOPS In Figure 21 below, SQL Server virtual machine storage IOPS from Test 1 (with FVP) is shown. The increased IOPS in this test translates to faster HammerDB query completion times. Figure 21. Virtual machine storage IOPS during Test 1 (with FVP) 23

24 In Figure 22 below, SQL Server virtual machine storage IOPS from Test 2 (without FVP) is shown. Figure 22. Virtual machine storage IOPS during Test 2 (without FVP) Virtual Machine Storage Throughput In Figure 23 below, SQL Server virtual machine storage throughput from Test 1 (with FVP) is shown. The increased throughput in this test translates to faster HammerDB query completion times. Figure 23. Virtual machine storage throughput during Test 1 (with FVP) In Figure 24 below, SQL Server virtual machine storage IOPS from Test 2 (without FVP) is shown. Figure 24. Virtual machine storage throughput during Test 2 (without FVP) 24

25 Appendix C: Bill of Materials The test configuration bill of materials is summarized in the following table: AREA COMPONENT QUANTITY Host Hardware Cisco UCS C220 M3 (2 x Intel E5-2640, 256 GB RAM) 2 Storage Hardware EMC VNXe3200 Array 8 x 200 GB MLC SSDs 16 x 600 GB 10K RPM SAS Disks 12 x 4 TB 7.2K RPM NL-SAS Disks 1 Network Hardware Cisco UCS 6248UP Fabric Interconnect 2 Cisco Nexus 2232TM Fabric Extender 2 PernixData FVP VMware ESXi 5.5 Update 2 2 Software VMware vcenter 5.5 Update 2c 1 Microsoft Windows 2012 R2 Standard 5 Microsoft SQL Server 2012 Standard SP1 1 Microsoft SQL Server 2014 Enterprise 2 Table 15. Bill of Materials 25

26 References PernixData Product Page PernixData Proven Designs Page Microsoft SQL Server Product Page VMware vsphere Product Page SQL Server on VMware Best Practices Guide HammerDB Product Page HammerDB Introduction to Transactional (TPC-C) Testing for all Databases HammerDB Microsoft SQL Server OLTP (Transactional) Load Testing HammerDB Introduction to Decision Support, Data Warehousing, Business Intelligence, and Analytical Load Testing for all Databases HammerDB Microsoft SQL Server Decision Support (DSS) Load Testing Copyright 2016 PernixData, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. 26 PernixData, FVP, the, and Flash Cluster are registered trademarks and Architect is a trademark of PernixData, Inc. in the United States and/or other jurisdictions. All other brands, products, marks and names mentioned herein may be trademarks or service marks of, and are used to identify, products or services of their respective owners.