FlashSoft/SanDisk SQL Server Performance Enhancement commissioned by
SanDisk manufactures enterprise-grade SSD units, referred to as Enterprise Flash Drives (EFDs) under the Lightning brand name. FlashSoft provides software that intelligently caches data from traditional hard disks to faster solid-state drives (SSDs) installed within a server. The two companies have worked together to develop and test a solution offering that includes both the FlashSoft software and SanDisk EFD devices. The combination of EFDs and FlashSoft SE software can provide substantial performance improvements in disk-intensive server applications. For this analysis, Microsoft SQL Server was considered and tested, although benefits should be similar for products such as Microsoft Exchange Server, Microsoft SharePoint Server, and other disk-intensive services. In tests, the team at Concentrated Technology witnessed consistent performance improvements that enabled existing SQL Server hardware to handle triple the amount of client workload as in a non-enhanced configuration. Performance improvements like these offer the potential for significant business advantages, including increasing the utilization of existing server assets, enabling greater workload consolidation, and enabling scale-up of key enterprise applications. Concentrated Technology believes this kind of improvement may also have use in alleviating performance problems associated with poorly designed databases and badly written code or queries, as well as in any I/O-intensive database application. Contents 2 Executive Overview 3 Technology Backgrounder 4 Performance Analysis 6 Business Analysis 8 Additional Considerations 9 Conclusions 10 Appendix A: FlashSoft/SanDisk OLTP Test Validation 11 Appendix B: efdmark for FlashSoft/SanDisk 2
Solid-state drive (SSD) technology has improved enough to permit the creation of enterprise-grade devices, or enterprise flash drives (EFDs). Typically connected to the server via 6Gbps serial-attached SCSI (SAS) or via the server motherboard s PCIe bus, these drives offer incredibly fast transfer times for both read and write operations many times faster than even high-speed, 15,000 RPM rotational disks. EFDs, however, are still significantly more expensive than an equivalent-sized fixed disk. Thus, it is not yet practical to completely replace rotational disks with EFDs. The FlashSoft SE software creates a two-tiered primary storage system. Less-expensive rotational disks are still used for main storage, but that storage is accessed through FlashSoft SE. The software utilizes an EFD-based cache to intelligently cache commonly used data on the faster EFD, as well as writing data from EFD to rotational disks as the usage of that data usage diminishes. The concept of using SSD in this fashion is relatively recent, and to date most implementations have been part of a back-end Storage Area Network (SAN). FlashSoft SE and SanDisk EFDs make this kind of caching capability an add-on for any server, whether or not it uses a SAN that is capable of independent SSD-based enhancement. SSD-based caching as a way to improve disk performance is a hot topic that is gaining wider acceptance in the industry. 3
FlashSoft provided performance test results to Concentrated Technology, and then our team performed our own benchmark testing. Both tests used FlashSoft SE software combined with SanDisk EFD hardware and Microsoft SQL Server-based database applications. FlashSoft and SanDisk provided a set of performance test results conducted with their business partners and customers. This test emulated a common online transaction processing (OLTP) order management application, run on Microsoft SQL Server 2008 R2. It reflects a reasonable emulation of production-type traffic, including data for customers, products, order history, and other expected elements, along with query traffic that reflects a common mix of query and insert/update actions. The company also provided its test data and methodology, which Concentrated Technology reviewed. Our team was able to validate both the test and its results (see Appendix A for our validation statement), which indicated an increase of 317% in the number of transactions per minute (TPM) that the database system could sustain. In conjunction with a Concentrated Technology consulting client, our team conducted an independent test of the FlashSoft/SanDisk configuration. The team found the FlashSoft SE software installation to be trivial, and had the enhanced configuration up and running in under an hour (including installation of the EFDs). Our efdmark test is derived from a real-life, production database application utilized by a Concentrated Technology customer. This application is designed to manage real-estate data for a large, distributed real-estate brokerage firm, and the application reflects most commonly used Online Transaction Processing (OLTP) application scenarios. The database consists of some three dozen tables, numerous indexes, and a great many stored procedures, user-defined functions, and other back-end elements of a SQL Server system. More importantly, the database schema design is specifically reflective of a common real-world condition: It is not perfect. For a complete description of the test methodology, as well as the full FlashSoft/SanDisk results, please see Appendix B. Our test revealed a 308% improvement in the client workload capacity of the FlashSoft/ SanDisk-enhanced configuration. 4
In the test, the baseline configuration sustained 1,250 client connections with a 3-second Service Level Agreement (SLA); in the FlashSoft/SanDisk-enhanced test, the same server sustained 3,850 client connections while maintaining the SLA (see Appendix B for the complete FlashSoft/SanDisk test results). It should be noted that these results were all achieved using SAS EFD units; our team did not test EFD units attached using the server s internal PCIe bus. 5
SQL Server has always been difficult to scale out, meaning it is architecturally challenging to create database applications that run across multiple database servers. The primary approach for accommodating more SQL Server workload is to scale up, using more processors, more memory, and faster storage to deal with the workload. Scaling up obviously has limits, and can become prohibitively expensive. The FlashSoft SE software is licensed at $2,000 for a single standalone server that is using the 150GB Serial-Attached SCSI (SAS) EFDs used in Concentrated Technology s test. SanDisk EFDs of that type have street prices of around $4,000. Our test configuration therefore had a total cost of approximately $10,000, in addition to the cost of the server, software, hard drive storage, and so forth. Our team presented the test results to several Concentrated Technology clients and asked how, and if, those results would impact their server plans. Here are some of their views: The client who participated in our test was considering a move to massive new server hardware: 40 processor cores, 128GB of memory, and so forth. After concluding our test, the client now feels that they could actually remain on their existing hardware, utilizing 8-core processors and 32GB of memory with, in their words, room to spare. Being able to handle literally triple the workload on the same hardware is a massive business benefit, increasing the lifespan and utilization of an existing asset with minimal incremental investment an outlay of approximately $10,000 versus spending $30,000 for a new server. A client in the aerospace industry indicated that it would enable the level of server consolidation they have been seeking to achieve without purchasing a new server. He estimated that they could eliminate or re-purpose as many as two-thirds of their existing SQL Server computers by consolidating onto existing machines that had been EFD-enhanced. He also estimated that his savings would be around $35,000. A large retail client saw the ability to potentially abandon a nascent scale-out project and instead enhance their existing servers to handle workload. Their servers have an extremely unbalanced workday, meaning much of the application s work is done in the evenings, and the servers sit relatively idle during the day. Their scale-out project was only just beginning, but was proving to be architecturally difficult and potentially expensive given the schema changes, client application reprogramming, and other tasks that would have to be completed. They told us: We can probably just forget about that if we can get triple performance out of the existing hardware. They could not estimate a potential savings, but suggested that the scale-out project would have involved at least $30,000 in new hardware and thousands of man-hours in programming time. One of the most significant and frustrating performance problems in the SQL Server world 6
is the poorly built application. Whether from a badly designed database schema, poorly maintained indexes, or inexpertly written code and queries, these problems are frustrating because an expert technologist can see the problem but is often powerless to fix it due to the business risk and expense involved. EFD enhancement of existing servers is a way to spend less money and relieve the problem, an uncomfortable truth that from a tactical perspective is often the best short- or medium-term solution for the business. In fact, our independent test reflected exactly this situation. The database schema was overnormalized, entailing the use of unnecessary multi-table joins for some of the most commonly executed queries. Due to business conditions, index maintenance didn t happen frequently enough. Fixing these problems would involve significant re-coding, re-design, and political will within the organization; EFD enhancement relieved the bottleneck without any of those concerns coming into play. 7
It should be noted that EFD enhancement, whether from FlashSoft/SanDisk or anyone else, is not a panacea for all SQL Server performance problems. Both in our tests and in the situations of the clients our team spoke with, disk throughput had already been identified as the major performance bottleneck of the applications. EFD enhancement can significantly relieve that bottleneck, but, as in all performance-optimization scenarios, that simply moves the bottleneck to another subsystem. After relieving the disk performance problem, memory or CPU utilization typically became the next bottleneck to solve. However, the point is that EFD enhancement can provide marked performance improvement to the disk subsystem something which has often been challenging and expensive to address. In scenarios where the SQL Server computer is already approaching maximum CPU or memory utilization, EFD enhancement will likely yield minimal improvement, because once the disk performance bottleneck is relieved, you will quickly hit the CPU or memory ceiling. However, even if your ultimate solution is to purchase a new server with more memory and more CPU capacity, the inclusion of EFD enhancement in that server can enable you to go with a much smaller and less-expensive server than you might have otherwise required. Our team has seen instances where disk bottlenecks can be partially overcome by installing more CPU and memory capacity into the server. SQL Server has its own caching mechanisms to help minimize disk access, so more memory and enough processor power to manage it can let SQL Server do a better job of that. EFD enhancement, however, directly alleviates the disk performance bottleneck, enabling your existing hardware to accommodate more workload. Finally, note that the tests conducted for this paper, both by Concentrated Technology and by FlashSoft, were not mere load-test exercises. Such exercises can often fail to completely engage the various disk activities intrinsic in SQL Server, including the use of indexes, fragmented database files, and other overhead. Our independent test in particular ensured that these factors were all in play, making us confident that our 3x performance improvement is a reasonable, practical expectation for similar database applications. Our team especially appreciates the portable nature of EFD enhancement. Memory and processors, for example, can rarely be utilized in an upgrade to a new server. The new server usually requires different memory specifications, and likely can t use older processors (the need for new processors is often the driving decision behind a new server acquisition in the first place). However, your EFD enhancement investment can be moved to the new hardware, providing additional performance from the outset. 8
Triple or better performance improvement in SQL Server applications is something that no technologist or decision-maker should ignore. The business benefits are exceedingly clear: Get more workload on your existing hardware with a minimal incremental investment. In most foreseeable situations, EFD enhancement of existing servers should be less expensive than an equivalent scale-up purchase or scale-out project that achieved the same performance gains. Our guidance to clients going forward will be to always consider EFD enhancement of existing compatible hardware as a first scale-up option when disk throughput is the primary bottleneck. Although our tests and client feedback focused only on Microsoft SQL Server performance, our team anticipates similar benefits with products such as Microsoft Exchange Server, SharePoint Server, and other storage-intensive server products. 9
FlashSoft provided Concentrated Technology with a database-performance test suite intended to emulate a production Online Transaction Processing (OLTP) application, for the purpose of comparative performance testing. Concentrated Technology has verified the validity of the test suite. It is substantially similar to other well-known, commonly used OLTP benchmark tests, and reflects a reasonable emulation of production-type traffic. The test s database schema represents a common order-processing system including products, customers, order history, and other expected elements. The schema includes the use of primary key constraints. The query traffic used against this schema reflects a common mix of query and insert/update traffic. The test utilized FlashSoft SE software in combination with two SanDisk 3Gb SAS 150GB enterprise flash drives (EFDs). Our team ran the test once on the server without the software or EFDs installed, and then again with the software and EFDs installed. For both tests, the server was a 6-core machine with 6GB of RAM, using 15 146GB 3Gb SAS 15,000 RPM drives in a RAID 5 array. We provided two separate disks for the operating system and applications. The server ran Windows Server 2008 R2 and SQL Server 2008. The test data set began at 330GB and grew to 450GB in three hours. For the EFD-enhanced test, our team configured FlashSoft SE to use a 139GB read-write cache on the EFDs. When configuring the two EFDs in a fault-tolerant RAID1 set, FlashSoft observed a 390% improvement in performance, achieving 12,985 transactions per minute (TPM) over the server s non-enhanced baseline performance of 3,323 TPM. When configuring the two EFDs in a RAID0 set, FlashSoft observed a 415% improvement in performance, with 13,795 TPM. FlashSoft has provided Concentrated Technology with sufficient evidence for us to validate these test results. Concentrated Technology has also conducted and documented its own independent benchmark tests of FlashSoft SE and SanDisk enterprise flash drives (EFDs), with significantly similar performance improvements. 10
June 2011 The database industry is full of benchmark tests, the most well-known of which are produced by the Transaction Performance Council (TPC). Most of these tests are designed to examine comparative performance between database engines, such as Microsoft SQL Server, Oracle, and IBM DB/2. Hardware configuration obviously plays a role in this performance, so certified test results include the processor and memory configuration, as well as the server make and model. However, the disk system often the main performance bottleneck in production database applications is not specifically stressed by many common benchmark tests. The Concentrated Technology efdmark test is designed to reflect absolutely real-world conditions for a production SQL Server application, including targeting of the disk system, memory configuration, CPUs, and so forth. This document describes the makeup of the test. The primary purpose of efdmark is to test the performance difference between a given usage scenario both with and without Enterprise Flash Drives (EFD, using Solid-State Drive, or SSD, technology) and compatible flash drive management/caching software. This paper describes the test results from running Concentrated Technology s efdmark performance benchmark using FlashSoft for Windows Server and SanDisk LB 150S enterprise flash drives. This test was conducted in June 2011, using the then-current versions of the software and hardware. In this test, the FlashSoft/SanDisk combination resulted in more than triple the performance of the non-enhanced baseline test. 11
The efdmark test is derived from a real-life, production database application utilized by a Concentrated Technology customer. This application is designed to manage real-estate data for a large, distributed real-estate brokerage firm, and the application reflects most commonly used Online Transaction Processing (OLTP) application scenarios. The database consists of some three dozen tables, numerous indexes, and a great many stored procedures, user-defined functions, and other back-end elements of a SQL Server system. More importantly, the database schema design is specifically reflective of a common real-world condition: It is not perfect. In particular: In some areas, the schema is over-normalized, impacting performance by requiring an unusually high number of table joins for some commonly performed queries. The database s data pages are approximately 10% fragmented, again reflecting a real-world database that has been in use for some time but which receives regular maintenance. Indexes are also somewhat fragmented. All indexes were built with a 50% fill factor, allowing for growth while minimizing page splits but also increasing the I/O load on the server for index reads. When the test is conducted, index fragmentation is at approximately 5%, meaning that some index pages are beginning to fill and that page splits a disk-intensive operation are beginning to occur. Usage patterns provide about 62% query (read) traffic, and 38% commit (write) traffic. The test database begins at 460GB in size, and grows to 610GB over the course of the test. Client traffic is duplicated by playing back traffic captured from actual production clients over the course of several days. This client traffic reflects several different distinct usage patterns, including users who add new data, users who query data, users who update data, users who perform aggregate queries, and so forth. The presence of secondary indexes, page splits, and fragmentation are especially important because in a production scenario these factors are always present, and represent a significant disk I/O load to the database engine. 12
The standard test platform uses the following hardware: Element Processors Memory Operating System Database Engine Data Storage OS/Application Storage Configuration 2x Intel Xeon X5647 4-core 32GB RAM Microsoft Windows Server 2008 R2 Enterprise (x64) Microsoft SQL Server 2008 R2 Enterprise (x64) 36 disks, 6G SAS, 15k RPM, RAID5 2 disks, 6G SAS, 15k RPM, RAID1 For the FlashSoft/SanDisk test, the following was added: FlashSoft for Windows Server SSD cache management software Two SanDisk LB 150S SAS-attached 150GB enterprise flash drives, configured in a RAID1 array The first test measures average response time under an average, non-peak load. The following performance characteristics include the baseline (non-efd enhanced) performance as well as the observed performance of the EFD-enhanced configuration: Performance Characteristic Baseline Performance Enhanced Performance Average response time 3.12 seconds 72 milliseconds The second test attempts to push the hardware to its maximum capability by increasing the number of clients that are utilizing the database application. Performance Characteristic Baseline Performance Enhanced Performance Transactions per second 89 192 Number of clients 1,750 3,350 The third test establishes a 3-second SLA for query response time, and measures the number of clients that the server can support within this SLA. Client workload is increased until the SLA is surpassed. Performance Characteristic Baseline Performance Enhanced Performance Number of clients 1,250 3,850 The third test is the primary benchmark used for the efdmark score. The performance of the unenhanced test platform is represented by the baseline efdmark score of 1. For this FlashSoft/SanDisk test, the efdmark score is 3.08, meaning 3.08 times the maximum number of clients as the unenhanced baseline configuration. 13
The following graph compares the baseline (non-enhanced) performance with the tested (EFD-enhanced) system. This graph applies only to the third test, from which the efdmark score is derived. 14
Concentrated Technology, LLC is a technology education and analysis consulting firm founded by industry luminaries Don Jones and Greg Shields. The company provides strategic consulting services to companies around the world, and performs market research and analysis for technology vendors and customers. The company distinguishes itself through its motto, Maximum Knowledge - Minimum Time, focusing on producing concise, accurate materials that help modern businesses accomplish more, with less effort. SanDisk Corporation (NASDAQ: SNDK) is a global leader in flash memory storage solutions, from research and development, product design and manufacturing to branding and distribution for OEM and retail channels. Since 1988, SanDisk s innovations in flash memory and storage system technologies have provided customers with new and transformational digital experiences. SanDisk s diverse product portfolio includes flash memory cards and embedded solutions used in smart phones, tablets, digital cameras, camcorders, digital media players and other consumer electronic devices, as well as USB flash drives and solid-state drives (SSD) for the computing market. SanDisk s products are used by consumers and enterprise customers around the world. FlashSoft brings the performance and scalability of SSDs to enterprise computing architecture without requiring changes to servers, applications, or disk-based storage systems. FlashSoft has achieved this with innovative software that enables any SSD or Flash PCIe to act as a high-performance memory cache for the server. Because flash memory is many times faster than HDD storage and is connected directly to the server, it overcomes the IO bottleneck. FlashSoft enables an SSD to transparently cache critical data without any modification to server operating systems or applications, and it works seamlessly with underlying storage systems. Because FlashSoft caches only the hottest - most in-demand - data, it enables an SSD to dramatically increase the performance of servers processing tens or even hundreds of terabytes of data. 15