WITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE 1 W W W. F U S I ON I O.COM
Table of Contents Table of Contents... 2 Executive Summary... 3 Introduction: In-Memory Meets iomemory... 4 What is In-Memory OLTP?... 4 Unleash SQL Server 2014 In-Memory OLTP Database from I/O Constraints... 5 Eliminating Continuous Checkpoint slowdowns... 5 Improving HA and Maintenance by Accelerating Reboots, Restarts, and Failovers... 5 About the Tests... 6 Server Configuration... 6 Storage Configuration... 6 Operating System Configuration... 7 SQL Server 2014 Configuration... 7 Workload Configuration... 7 Test Results... 8 40GB In-Memory OLTP Transaction workload Test Results... 8 Overall Performance... 8 Memory-Optimized Filegroup Performance... 9 Transaction Log Performance... 10 Database Startup Performance... 13 Scaling Beyond 12,000 Users... 14 Summary... 15 Appendix A: Is SQL Server In-Memory OLTP Right for You?... 16 Table Usage Analysis... 26 Table Contention Analysis... 28 Stored Procedure Usage Analysis... 29 Memory Optimization Advisor... 30 Native Compilation Advisor... 37 2 W W W. F U S I ON I O.COM
Executive Summary There has been a lot of press recently about in-memory databases, and with good reason: business user requests served from memory are lightning-fast. An in-memory database can easily process an order of magnitude more transactions per day, and is therefore capable of order of magnitude greater business productivity. SQL Server In- Memory OLTP allows specific key tables to be memory-optimized while maintaining the investment in the T-SQL code base. SQL Server In-Memory OLTP shows dramatic performance increases in a typical system and allows business users to choose which business objects should be placed in memory, based on critical need. This paper shows how combining Fusion iomemory with SQL Server In-Memory OLTP improves the customer experience even more. It does this by improving system response time and business productivity (transactional processing capabilities), allowing more customers to be served per server, all the while dramatically reducing costs compared to traditional storage solutions. The following table summarizes results of performance testing performed by Fusion-io on a Dell PowerEdge R720 and validated by Microsoft. Business Value Key Performance Indicators Enterprise- Class Disk Array with SQL Server 2014 In- Memory OLTP iodrive2 Duo 2.4TB with SQL Server 2014 In- Memory OLTP Business Process Improvement Improve Customer Experience User Transaction Wait Time (µ) 1329 117 Serve More Customers Transaction Throughput (MB/s) 42 172 Reduced user wait times by 91% Increased throughput by 409% Improve Business Productivity Total Transactions Processed (over 45 minutes) 6,362,883 28,328,639 Processed 445% more transactions Deliver on Internal Service Level Agreements Database Startup Time (sec) 222 72 Reduced startup time by 67% 3 W W W. F U S I ON I O.COM
Introduction: In-Memory Meets iomemory While SQL Server 2014 In-Memory OLTP brings never-before-seen performance levels and capabilities to SQL Server databases, these new capabilities require a storage subsystem that delivers consistent low latency, ultra-high bandwidth, and strong reliability. Backing SQL Server In-Memory OLTP databases with Fusion iomemory products is critical in driving the highest transaction performance levels. This is done with the simplest, most cost-effective approach, compared to legacy storage architecture. WHAT IS IN-MEMORY OLTP? In-Memory OLTP is a memory-optimized database engine for SQL Server 2014. In-Memory OLTP enables transactional database applications to significantly improve transactional throughput by providing contention-free access to tables, by way of native code. Other In-Memory database platform offerings often require re-tooling application code. This could be due to the need for deploying a new data access method, or even having enough RAM to house the entire database. SQL Server 2014 does not require the entire database to be placed into RAM the developer can choose which tables would benefit the most from becoming memory-optimized tables. While there are some restrictions as to data types that may be a part of a memory-optimized table, the base data types are supported. Stored procedures that access the selected memoryoptimized tables may be compiled into native code, which executes faster. The end result is that much of the T-SQL code can be leveraged, taking advantage of new SQL Server In-Memory OLTP paradigms where the most value can be gained. Even when there is enough RAM in the server to hold the entire database in memory, SQL Server s access mechanism still operates on the concept of a page of data. It goes through the process of latching and locking to ensure data consistency. SQL Server In-Memory OLTP has an optimistic approach to concurrency. There is no latching or locking, but there is a mechanism to detect conflict based on row versioning. Also, there is no longer a concept of a page of data within the SQL Server In-Memory data structures. With no pages to latch or lock, there is a significant improvement in performance, as there is no waiting on resource availability and execution is unleashed. 4 W W W. F U S I ON I O.COM
UNLEASH SQL SERVER 2014 IN-MEMORY OLTP DATABASE FROM I/O CONSTRAINTS No matter how powerful the database system, performance will always be limited at some point by chokepoints and bottlenecks. Fusion iomemory resolves several I/O bottlenecks to raise the bar on capabilities for SQL Server 2014 In- Memory OLTP database. ELIMINATE CONTINUOUS CHECKPOINT SLOWDOWNS To ensure durability, SQL Server In-Memory OLTP relies heavily on the NTFS file system to store data and delta files in a database file group optimized for in-memory data. SQL Server In-Memory OLTP uses an offline checkpoint process to track and manage inserted and deleted data. This generates a considerable amount of sequential disk I/O activity and increases transaction log activity. Write performance is key as transaction log activity is increased. This is due to the highly concurrent, contention-free transactions generated in memory. iomemory products direct-path architecture provide industry-leading, near-dram write performance, which is often overlooked when discussing flash storage. This architecture enables performance more effectively than SSDs do, because SSDs place traditional storage protocols and RAID controllers between the data and the application, making it much more difficult to deliver consistent write performance. IMPROVE HA AND MAINTENANCE BY ACCELERATING REBOOTS, RESTARTS, AND FAILOVERS In addition to the continuous checkpoint process, SQL Server In-Memory OLTP benefits by reading data in parallel. The parallel read process happens a) when loading data from the data and delta files into memory during a server reboot, b) when restarting the SQL Server service, or c) when setting the database offline. This process generates random I/O and relies on a high-performing disk subsystem to load tables into memory. The closer the data is to the CPU, the faster it will load into memory. As with the continuous checkpoint, iomemory products direct-path architecture gives the database server CPUs direct access to the data and delta files, which accelerates reboots, restarts, and failover processes. See the white paper Going Beyond SSD: The Fusion Software Defined Flash Memory Approach for more information about how iomemory differs from SSDs. 5 W W W. F U S I ON I O.COM
About the Tests Testing compared performance of an iodrive2 Duo 2.4TB in a Dell PowerEdge R720 server to an enterprise-class disk array in three key areas: 1. Transactional Performance. How the two systems performed under the heavy I/O workload generated by the in-memory transaction log activity and continuous checkpoints. Tests were conducted on latency (responsiveness), and bandwidth (transactional workload) with 12,000 users. 2. Database Startup. As the database is running out of memory on servers, loading the database quickly becomes a key factor in system resilience. We tested how fast the transactional database could be brought online to compare how the two systems would handle a server reboot or cluster failover event. 3. User Load. Tests were limited to 12,000 users because, at that point, the disk array started experiencing timeouts. Additional testing measured the total user load of an iomemory system to demonstrate an additional aspect of scalability. SERVER CONFIGURATION The following settings were enabled in the BIOS: Logical Processor was enabled. With Hyper-threading enabled, we were able to increase the number of users by 8000 in further testing. System Profile Settings was set to performance. Virtualization was disabled. Fan Offset was set to high. Node Interleaving was disabled. STORAGE CONFIGURATION One iodrive2 Duo 2.4TB was used as the storage for the in-memory database within the Dell R720 server. The iodrive2 Duo product appears as two physical block devices to the host operating system. One block device was used for the memory-optimized filegroup and disk-based filegroup. The remaining block device was used exclusively for the transaction log. The enterprise class storage array was configured with two storage pools. The first storage pool was comprised of 12 spindles and was used to store the memory-optimized file group, the disk-based data files, and tempdb. The second storage pool was comprised of 12 spindles and was used exclusively for the in-memory database transaction log. 12x HDD RAID10 Disk Group Checkpoint Files 12 x HDD RAID10 Disk Group Logs iodrive2 Duo 2.4TB (used as 2 x 1.2TB block devices) Drive 1: Memory-Optimized FG Drive 2: Logs H: O: N: L: 2000GB 500GB 1.2TB 1.2TB Figure 1. Storage layout of enterprise class disk array and Fusion-io iodrive2 Duo 2.4TB. Note: Neither tempdb nor the disk-based tables were accessed during testing. All tables were marked for in-memory use. 6 W W W. F U S I ON I O.COM
OPERATING SYSTEM CONFIGURATION We used the Interrupt-Affinity Policy Tool to bind the iodrive2 interrupts to CPU0. The Interrupt-Affinity Policy Tool can be found at http://msdn.microsoft.com/en-us/windows/hardware/gg463378. SQL SERVER 2014 CONFIGURATION Dell PowerEdge R720 Name Baseline Values Fusion-io Values Transactional Database data files 1 per core 1 per core Tempdb data files 1 per core 1 per core Max Degree of Parallelism 1 1 Min/Max Server Memory (MB) 256,000 256,000 Lock Pages in Memory Enabled Enabled WORKLOAD CONFIGURATION A custom benchmark application generated a transactional workload on the systems. The benchmark application was configured with the following parameters. Parameter Duration In-Memory OLTP Database Size Values 45 minutes 30GB Users to generate load 12000 User Delay (ms) 0 Repeat Delay (ms) 0 Minutes of Ramp Time 0 7 W W W. F U S I ON I O.COM
Test Results This section shows the test results by each category of test: transactional workload performance and database start-up performance. The transactional workload performance results are shown by category: overall performance, memoryoptimized filegroup performance, and transaction log performance. The charts reflect testing with a scale of 12,000 users because the storage array s capabilities peaked at that workload. Testing beyond a 12,000 user load resulted in timeouts with In-Memory OLTP, which stopped data processing for extended periods of time. 40GB IN-MEMORY OLTP TRANSACTIONAL WORKLOAD TEST RESULTS OVERALL PERFORMANCE The chart below shows a summarized view of the transactional performance testing results. Across all performance indicators, iomemory increases business productivity by providing more transactional throughput with significantly lower latency. Overall TPCC Performance Comparison SQL Server 2014 In-Memory OLTP iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array 3000 2500 2,621 2000 1500 1,329 1,408 1000 500 0 117 Transaction Log Write Latency (microseconds) 640 Transaction Log Read Latency (microseconds) 172 42 106 31 Transaction Log Write Megabytes/sec Transaction Log Read Megabytes/sec 217 Memory-Optimized Filegroup Write Latency (microseconds) 350 85 Memory-Optimized Filegroup Write Megabytes/sec iodrive2 Duo 2.4TB Enterprise-Class Disk Array Figure 2. Fusion iomemory products increase In-Memory OLTP performance more effectively than an enterprise disk array. 8 W W W. F U S I ON I O.COM
1 10 19 28 37 46 55 64 73 82 91 100 109 118 127 136 145 154 163 172 181 190 199 208 217 226 235 244 253 262 271 280 289 298 307 316 325 334 343 352 361 370 379 388 397 406 415 424 433 442 451 460 469 478 487 496 505 514 523 532 1 11 21 31 41 51 61 71 81 91 101 111 121 131 141 151 161 171 181 191 201 211 221 231 241 251 261 271 281 291 301 311 321 331 341 351 361 371 381 391 401 411 421 431 441 451 461 471 481 491 501 511 521 531 MEMORY-OPTIMIZED FILEGROUP PERFORMANCE One set of tests compared write bandwidth and latency of iomemory products to an enterprise disk array to see how each handled the impact of background processes, such as continuous checkpoint. As the charts show, the iomemory system delivers consistent high performance, which helps ensure durability. 6000 SQL Server 2014 In-Memory OLTP Memory-Optmized Filegroup Write Latency (microseconds) iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array 5000 4000 3000 2000 1000 0 iodrive2 Duo 2.4TB Enterprise-Class Disk Array Figure 3. iomemory products enable 84% faster in-memory checkpoint processes with much less variation. 500 450 400 350 300 250 200 150 100 50 0 SQL Server 2014 In-Memory OLTP Memory-Optmized Filegroup Write Megabytes/sec iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array iodrive2 Duo 2.4TB Enterprise-Class Disk Array Figure 4. Continuous checkpoints are faster due to 75% higher average bandwidth. 9 W W W. F U S I ON I O.COM
TRANSACTION LOG PERFORMANCE The in-memory database results in much greater workload on transaction logs. Slow I/O on the transaction log reduces database performance. Tests measured the average and maximum transactions per second that the two systems completed over a 45-minute period. Tests also measured latency to compare transactional processing speeds to show the responsiveness of the database under load, and the speed it can serve individual requests. iomemory products completed 4.5x more transactions over the test period, and did so with much higher response times, thus making each database server more scalable. SQL Server 2014 In-Memory OLTP Transactions/sec iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array 60000 50000 52,594 40000 30000 20000 10000 4.4x More Transactions 11,810 0 iodrive2 Duo 2.4TB Enterprise-Class Disk Array Figure 5. iomemory system enables 4.5X more user transactions per second. 10 W W W. F U S I ON I O.COM
1 11 21 31 41 51 61 71 81 91 101 111 121 131 141 151 161 171 181 191 201 211 221 231 241 251 261 271 281 291 301 311 321 331 341 351 361 371 381 391 401 411 421 431 441 451 461 471 481 491 501 511 521 531 1 11 21 31 41 51 61 71 81 91 101 111 121 131 141 151 161 171 181 191 201 211 221 231 241 251 261 271 281 291 301 311 321 331 341 351 361 371 381 391 401 411 421 431 441 451 461 471 481 491 501 511 521 531 4500 SQL Server 2014 In-Memory OLTP - 12000 Users Transaction Log Read Latency - 480K block size (microseconds) iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array 4000 3500 3000 2500 2000 1500 1000 500 0 iodrive2 Duo 2.4TB Enterprise-Class Disk Array Figure 6. 76% faster continuous checkpoint performance increases transactional processing speeds and business productivity. 2500 SQL Server 2014 In-Memory OLTP Transaction Log Write Latency (microseconds) iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array 2000 1500 1000 500 0 iodrive2 Duo 2.4TB Enterprise-Class Disk Array Figure 7. 91% lower latency delivers consistently fast response times, which improves the user experience and helps IT confidently meet SLAs. 11 W W W. F U S I ON I O.COM
1 11 21 31 41 51 61 71 81 91 101 111 121 131 141 151 161 171 181 191 201 211 221 231 241 251 261 271 281 291 301 311 321 331 341 351 361 371 381 391 401 411 421 431 441 451 461 471 481 491 501 511 521 531 200 180 160 140 120 100 80 60 40 20 0 SQL Server 2014 In-Memory OLTP Transaction Log Write Megabytes/sec iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array iodrive2 Duo 2.4TB Enterprise-Class Disk Array Figure 8. 4X faster transaction log writes eliminates I/O slowdowns. 12 W W W. F U S I ON I O.COM
1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 148 155 162 169 176 183 190 197 204 211 218 225 232 239 246 253 260 267 274 281 288 295 302 309 316 323 330 337 344 351 Megabytes DATABASE STARTUP TESTS DATABASE STARTUP PERFORMANCE The database startup test measured how long it took to load 120GB of data into DRAM. As the chart below shows, the data loaded in 72 seconds from the iodrive2 Duo, as opposed to 222 seconds from the disk array or 308% faster. This performance increases system resilience and facilitates maintenance, enabling organizations to deliver on internal Service Level Agreements (SLAs). 1600 1400 1200 1000 SQL Server 2014 In-Memory OLTP Loading 120GB In-Memory Data Read Megabytes/sec iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array 800 600 400 200 0 iodrive2 Duo 2.4TB Enterprise-Class Disk Array Figure 9. Database startup on iomemory is 308% faster than from an enterprise disk array. 13 W W W. F U S I ON I O.COM
SCALING BEYOND 12,000 USERS In addition to testing transaction processing scalability, it s useful to know the user load each server can support. Accordingly, testing was extended beyond 12,000 users for iomemory. As stated above, our storage array testing peaked at 12,000 users due to severe write latency. With iomemory, our scalability test peaked at 24,000 users before processor utilization reached 80%. SQL Server In-Memory OLTP iodrive2 Duo 2.4TB Memory-Optimized File Group and Transaction Log Performance 450 400 409 350 300 250 200 150 100 50 0 218 248 140 Average of Transaction Log Write Latency (microseconds) Average of Transaction Log Write Megabytes/sec Average of Memory-Optimized Filegroup Write Latency (microseconds) Average of Memory-Optimized Filegroup Write Megabytes/sec Figure 10. With 24,000 users write latency consistently stays under 220 microseconds. 1000000 SQL Server In-Memory OLTP iodrive2 Duo 2.4TB Transactions/sec 100000 10000 1000 100 10 1 Time Series (45 minutes) Figure 11. Database transactions average 109,000 per second once all 24,000 users are on the system. 14 W W W. F U S I ON I O.COM
Summary Fusion iomemory products extend the power of SQL Server 2014 In-Memory OLTP databases by removing I/O bottlenecks associated with transaction logs and checkpoints that become more severe under the workload of an inmemory database. iomemory products also increase startup, reboot, and failover speeds to increase resiliency. Finally, they also increase the user load each server can support. The table below summarizes the benefits. Business Value Key Performance Indicators Enterprise- Class Disk Array with SQL Server 2014 In- Memory OLTP iodrive2 Duo 2.4TB with SQL Server 2014 In- Memory OLTP Improve Customer Experience User Transaction Wait Time (µ) 1329 117 Serve More Customers Transaction Throughput (MB/s) 42 172 Business Process Improvement Reduced user wait times by 91% Increased throughput by 409% Improve Business Productivity Deliver on Internal Service Level Agreements Total Transactions Processed (over 45 minutes) 6,362,883 28,328,639 Database Startup Time (sec) 222 72 Processed 445% more transactions Reduced startup time by 67% 15 W W W. F U S I ON I O.COM
Appendix A: Is SQL Server In-Memory OLTP Right for You? On the surface, workloads that are characterized by a high amount of latching or locking would be classified as good candidates for the SQL Server 2014 In-Memory OLTP feature. While that may be true, Microsoft SQL Server 2014 provides several advisor tools and usage reports to help identify which tables and stored procedures are the best candidates for storing in memory. It is highly recommended that you a) configure the Management Data Warehouse to collect usage analysis on tables and stored procedures; and b) run the advisory tool to determine which tables and stored procedures would benefit from In-Memory OLTP. As mentioned previously, a full database migration isn t required. Database administrators have the flexibility to move individual tables and stored procedures into SQL Server In-Memory OLTP. More information on In-Memory OLTP migration can be found here: http://msdn.microsoft.com/en-us/library/dn247639(v=sql.120).aspx Below is the step-by-step procedure to analyze and migrate data: 1. Configure the Management Data Warehouse (Data Collection > Tasks > Configure Management Data Warehouse). 16 W W W. F U S I ON I O.COM
2. Click 'Next' to progress through the wizard. 17 W W W. F U S I ON I O.COM
3. Create your Management Data Warehouse (MDW) database. 18 W W W. F U S I ON I O.COM
4. Select the appropriate server and newly created MDW database. 19 W W W. F U S I ON I O.COM
5. Add necessary logins and users. 20 W W W. F U S I ON I O.COM
6. Verify actions and click Finish. 21 W W W. F U S I ON I O.COM
7. Click Close. 22 W W W. F U S I ON I O.COM
8. Select the appropriate server and database. Check the box for Transaction Performance Collection Analysis. 23 W W W. F U S I ON I O.COM
9. Click Finish. 24 W W W. F U S I ON I O.COM
10. In Object Explorer navigate to Data Collection, right click Stored Procedure Usage Analysis and select Start. Then right click Table Usage Analysis and select Start. 25 W W W. F U S I ON I O.COM
11. Right click the MDW database. Select Reports > Management Data Warehouse > Transaction Performance Analysis Overview. The AMR tool for In-Memory OLTP opens. : TABLE USAGE ANALYSIS In the AMR tool for In-Memory OLTP you can view analysis on tables, stored procedure usage, and table contention. 26 W W W. F U S I ON I O.COM
The Table Usage chart illustrates which tables would benefit from SQL Server In-Memory OLTP, based on usage. The chart also illustrates the degree of migration difficulty. 27 W W W. F U S I ON I O.COM
TABLE CONTENTION ANALYSIS The Table Contention chart illustrates which tables would benefit from In-Memory OLTP, based on table contention. The chart also illustrates the degree of migration difficulty. 28 W W W. F U S I ON I O.COM
STORED PROCEDURE USAGE ANALYSIS The Stored Procedure report uses a bar chart to identify which procedures will be benefit from In-Memory OLTP based on usage, with CPU time measured in milliseconds. 29 W W W. F U S I ON I O.COM
MEMORY OPTIMIZATION ADVISOR Before starting a table migration to In-Memory OLTP, we recommend reading the migration information on Microsoft TechNet at http://technet.microsoft.com/en-us/library/dn247639(v=sql.120).aspx. Below is a step-by-step procedure for migrating tables: 1. This advisor is based on the recommendations from the AMR reports. Right click the table to migrate and select Memory Optimizer Advisor. 30 W W W. F U S I ON I O.COM
2. Follow the screens in the wizard. : 31 W W W. F U S I ON I O.COM
3. If necessary, fix the items that failed during the Memory Optimization Checklist step. Click Next.. 32 W W W. F U S I ON I O.COM
4. Fix all warnings to ensure a smooth migration.. 33 W W W. F U S I ON I O.COM
5. Name the memory-optimized file group, change the logical file name if necessary, and change the table name.. 34 W W W. F U S I ON I O.COM
6. Select the appropriate primary key column(s) and hash bucket size for the non-clustered index. Click Next. 35 W W W. F U S I ON I O.COM
7. Complete the migration. 36 W W W. F U S I ON I O.COM
NATIVE COMPILATION ADVISOR Below is a step-by-step procedure for using the Native Compilation Advisor: 1. Click Next. 37 W W W. F U S I ON I O.COM
2. Fix the code in the stored procedure associated with the warnings listed in the dialog. 38 W W W. F U S I ON I O.COM
3. Develop workarounds for unsupported TSQL elements. 2014 Fusion-io, Inc. All Rights Reserved. iomemory and iodrive2 Duo are trademarks or registered trademarks of Fusion-io, Inc.; all other product names may be 39 W W W. F U S I ON I O.COM
trademarks of the companies with which they are associated. 40 W W W. F U S I ON I O.COM