Nimble Storage Exchange 2010 20,0-Mailbox Resiliency Storage Solution Tested with: ESRP Storage Version 3.0 Test date: February 10, 2012 Overview This document provides information on Nimble Storage's storage solution for Microsoft Exchange Server, based the Microsoft Exchange Solution Reviewed Program (ESRP) Storage program*. For any questions or comments regarding the contents of this document, see Contact for additional information. *The ESRP Storage program was developed by Microsoft Corporation to provide a common storage testing framework for vendors to provide information on its storage solutions for Microsoft Exchange Server software. For more details on the Microsoft ESRP Storage program, please click http://www.microsoft.com/technet/prodtechnol/exchange/27/esrp.mspx Disclaimer This document has been produced independently of Microsoft Corporation. Microsoft Corporation expressly disclaims responsibility for, and makes no warranty, express or implied, with respect to, the accuracy of the contents of this document. The information contained in this document represents the current view of Nimble Storage on the issues discussed as of the date of publication. Due to changing market conditions, it should not be interpreted to be a commitment on the part of Nimble Storage, and Nimble Storage cannot guarantee the accuracy of any information presented after the date of publication. Features This document describes the testing of a 20,0 mailbox resiliency (2-copy) database availability group (DAG) configuration for Exchange Server 2010 with a Nimble Storage CS-240G storage system. Nimble Storage has developed an entirely new approach to data storage that converges primary and backup storage into one array, dramatically cutting costs and complexity. To better understand why existing storage systems cannot solve today's IT problems, it helps to step back a few years and examine several key storage technologies that are still in use today. Meeting the Needs of the Modern Datacenter The growth of virtualization and server consolidation has compounded the need for storage to simultaneously deliver superior performance, increased capacity, and simplified data protection. N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 1
Traditional storage architectures are unable to comprehensively meet these demands. Nimble s patented Cache Accelerated Sequential Layout (CASL ) is the first storage architecture designed from the ground up to seamlessly integrate SSDs with high-capacity disks, delivering affordable performance, integrated backup and disaster recovery, and pain-free operations. Performance and Capacity Efficiency Nimble s CASL architecture coalesces random writes into a single RAID stripe, improving write performance by 10x -1x compared to traditional systems. Hot data is served from flash which responds to workload changes within milliseconds instead of hours or days. Nimble s variable block-based compression technology delivers capacity savings of 30%-75% with no impact on system performance. Combined with highly efficient snapshots, thin provisioning, and zerocopy clones, Nimble offers the industry s best primary storage density, all in a 3U form factor. Instant Backups and Restores Nimble eliminates the constraints of traditional backup solutions. Thousands of snapshots can be stored with minimal space consumption, eliminating backup windows, allowing for more frequent recovery points, and enabling quick and efficient replication over a WAN. Nimble integration simplifies application-consistent backups for Microsoft, Oracle, and VMware. Backups and restores can be completed in seconds, improving application-level recovery point objectives and recovery time objectives. Pain-Free Operations Nimble arrays are easy to purchase, set up, and use. A single SKU includes all hardware and software features; there are no extra-cost options. The arrays are designed to be managed by an IT generalist. An intuitive UI simplifies management and eliminates configuration and administration complexities. Built-in application profiles and protection templates make it easy to tune storage systems to optimize performance and data protection for individual workloads. It is easy to keep Nimble arrays up and running. Firmware upgrades are completely non-disruptive. Built-in real-time monitoring and phone-home support flag problems before failures occur. Remote support access enables quick triage, troubleshooting, and remediation without the need for onsite access. Solution Description The following section outlines the Exchange 2010 mailbox resiliency solution that Nimble Storage implemented to run the ESRP tests. N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 2
Storage: Nimble Storage CS-240 array Dual Storage Controllers (2) 10 Gbps Ethernet ports per controller with Jumbo Frames Nimble Operating System 1.1 32 TB of Usable Storage Capacity 1.3 TB of Usable Flash Cache Link to CS240 Windows Server Catalog listing: http://windowsservercatalog.com/item.aspx?iditem=4e47eb71-ac19-bba5-7ba1-9f8b9ff2a1d0&bcatid=1282 The primary copy storage is described in this document. The secondary copy storage is configured identically (from the host on down to the spindle including brand, model, firmware, drivers, etc.). N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 3
The Exchange DAG storage was configured as follows: 2 Active Mailbox Servers and 2 Passive Mailbox Servers. 2 Nimble Storage CS-240G Arrays. 10 Active Databases, 2,0 mailboxes per database. Nimble Storage arrays use double-parity to protect against drive failure. The Nimble Storage CASL file system uses a block checksum to ensure data integrity. 2 copies of each database. The ESRP-Storage program focuses on storage solution testing to address performance and reliability issues with storage design. However, storage is not the only factor to take into consideration when designing a scale up Exchange solution. Other factors which affect the server scalability are: server processor utilization, server physical and virtual memory limitations, resource requirements for other applications, directory and network service latencies, network infrastructure limitations, replication and recovery requirements, and client usage profiles. All these factors are beyond the scope for ESRP- Storage. Therefore, the number of mailboxes hosted per server as part of the tested configuration may not necessarily be viable for some customer deployment. For more information on identifying and addressing performance bottlenecks in an Exchange system, please refer to Microsoft's Troubleshooting Microsoft Exchange Server Performance, available at http://technet.microsoft.com/en-us/library/dd335215.aspx. Targeted Customer Profile The Nimble Storage solution for a mid to large-sized Enterprise Exchange environment is designed to support: 20,0 Mailboxes 4 Servers (2 Active, 2 Passive) 0.15 IOPS / Mailbox (0.23 IOPS / Mailbox achieved for 53% headroom) 1,0 MB per mailbox 24 x 7 background database maintenance enabled Providing controller capacity for replication and disk reconstruction Permitting space for backups and restores using Nimble Storage Snapshot Backup technology Mailbox Resiliency (2-Copy) Tested Deployment The following tables summarize the testing environment: N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 4
Simulated Exchange Configuration: Number of Exchange mailboxes simulated 20,0 Number of Database Availability Groups (DAGs) 2 Number of servers/dag 2 Number of active mailboxes/server 10,0 Number of databases/host 5 (10 Total) Number of copies/database 2 Number of mailboxes/database 2,0 Simulated profile: s per second per mailbox (IOPS, include 20% headroom) 0.15 IOPS / Mailbox (0.23 IOPS / Mailbox achieved for 53% headroom) Database LUN size 2.08 TB Log LUN size 5 GB Total database size for performance testing 21.6 TB % storage capacity used by Exchange 69% database** **Storage performance characteristics change based on the percentage utilization of the individual disks. Tests that use a small percentage of the storage (~25%) may exhibit reduced throughput if the storage capacity utilization is significantly increased beyond what is tested in this paper. Storage Hardware Storage Connectivity (Fiber Channel, SAS, SATA, iscsi SAN iscsi) Storage model and OS/firmware revision CS240 Nimble OS 1.1 Storage cache 1 GB NVRAM Cache (Write), plus 24 GB RAM Cache, plus 1.3 TB Flash Cache (Read) Number of storage controllers 2 Number of storage ports (2) 10 Gbps Ethernet per Controller Maximum bandwidth of storage connectivity to 20 Gbps host Switch type/model/firmware revision Blade RackSwitch G8124 10 Gigabit Ethernet Switch HBA model and firmware Silicom PE210G2SPI9-XR v:1.4 Jumbo Frames enabled Number of HBA s/host 2 Total number of disks tested in solution 12 SATA and 4 Flash Cache drives per array Maximum number of spindles can be hosted in 12 SATA and 4 Flash Cache drives per array the storage Storage Software HBA driver Intel 10 Gbps PCIe v. 2.1.4 HBA QueueTarget Setting N/A HBA QueueDepth Setting 512 Multi-Pathing Nimble Operating System 1.1, Windows MPIO Host OS Windows 28 R2 SP1 ESE.dll file version 14.0.639.19 Replication solution name/version N/A for Exchange 2010 DAG Solutions N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 5
Storage Disk Configuration (Mailbox Store/Log Disks) Disk type, speed and firmware revision SATA 7,2 RPM, Flash Cache Raw capacity per disk (GB) SATA: 2 TB Flash Cache: 160 GB Number of physical disks in test SATA: 12 Flash Cache: 4 Total raw storage capacity (GB) SATA: 24 TB Flash Cache: 640 GB Nimble native 50% compression SATA: 32 TB usable Flash Cache: 1.3 TB usable Disk slice size (GB) N/A Number of slices per LUN or number of disks per 12 Disks for both database and log files LUN Raid level RAID 6 *** Total formatted capacity 24 TB Storage capacity utilization**** 1% (Formatted Capacity / Raw Capacity) 75% (Formatted Capacity / Usable Capacity) Database capacity utilization**** 108% (Database & Log Size / Raw Capacity) 80.6% (Database & Log Size / Usable Capacity) ***Nimble's CASL file system uses a performance optimized RAID6 scheme that greatly reduces the rebuild time of traditional RAID6 implementations. CASL intelligently decides which drive sectors to rebuild (for example only populated data blocks are rebuilt and not empty drive space). As a result a lightly loaded array can rebuild a whole 1TB drive in about 4 hours, and a 2TB drive in about 7 hours. If the drive is not full the rebuild time would be proportionally shorter. So a half full drive 2TB drive would only take 3.5 hours to rebuild on a lightly loaded system. When the storage array is very busy, rebuild times can take longer to reduce performance latency for active applications. **** Nimble Storage arrays use real-time compression algorithms that provide 50% capacity savings over raw disk sizes. Thus, usable capacity is 50% larger than the raw capacity of the storage and represents a more realistic expectation for sizing than raw capacity. Best Practices Exchange server is a disk-intensive application. Based on the testing run using the ESRP framework, we would recommend the following to improve the storage performance. For Exchange 2010 best practices on storage design, please visit http://technet.microsoft.com/enus/library/dd346703.aspx Nimble Storage provides an auto-tuning storage solution that leverages flash cache and greatly reduces the amount of effort to size and layout Exchange mailstores. Core Storage N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 6
1. This solution architecture was designed to show the maximum Exchange performance capability of a Nimble Storage array. Environments smaller than 20,0 mailboxes routinely share Nimble arrays with other applications such as SQL Server and SharePoint. 2. Separate Exchange Database and Log volumes since they have different performance characteristics and will tune differently. Use the Nimble OS performance policies for Exchange Database and Exchange Logs when creating the respective volumes. 3. Nimble Storage arrays automatically provision storage from a pool of high-density disks protected by double-parity and are intelligently accelerated using NVRAM, DRAM and Flash SSD Cache. Nimble's CASL File System leverages these storage technologies natively to automatically provide scale-out performance dramatically higher than traditional tiered storage architectures. Therefore, you do not need to spend extensive time provisioning storage such that legacy storage architectures require, Nimble does this for you. 4. Please consult the "Nimble Storage Best Practices Guide for Exchange 2010" for current implementation best practices. Backup strategy Nimble Storage provides the first converged storage platform that combines primary storage and backup storage in the same architecture. Virtually all enterprise storage systems provide the ability to take snapshots, or point-in-time copies of data. Snapshots provide a very fast and efficient way to recover from user errors and application crashes. However, enterprises are usually limited to keeping at most a few days worth of snapshots on disk, because storing multiple snapshots can consume large amounts of expensive and limited primary storage capacity. This mandates backing up primary data to a separate disk-based backup system frequently in order to store 60-90 days worth of backups. CASL enables instant, application-consistent backups on the same array with very efficient (up to 20x) backup capacity optimization. It captures compressed, incremental snapshots of the primary data on low-cost drives at pre-configured intervals. These backups are both extremely fast (on the order of milliseconds) and non-disruptive, because they do not impact application or storage performance. It therefore becomes feasible for enterprises to capture frequent backups spaced just minutes apart (instead of the typical daily backup), enabling much finer recovery point objectives (RPOs) and improving data protection. The Nimble approach significantly reduces costs by eliminating the need for a separate backup tier. It also eliminates the complexity and administrative overhead of managing massive data copies between tiers. Finally, it enables much faster restores, with recovery time objectives (RTOs) measured in seconds instead of hours in the event of application corruption or user error, further improving application availability. Contact for additional information N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 7
Nimble Storage, Inc. 2740 Zanker Rd. San Jose, CA 95134 Main: 408-432-96 Support: 877-3NIMBLE (877-364-6253) Fax: 408-899-5158 info@nimblestorage.com N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 8
Sales To learn more about Nimble Storage s breakthrough technology or to schedule a briefing, please contact our technical sales team: Sales: 877-3NIMBLE (877-364-6253) sales@nimblestorage.com Test Result Summary This section provides a high level summary of the test data from ESRP and the link to the detailed html reports which are generated by ESRP testing framework. Please click on the underlined headings below to view the html report for each test. Reliability A number of tests in the framework are to check Reliability tests runs for 24 hours. The goal is to verify the storage can handle high IO load for a long period of time. Both log and database files will be analyzed for integrity after the stress test to ensure no database/log corruption. The following list provides an overview: (click on the underlined word will show the html report after the reliability tests run) There were no errors reported in any of the Event Logs collected for the reliability tests. There were no errors reported during the database and log checksum process. Storage Performance Results The Primary Storage performance testing is designed to exercise the storage with maximum sustainable Exchange type of IO for 2 hours. The test is to show how long it takes for the storage to respond to an IO under load. The data below is the sum of all of the logical disk s and average of all the logical disks latency in the 2 hours test duration. Each server is listed separately and the aggregate numbers across all servers is listed as well. Individual Server Metrics: The sum of s across Storage Groups and the average latency across all Storage Groups on a per server basis. Host 1 Database Database Disks Transfers/sec 2,452.799 Database Disks /sec 1,544.614 Database Disks /sec 908.187 Database Disk Read Latency (ms) 14.6144 Database Disk Write Latency (ms) 10.9306 N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 9
Transaction Log Log Disks /sec 386.414 Log Disk Write Latency (ms) 4.045 Host 2 Database Database Disks Transfers/sec 2,215.107 Database Disks /sec 1,395.669 Database Disks /sec 819.437 Database Disk Read Latency (ms) 15.8786 Database Disk Write Latency (ms) 11.0924 Transaction Log Log Disks /sec 362.483 Log Disk Write Latency (ms) 4.0564 Aggregate Performance across all servers Metrics: The sum of s across servers in solution and the average latency across all all servers in solution. Database Database Disks Transfers/sec 4,667.906 Database Disks /sec 2,940.283 Database Disks /sec 1,727.624 Database Disk Read Latency (ms) 15.2465 Database Disk Write Latency (ms) 11.0115 Transaction Log Log Disks /sec 748.897 Log Disk Write Latency (ms) 4.0507 Database Backup/Recovery Performance There are two tests reports in this section. The first one is to measure the sequential read rate of the database files, and the second is to measure the recovery/replay performance (playing transaction logs in to the database). Database Read-only Performance The test is to measure the maximum rate at which databases could be backed up via VSS. The following table shows the average rate for a single database file. Host 1 MB read/sec per database 59.88 MB read/sec total per server 299.4 Host 2 MB read/sec per database 60.122 N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 10
MB read/sec total per server 3.61 Transaction Log Recovery/Replay Performance The test is to measure the maximum rate at which the log files can be played against the databases. The following table shows the average rate for 5 log files played in a single storage group. Each log file is 1 MB in size. Host 1 time to play one Log file (sec) 1.124 Host 2 time to play one Log file (sec) 1.101 Conclusion This report demonstrates Nimble Storage's ability to meet the performance needs of a 20,0 mailbox Exchange 2010 implementation at 0.15 IOPS and 1,024 MB per mailbox. Nimble's CASL file system provides true efficiency by combining high-density storage with flash SSD to enable groundbreaking performance within a small rack form factor. The Nimble storage architecture also provides additional space for highly-efficient redirect on write snapshots to provide fast full backups of Exchange using Microsoft VSS and Nimble Protection Manager. This document is developed by Nimble Storage, and reviewed by Microsoft Exchange Product team. The test results/data presented in this document is based on the tests introduced in the ESRP test framework. Customer should not quote the data directly for his/her pre-deployment verification. It is still necessary to go through the exercises to validate the storage design for a specific customer environment. ESRP program is not designed to be a benchmarking program; tests are not designed to getting the maximum throughput for a giving solution. Rather, it is focused on producing recommendations from vendors for Exchange application. So the data presented in this document should not be used for direct comparisons among the solutions. Nimble Storage provides complete sizing information for Exchange Server deployments through its systems engineers. Contact your local Nimble Storage sales office for additional information. N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 11
Appendix A - Test Results This section provides representative test results from one of the two servers under test: Host 1 (MKTG01). It is also representative of both server's test results, which are virtually identical. 24-Hour Stress Test Results Test Summary Overall Test Result Pass Machine Name MKTG01 Test Description Test Start Time 1/31/2012 8:54:03 AM Test End Time 2/1/2012 9:32:59 AM Collection Start Time 1/31/2012 8:56:58 AM Collection End Time 2/1/2012 8:56:51 AM Jetstress Version 14.01.0225.017 ESE Version 14..0639.019 Operating System Windows Server 28 R2 Enterprise Service Pack 1 (6.1.7601.65536) Performance Log C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - 24 hour\performance_2012_1_31_8_54_14.blg Database Sizing and Throughput Achieved Transactional per Second 2045.649 Target Transactional per Second 15 Initial Database Size (bytes) 10751591120896 Final Database Size (bytes) 108032901120 Database Files (Count) 5 Jetstress System Parameters Thread Count 10 (per database) Minimum Database Cache 160.0 MB Maximum Database Cache 1280.0 MB Insert Operations 40% Delete Operations 20% Replace Operations 5% Read Operations 35% Lazy Commits 70% Run Background Database Maintenance True Number of Copies per Database 2 N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 12
Database Configuration Instance6052.1 Instance6052.2 Instance6052.3 Instance6052.4 Instance6052.5 Log path: O:\ Database: E:\Jetstress11.edb Log path: P:\ Database: F:\Jetstress21.edb Log path: Q:\ Database: G:\Jetstress31.edb Log path: R:\ Database: H:\Jetstress41.edb Log path: T:\ Database: I:\Jetstress51.edb Transactional Performance MSExchange Database ==> Instances Database Latency Database Latency Database /sec Database /sec Database Database Log Latency Log Latency Log /sec Log /sec Instance6052.1 19.516 9.864 256.392 151.378 34051.951 34525.283 0.0 3.023 0.0 74.459 0.0 5583.360 Instance6052.2 16.651 9.822 256.202 152.312 34211.876 34502.757 0.0 3.041 0.0 73.721 0.0 5664.190 Instance6052.3 16.430 9.371 256.890 152.704 34210.104 34502.129 0.0 2.944 0.0 74.432 0.0 5610.964 Instance6052.4 16.536 8.850 256.975 152.757 34221.165 34504.856 0.0 3.043 0.0 73.651 0.0 5661.173 Instance6052.5 16.453 8.334 257.192 152.847 34166.514 34493.693 0.0 3.047 0.0 73.650 0.0 5647.076 Log Log Background Database Maintenance Performance MSExchange Database ==> Instances Database Maintenance IO /sec Instance6052.1 21.570 261953.333 Instance6052.2 21.510 261954.914 Instance6052.3 22.199 261944.620 Instance6052.4 21.677 261950.326 Instance6052.5 22.069 261953.161 Database Maintenance IO Log Replication Performance MSExchange Database ==> Instances Log /sec Log Instance6052.1 1.715 232393.322 Instance6052.2 1.727 232270.744 Instance6052.3 1.725 232352.057 Instance6052.4 1.724 232273.491 Instance6052.5 1.719 232310.837 N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 13
Total Performance MSExchange Database ==> Instances Database Database Database Latency Latency /sec Database /sec Database Database Log Latency Log Latency Log /sec Log /sec Log Instance6052.1 19.516 9.864 277.962 151.378 51737.452 34525.283 3.995 3.023 1.715 74.459 232393.322 5583.360 Instance6052.2 16.651 9.822 277.713 152.312 51851.835 34502.757 4.073 3.041 1.727 73.721 232270.744 5664.190 Instance6052.3 16.430 9.371 279.089 152.704 52324.409 34502.129 3.942 2.944 1.725 74.432 232352.057 5610.964 Instance6052.4 16.536 8.850 278.652 152.757 51936.804 34504.856 4.037 3.043 1.724 73.651 232273.491 5661.173 Instance6052.5 16.453 8.334 279.261 152.847 52167.654 34493.693 4.145 3.047 1.719 73.650 232310.837 5647.076 Log Host System Performance Counter Minimum Maximum % Processor Time 4.183 0.886 6.777 Available M 59657.244 59622.0 59756.0 Free System Page Table Entries 33555624.767 335553.0 33556356.0 Transition Pages RePurposed/sec 0.0 0.0 0.0 Pool Nonpaged 75450354.451 74825728.0 75911168.0 Pool Paged 215865863.131 214663168.0 217677824.0 Database Page Fault Stalls/sec 0.0 0.0 0.0 Test Log 1/31/2012 8:54:03 AM -- Jetstress testing begins... 1/31/2012 8:54:03 AM -- Preparing for testing... 1/31/2012 8:54:09 AM -- Attaching databases... 1/31/2012 8:54:09 AM -- Preparations for testing are complete. 1/31/2012 8:54:09 AM -- Starting transaction dispatch.. 1/31/2012 8:54:09 AM -- Database cache settings: (minimum: 160.0 MB, maximum: 1.2 GB) 1/31/2012 8:54:09 AM -- Database flush thresholds: (start: 12.8 MB, stop: 25.6 MB) 1/31/2012 8:54:14 AM -- Database read latency thresholds: (average: 20 msec/read, maximum: 1 msec/read). 1/31/2012 8:54:14 AM -- Log write latency thresholds: (average: 10 msec/write, maximum: 1 msec/write). 1/31/2012 8:54:24 AM -- Operation mix: Sessions 10, Inserts 40%, Deletes 20%, Replaces 5%, 35%, Lazy Commits 70%. 1/31/2012 8:54:24 AM -- Performance logging started (interval: 150 ms). 1/31/2012 8:54:24 AM -- Attaining prerequisites: 1/31/2012 8:56:58 AM -- \MSExchange Database(JetstressWin)\Database Cache Size, Last: 12099010.0 (lower bound: 120796.0, upper bound: none) 2/1/2012 8:56:58 AM -- Performance logging has ended. 2/1/2012 9:32:55 AM -- JetInterop batch transaction stats: 721310, 724430, 723810, 723196 and 723803. 2/1/2012 9:32:55 AM -- Dispatching transactions ends. 2/1/2012 9:32:55 AM -- Shutting down databases... 2/1/2012 9:32:59 AM -- Instance6052.1 (complete), Instance6052.2 (complete), Instance6052.3 (complete), Instance6052.4 (complete) and Instance6052.5 (complete) 2/1/2012 9:32:59 AM -- C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - 24 hour\performance_2012_1_31_8_54_14.blg has 5754 samples. 2/1/2012 9:32:59 AM -- Creating test report... 2/1/2012 9:33:36 AM -- Instance6052.1 has 19.5 for Database Latency. 2/1/2012 9:33:36 AM -- Instance6052.1 has 3.0 for Log Latency. 2/1/2012 9:33:36 AM -- Instance6052.1 has 3.0 for Log Latency. 2/1/2012 9:33:36 AM -- Instance6052.2 has 16.7 for Database Latency. 2/1/2012 9:33:36 AM -- Instance6052.2 has 3.0 for Log Latency. 2/1/2012 9:33:36 AM -- Instance6052.2 has 3.0 for Log Latency. 2/1/2012 9:33:36 AM -- Instance6052.3 has 16.4 for Database Latency. 2/1/2012 9:33:36 AM -- Instance6052.3 has 2.9 for Log Latency. N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 14
2/1/2012 9:33:36 AM -- Instance6052.3 has 2.9 for Log Latency. 2/1/2012 9:33:36 AM -- Instance6052.4 has 16.5 for Database Latency. 2/1/2012 9:33:36 AM -- Instance6052.4 has 3.0 for Log Latency. 2/1/2012 9:33:36 AM -- Instance6052.4 has 3.0 for Log Latency. 2/1/2012 9:33:36 AM -- Instance6052.5 has 16.5 for Database Latency. 2/1/2012 9:33:36 AM -- Instance6052.5 has 3.0 for Log Latency. 2/1/2012 9:33:36 AM -- Instance6052.5 has 3.0 for Log Latency. 2/1/2012 9:33:36 AM -- Test has 0 Maximum Database Page Fault Stalls/sec. 2/1/2012 9:33:36 AM -- The test has 0 Database Page Fault Stalls/sec samples higher than 0. 2/1/2012 9:33:36 AM -- C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - 24 hour\performance_2012_1_31_8_54_14.xml has 5743 samples queried. 2-Hour Stress Test Results Test Summary Overall Test Pass Result Machine Name MKTG01 Test Description Test Start Time 1/30/2012 3:05:33 PM Test End Time 1/30/2012 5:29:44 PM Collection Start 1/30/2012 3:07:17 PM Time Collection End 1/30/2012 5:07:14 PM Time Jetstress Version 14.01.0225.017 ESE Version 14..0639.019 Operating Windows Server 28 R2 Enterprise Service Pack 1 (6.1.7601.65536) System Performance Log C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - 2 hour\performance_2012_1_30_15_5_44.blg Database Sizing and Throughput Achieved Transactional per Second 2452.799 Target Transactional per Second 15 Initial Database Size (bytes) 10745878478848 Final Database Size (bytes) 10751591120896 Database Files (Count) 5 Jetstress System Parameters Thread Count 10 (per database) Minimum Database Cache 160.0 MB Maximum Database Cache 1280.0 MB Insert Operations 40% Delete Operations 20% Replace Operations 5% Read Operations 35% Lazy Commits 70% Run Background Database Maintenance True Number of Copies per Database 2 N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 15
Database Configuration Instance6052.1 Log path: O:\ Database: E:\Jetstress11.edb Instance6052.2 Log path: P:\ Database: F:\Jetstress21.edb Instance6052.3 Log path: Q:\ Database: G:\Jetstress31.edb Instance6052.4 Log path: R:\ Database: H:\Jetstress41.edb Instance6052.5 Log path: T:\ Database: I:\Jetstress51.edb Transactional Performance MSExchange Database ==> Instances Database Latency Database Latency Database /sec Database /sec Database Database Log Latency Log Latency Log /sec Log /sec Instance6052.1 15.609 11.635 306.908 180.5 33673.510 34735.654 0.0 4.045 0.0 77.310 0.0 6160.546 Instance6052.2 14.335 11.455 310.682 182.622 33667.575 34712.544 0.0 4.018 0.0 77.470 0.0 6170.807 Instance6052.3 14.567 11.092 306.390 180.333 33558.656 34705.871 0.0 4.019 0.0 76.622 0.0 6152.962 Instance6052.4 14.177 10.507 311.384 183.376 33816.012 34717.885 0.0 4.026 0.0 77.724 0.0 6209.342 Instance6052.5 14.384 9.964 309.250 181.851 33821.313 34714.755 0.0 4.010 0.0 77.288 0.0 6192.924 Background Database Maintenance Performance MSExchange Database ==> Instances Database Maintenance IO /sec Instance6052.1 26.033 261953.837 Instance6052.2 26.183 261949.444 Instance6052.3 24.934 261934.882 Instance6052.4 26.613 261924.591 Instance6052.5 26.043 261942.496 Log Replication Performance MSExchange Database ==> Instances Log /sec Log Instance6052.1 1.978 232562.362 Instance6052.2 1.986 232563.526 Instance6052.3 1.959 232560.181 Instance6052.4 2.4 232574.724 Instance6052.5 1.989 232559.090 Total Performance MSExchange Database ==> Instances Database Database Database Latency Latency /sec Database /sec Database Database Log Latency Log Log Database Maintenance IO Log Latency Log /sec Log /sec Log Instance6052.1 15.609 11.635 332.941 180.5 51523.121 34735.654 5.303 4.045 1.978 77.310 232562.362 6160.546 Instance6052.2 14.335 11.455 336.864 182.622 51410.668 34712.544 4.857 4.018 1.986 77.470 232563.526 6170.807 Instance6052.3 14.567 11.092 331.324 180.333 50745.323 34705.871 4.740 4.019 1.959 76.622 232560.181 6152.962 Log N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 16
Instance6052.4 14.177 10.507 337.997 183.376 51776.594 34717.885 5.586 4.026 2.4 77.724 232574.724 6209.342 Instance6052.5 14.384 9.964 335.293 181.851 51539.958 34714.755 4.893 4.010 1.989 77.288 232559.090 6192.924 Host System Performance Counter Minimum Maximum % Processor Time 4.913 1.776 6.604 Available M 59732.326 59716.0 59809.0 Free System Page Table Entries 33555629.785 335553.0 33555846.0 Transition Pages RePurposed/sec 0.0 0.0 0.0 Pool Nonpaged 75969512.484 75288576.0 76279808.0 Pool Paged 216220426.154 216178688.0 216272896.0 Database Page Fault Stalls/sec 0.0 0.0 0.0 Test Log 1/30/2012 3:05:33 PM -- Jetstress testing begins... 1/30/2012 3:05:33 PM -- Preparing for testing... 1/30/2012 3:05:39 PM -- Attaching databases... 1/30/2012 3:05:39 PM -- Preparations for testing are complete. 1/30/2012 3:05:39 PM -- Starting transaction dispatch.. 1/30/2012 3:05:39 PM -- Database cache settings: (minimum: 160.0 MB, maximum: 1.2 GB) 1/30/2012 3:05:39 PM -- Database flush thresholds: (start: 12.8 MB, stop: 25.6 MB) 1/30/2012 3:05:44 PM -- Database read latency thresholds: (average: 20 msec/read, maximum: 1 msec/read). 1/30/2012 3:05:44 PM -- Log write latency thresholds: (average: 10 msec/write, maximum: 1 msec/write). 1/30/2012 3:05:53 PM -- Operation mix: Sessions 10, Inserts 40%, Deletes 20%, Replaces 5%, 35%, Lazy Commits 70%. 1/30/2012 3:05:53 PM -- Performance logging started (interval: 150 ms). 1/30/2012 3:05:53 PM -- Attaining prerequisites: 1/30/2012 3:07:17 PM -- \MSExchange Database(JetstressWin)\Database Cache Size, Last: 12122810.0 (lower bound: 120796.0, upper bound: none) 1/30/2012 5:07:18 PM -- Performance logging has ended. 1/30/2012 5:29:37 PM -- JetInterop batch transaction stats: 79942, 79968, 79440, 80346 and 80139. 1/30/2012 5:29:37 PM -- Dispatching transactions ends. 1/30/2012 5:29:37 PM -- Shutting down databases... 1/30/2012 5:29:44 PM -- Instance6052.1 (complete), Instance6052.2 (complete), Instance6052.3 (complete), Instance6052.4 (complete) and Instance6052.5 (complete) 1/30/2012 5:29:44 PM -- C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - 2 hour\performance_2012_1_30_15_5_44.blg has 484 samples. 1/30/2012 5:29:44 PM -- Creating test report... 1/30/2012 5:29:48 PM -- Instance6052.1 has 15.6 for Database Latency. 1/30/2012 5:29:48 PM -- Instance6052.1 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Instance6052.1 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Instance6052.2 has 14.3 for Database Latency. 1/30/2012 5:29:48 PM -- Instance6052.2 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Instance6052.2 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Instance6052.3 has 14.6 for Database Latency. 1/30/2012 5:29:48 PM -- Instance6052.3 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Instance6052.3 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Instance6052.4 has 14.2 for Database Latency. 1/30/2012 5:29:48 PM -- Instance6052.4 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Instance6052.4 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Instance6052.5 has 14.4 for Database Latency. 1/30/2012 5:29:48 PM -- Instance6052.5 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Instance6052.5 has 4.0 for Log Latency. 1/30/2012 5:29:48 PM -- Test has 0 Maximum Database Page Fault Stalls/sec. 1/30/2012 5:29:48 PM -- The test has 0 Database Page Fault Stalls/sec samples higher than 0. N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 17
1/30/2012 5:29:48 PM -- C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - 2 hour\performance_2012_1_30_15_5_44.xml has 478 samples queried. Database Backup Test Results Database Backup Statistics - All Database Instance Database Size (M) Elapsed Backup Time M Transferred/sec Instance2404.1 2048572.09 09:30:40 59.83 Instance2404.2 2048580.09 09:29:56 59.91 Instance2404.3 2048580.09 09:30:18 59.87 Instance2404.4 2048580.09 09:29:43 59.93 Instance2404.5 2048580.09 09:30:24 59.86 Jetstress System Parameters Thread Count 10 (per database) Minimum Database Cache 160.0 MB Maximum Database Cache 1280.0 MB Insert Operations 40% Delete Operations 20% Replace Operations 5% Read Operations 35% Lazy Commits 70% Database Configuration Instance2404.1 Log path: O:\ Database: E:\Jetstress11.edb Instance2404.2 Log path: P:\ Database: F:\Jetstress21.edb Instance2404.3 Log path: Q:\ Database: G:\Jetstress31.edb Instance2404.4 Log path: R:\ Database: H:\Jetstress41.edb Instance2404.5 Log path: T:\ Database: I:\Jetstress51.edb Transactional Performance MSExchange Database ==> Instances Instance240 4.1 Databa se Averag e Latenc y Databa se Averag e Latenc y Databas e /s ec 6.707 0.0 238.99 8 Databas e /s ec Database 0.0 262144.0 Databa se Averag e Log Avera ge Laten cy (msec ) Log Avera ge Laten cy (msec ) Log /s ec Log /s ec Log Avera ge Log Avera ge 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Instance240 7.390 0.0 239.61 0.0 262144.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 18
4.2 4 Instance240 4.3 Instance240 4.4 Instance240 4.5 7.106 0.0 239.19 3 7.396 0.0 239.62 9 7.354 0.0 239.39 8 0.0 262144.0 0.0 262144.0 0.0 262144.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Host System Performance Counter Minimum Maximum % Processor Time 5.267 0.690 8.456 Available M 61997.058 61903.0 624.0 Free System Page Table Entries 33555701.541 33555306.0 33556380.0 Transition Pages RePurposed/sec 0.0 0.0 0.0 Pool Nonpaged 55754036.369 55455744.0 56090624.0 Pool Paged 99484795.168 97026048.0 134209536.0 Database Page Fault Stalls/sec 0.0 0.0 0.0 Test Log 2/9/2012 11:52:53 AM -- Jetstress testing begins... 2/9/2012 11:52:53 AM -- Preparing for testing... 2/9/2012 11:52:59 AM -- Attaching databases... 2/9/2012 11:52:59 AM -- Preparations for testing are complete. 2/9/2012 11:53:09 AM -- Performance logging started (interval: 3 ms). 2/9/2012 11:53:09 AM -- Backing up databases... 2/9/2012 9:23:49 PM -- Performance logging has ended. 2/9/2012 9:23:49 PM -- Instance2404.1 (1% processed), Instance2404.2 (1% processed), Instance2404.3 (1% processed), Instance2404.4 (1% processed) and Instance2404.5 (1% processed) 2/9/2012 9:23:49 PM -- C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - Backup\DatabaseBackup_2012_2_9_11_52_59.blg has 1139 samples. 2/9/2012 9:23:49 PM -- Creating test report... Soft Recovery Test Results Soft-Recovery Statistics - All Database Instance Log files replayed Elapsed seconds Instance2448.1 501 572.396198 Instance2448.2 504 553.8321656 Instance2448.3 506 571.0701957 Instance2448.4 5 570.8049952 Instance2448.5 504 559.4013753 Database Configuration Instance2448.1 Log path: O:\ Database: E:\Jetstress11.edb Instance2448.2 Log path: P:\ Database: F:\Jetstress21.edb Instance2448.3 Log path: Q:\ Database: G:\Jetstress31.edb Instance2448.4 Log path: R:\ N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 19
Database: H:\Jetstress41.edb Instance2448.5 Log path: T:\ Database: I:\Jetstress51.edb Transactional Performance MSExchang e Database ==> Instances Instance24 48.1 Instance24 48.2 Instance24 48.3 Instance24 48.4 Instance24 48.5 Databa se Averag e Latenc y 26.67 8 23.30 1 23.66 6 23.56 4 23.87 6 Databa se Averag e Latenc y 16.50 6 15.21 5 14.40 1 13.71 6 14.35 4 Databa se / sec 878.52 5 9.76 5 879.36 7 873.36 7 892.41 8 Databa se / sec Databas e 5.171 36102.7 29 5.401 366.9 04 5.261 36048.0 20 5.209 36060.8 03 5.387 36076.0 36 Databas e 32768.0 32768.0 32768.0 32768.0 32768.0 Log Read s Avera ge Laten cy (msec ) Log Write s Avera ge Laten cy (msec ) Log / sec Log / sec Log 3.983 0.0 7.756 0.0 232581. 238 3.941 0.022 8.098 0.5 232603. 549 4.9 0.0 7.892 0.0 232550. 402 4.140 0.0 7.813 0.0 232561. 899 4.227 0.0 8.081 0.0 232299. 834 Log Write s Avera ge 0.0 3.765 0.0 0.0 0.0 Background Database Maintenance Performance MSExchange Database ==> Database Maintenance IO Instances /sec Instance2448.1 26.182 261844.264 Instance2448.2 26.541 2625.203 Instance2448.3 26.530 261785.962 Instance2448.4 26.369 262014.727 Instance2448.5 26.255 261970.675 Database Maintenance IO Total Performance MSExchang e Database ==> Instances Instance24 48.1 Instance24 48.2 Instance24 48.3 Databa se Averag e Latenc y 26.67 8 23.30 1 23.66 6 Databa se Averag e Latenc y 16.50 6 15.21 5 14.40 1 Databa se / sec 904.70 7 927.30 6 905.89 7 Databa se / sec Databas e 5.171 42635.6 22 5.401 42475.3 41 5.261 42659.0 08 Databas e 32768.0 32768.0 32768.0 Log Read s Avera ge Laten cy (msec ) Log Write s Avera ge Laten cy (msec ) Log / sec Log / sec Log 3.983 0.0 7.756 0.0 232581. 238 3.941 0.022 8.098 0.5 232603. 549 4.9 0.0 7.892 0.0 232550. 402 Log Write s Avera ge 0.0 3.765 0.0 N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 20
Instance24 48.4 Instance24 48.5 23.56 4 23.87 6 13.71 6 14.35 4 899.73 7 918.67 3 5.209 42683.0 26 32768.0 4.140 0.0 7.813 0.0 232561. 899 4.227 0.0 8.081 0.0 232299. 834 0.0 5.387 42531.9 63 32768.0 0.0 Host System Performance Counter Minimum Maximum % Processor Time 7.166 3.893 10.496 Available M 60611.092 60569.0 61612.0 Free System Page Table Entries 33555662.596 33555324.0 33556380.0 Transition Pages RePurposed/sec 0.0 0.0 0.0 Pool Nonpaged 55925302.468 54120448.0 56389632.0 Pool Paged 94358883.858 94244864.0 94470144.0 Database Page Fault Stalls/sec 0.0 0.0 0.0 Test Log 2/8/2012 3:13:21 PM -- Jetstress testing begins... 2/8/2012 3:13:21 PM -- Preparing for testing... 2/8/2012 3:13:27 PM -- Attaching databases... 2/8/2012 3:13:27 PM -- Preparations for testing are complete. 2/8/2012 3:13:27 PM -- Starting transaction dispatch.. 2/8/2012 3:13:27 PM -- Database cache settings: (minimum: 160.0 MB, maximum: 1.2 GB) 2/8/2012 3:13:27 PM -- Database flush thresholds: (start: 12.8 MB, stop: 25.6 MB) 2/8/2012 3:13:33 PM -- Database read latency thresholds: (average: 20 msec/read, maximum: 1 msec/read). 2/8/2012 3:13:33 PM -- Log write latency thresholds: (average: 10 msec/write, maximum: 1 msec/write). 2/8/2012 3:13:41 PM -- Operation mix: Sessions 10, Inserts 40%, Deletes 20%, Replaces 5%, 35%, Lazy Commits 70%. 2/8/2012 3:13:41 PM -- Performance logging started (interval: 150 ms). 2/8/2012 3:13:41 PM -- Generating log files... 2/8/2012 3:50:02 PM -- O:\ (1.4% generated), P:\ (1.8% generated), Q:\ (101.4% generated), R:\ (1.2% generated) and T:\ (101.0% generated) 2/8/2012 3:50:02 PM -- Performance logging has ended. 2/8/2012 3:50:02 PM -- JetInterop batch transaction stats: 21358, 21569, 21538, 21320 and 21636. 2/8/2012 3:50:02 PM -- Dispatching transactions ends. 2/8/2012 3:50:02 PM -- Shutting down databases... 2/8/2012 3:50:10 PM -- Instance2448.1 (complete), Instance2448.2 (complete), Instance2448.3 (complete), Instance2448.4 (complete) and Instance2448.5 (complete) 2/8/2012 3:50:10 PM -- C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - Soft Recovery\Performance_2012_2_8_15_13_33.blg has 145 samples. 2/8/2012 3:50:10 PM -- Creating test report... 2/8/2012 3:50:11 PM -- Instance2448.1 has 14.9 for Database Latency. 2/8/2012 3:50:11 PM -- Instance2448.1 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Instance2448.1 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Instance2448.2 has 14.2 for Database Latency. 2/8/2012 3:50:11 PM -- Instance2448.2 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Instance2448.2 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Instance2448.3 has 14.1 for Database Latency. 2/8/2012 3:50:11 PM -- Instance2448.3 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Instance2448.3 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Instance2448.4 has 14.3 for Database Latency. 2/8/2012 3:50:11 PM -- Instance2448.4 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Instance2448.4 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Instance2448.5 has 14.2 for Database Latency. 2/8/2012 3:50:11 PM -- Instance2448.5 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Instance2448.5 has 2.2 for Log Latency. 2/8/2012 3:50:11 PM -- Test has 0 Maximum Database Page Fault Stalls/sec. 2/8/2012 3:50:11 PM -- The test has 0 Database Page Fault Stalls/sec samples higher than 0. N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 21
2/8/2012 3:50:11 PM -- C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - Soft Recovery\Performance_2012_2_8_15_13_33.xml has 144 samples queried. 2/8/2012 3:50:11 PM -- C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - Soft Recovery\Performance_2012_2_8_15_13_33.html was saved. 2/8/2012 7:29:34 PM -- Performance logging started (interval: 40 ms). 2/8/2012 7:29:34 PM -- Recovering databases... 2/8/2012 7:39:07 PM -- Performance logging has ended. 2/8/2012 7:39:07 PM -- Instance2448.1 (572.396198), Instance2448.2 (553.8321656), Instance2448.3 (571.0701957), Instance2448.4 (570.8049952) and Instance2448.5 (559.4013753) 2/8/2012 7:39:08 PM -- C:\Users\Administrator\Documents\Dual Server 1,.15, 1024, 10, Jumbo, No MPIO - Soft Recovery\SoftRecovery_2012_2_8_19_29_29.blg has 141 samples. 2/8/2012 7:39:08 PM -- Creating test report... N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 22
Nimble Storage, Inc. 2740 Zanker Rd., San Jose, CA 95134 Tel: 877-3NIMBLE (877-364-6253) www.nimblestorage.com info@nimblestorage.com 2012 Nimble Storage N I M B L E S T O R A G E : E X C H A N G E 2 0 1 0 S T O R A G E S O L U T I O N 23