HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution

Transcription

1 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution Using RAIDless JBOD disks Table of contents Overview... 2 Disclaimer... 2 Features of the tested solution... 2 HP D6000 Disk Enclosure... 2 Key features and benefits... 3 HP ProLiant BL460c Gen8 Server Blade... 3 HP Smart Array P721m Controller... 4 Solution description... 5 Targeted customer profile... 5 Storage configuration... 7 Mailbox configuration primary site operational... 9 Mailbox configuration primary site unavailable (planned or unplanned) Mailbox configuration primary site offline and single server failure at disaster recovery site (planned or unplanned) Tested deployment Best practices Test results summary Reliability Storage performance results backup / recovery performance Conclusion Appendix A stress testing Appendix B performance testing Appendix C streaming backup testing Appendix D soft recovery testing For more information... 60

2 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution Overview This document provides information on an HP D6000 Disk Enclosure and HP ProLiant BL460c Gen8 Server Blades solution for Microsoft Exchange Server 2013 (Exchange), based on the Microsoft Exchange Solution Reviewed Program (ESRP) Storage program. For any questions or comments regarding the contents of this document, see the For more information section. The ESRP Storage program was developed by Microsoft Corporation to provide a common storage testing framework for vendors to provide information on its storage solutions for Microsoft Exchange Server software. For more details on the Microsoft ESRP Storage program, see: Tested with: ESRP Storage Version 4.0 Test Date: October 2013 Disclaimer HP produced this document independently of Microsoft Corporation. Microsoft Corporation expressly disclaims responsibility for, and makes no warranty, express or implied, with respect to the accuracy of the contents of this document. Features of the tested solution This solution implements the Availability Group (DAG) mailbox resiliency feature found in Microsoft Exchange Server 2013 (Exchange). This feature is the base component of the high availability and site resilience framework built into Exchange. A DAG is a group of up to 16 mailbox servers that hosts a set of databases and provides database-level recovery from failures that affect individual servers or databases. This solution brief describes a tested configuration for deploying Microsoft Exchange Server 2013 (Exchange) in a 4,000- user, mailbox-resilient environment. The performance results and best practices outlined in this document provide tested guidelines for configuring Exchange with HP D6000 disk enclosure, HP ProLiant BL460c Gen8 Server Blades, and HP Smart Array P721m controllers in a direct-attached storage (DAS) solution. This solution used a single DAG to support 4,000 mailboxes, each with a 4 GB mailbox capacity and a work profile of 200 messages sent and received for each user, each day. Four copies of each database provide mailbox resiliency, with two copies located at the primary site and two copies located at the disaster recovery site. The primary site hosts all 4,000 users during normal operations. The performance results outlined in this paper are from the testing of a single building block with a single server failed and two operational servers at the disaster recovery site. See the section Test results summary for more information. HP D6000 Disk Enclosure Overview With the 6 Gb/s SAS HP D6000 Disk Enclosure, HP extends and redefines direct-attached storage for HP ProLiant and BladeSystem servers, combining the simplicity and cost effectiveness of direct-attached storage without sacrificing flexibility or performance. Use the D6000 as part of a straightforward in-rack 6 Gb/s SAS implementation that delivers high-density, low-cost, external-zoned, direct-attach storage for HP ProLiant and BladeSystem servers. The D6000 provides a cost-effective, high capacity storage solution to configure storage on the fly with easy-to-use management software. Up to six D6000 disk enclosures are supported on a single BladeSystem enclosure for up to 1260 TB of capacity. End-toend 6 Gb/s SAS connectivity enables a high performance storage solution using an HP Smart Array P721m controller, 6 Gb/s SAS BL switches, and the D Gb/s host connectivity enables higher performance, eases configuration/deployment, and broadens supported features of a D6000 using an HP Smart Array P721m controller and 6 Gb/s SAS BL switches. For more information on blade external SAS storage solutions for HP BladeSystem, see: hp.com/go/directconnect. 2

3 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution Figure 1: Front view of the HP D6000 Dual Module Disk Enclosure Key features and benefits High-density, Low-cost, External Serial-Attached SCSI Storage for BladeSystem servers Designed with two pull-out disk drawers to support hot-plug, large form factor SAS or SAS MDL drives in just 5U of rack space, the 6 Gb/s SAS D6000 is meant for customers requiring a dense storage option with a low acquisition cost and low cost per U. Eliminates the expense and complexity from the storage architecture, with the simplicity of an in-rack storage solution using an HP Smart Array P721m controller in a server blade. 6 Gb/s SAS BL switches in the blade enclosure connected to up to six D6000 enclosures, provides 6 Gb/s SAS end-to-end connectivity. Supports up to six D6000 from a single BladeSystem enclosure, providing up to 1260 TB of in-rack storage for customers requiring high-density, low-cost storage. Configure Local Storage on the fly for BladeSystem Servers Use management software of the 6 Gb/s SAS BL switch to group D6000 drives and assign them to individual server blades. These drives will appear as local storage to that server. Build local storage on the fly by zoning direct attach with management software. Add additional drives to the D6000 and zone to a newly deployed server blade, or add capacity to the zone of an existing server blade configuration. No need for complex configuration or rewiring. High performance 6 Gb/s SAS connectivity to external storage 6 Gb/s host connectivity enables higher performance, eases configuration/deployment and broadens supported features of a D6000 using HP Smart Array P721m controller and 6 Gb/s SAS BL switches. Install application servers and storage in the same rack, reducing latency because data does not travel over long distances. Provides up to 2GB Flash-Backed Write Cache (FBWC) with Smart Array P721m Controller installed in the server blade. For more information on the D6000 Disk Enclosure, visit hp.com/go/d6000. HP ProLiant BL460c Gen8 Server Blade The HP ProLiant BL460c Gen8 Server Blade is a dual-socket server blade, engineered for unprecedented performance, enhanced flexibility, and simplified management, making it the standard for data center computing. The ProLiant BL460c Gen8 Server packs in more performance with a 33 percent increase in memory DIMM count, Intel Xeon E Processor with added support for 130W processors, faster slots, and enhanced Smart Array Controller that ships with 512 MB Flash-Backed Write Cache (FBWC) standard. In addition, it is more flexible with HP FlexibleLOM, which provides the ability to customize server networking today and to meet future needs without overhauling server hardware compared to previous generation. Key features of the HP ProLiant BL460c Gen8 Server Blade Dynamic workload acceleration delivers smarter data protection for ever-increasing drive capacities, real-time, workloadaware analysis to optimize and adapt storage performance, and 6x faster solid-state performance vs. previous generations. HP Active Health System provides 24x7 continuous health monitoring, 100 percent configuration change logging, and faster problem analysis with HP Service and Support. Start problem analysis 5x faster using unified diagnostic tools with secure data transfer to HP Service and Support. 3

4 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution HP SmartMemory has unmatched performance, quality, and manageability advantages above industry-standard memory. It supports Unbuffered DIMM (UDIMM), Registered DIMM (RDIMM), and Load Reduced DIMM (LRDIMM). SmartMemory in the ProLiant BL460c Gen8 Server Blade delivers up to 33 percent more memory capacity over previous generations. The embedded HP Smart Array P220i RAID controller has 512 MB Flash-Backed Write Cache (FBWC) standard, with support for HP Smart Storage and HP SmartDrives, and a choice of two 2.5-inch Small Form Factor (SFF) Serial Attached SCSI (SAS), Serial ATA (SATA), and SSD drives. Two x16 PCIe 3.0 expansion slots support the highest-performing mezzanine option cards now and into the future. HP FlexibleLOM provides the ability to customize server networking today and to meet future needs without overhauling server hardware. It provides a choice of bandwidth and fabric (Ethernet, FCoE, InfiniBand), supports Wake on LAN (WOL), and a shared, Integrated Lights-Out (ilo) port for ease of use. Intel Xeon E Processor family offers high performance, power efficiency, and more adaptability with Intel QuickPath, Integrated Memory Controller, Turbo Boost, and Intelligent Power Technologies. For more information on HP BladeSystem, visit: hp.com/go/blades. Figure 2: HP ProLiant BL460c Gen8 Server Blade and HP BladeSystem c7000 Enclosure HP Smart Array P721m Controller The HP Smart Array P721m Controller is a PCI-Express (PCIe) 3.0 mezzanine card, supporting direct-attached SAS Storage. It is one of the highest performing controllers in the HP 6Gb/s SAS portfolio. It provides new levels of reliability, connectivity, and storage performance for HP ProLiant Gen8 server blades through its support of the latest SAS technology and advanced RAID capabilities. HP Smart Array Advanced Pack (SAAP) 2.0 is included. Key features include: Storage interface (SAS/SATA) Eight 6Gb/s SAS physical links distributed equally across 4 external 2x ports to support up to 4 6Gb/s SAS Switches 6 Gb/s SAS technology delivers up to 600 MB/s theoretical bandwidth per physical link. 6 Gb/s SATA technology delivers up to 600 MB/s theoretical bandwidth per physical link for directly attached SATA drives. Mix-and-match SAS and SATA drives to the same controller. Deploy drive technology as needed to fit the computing environment. Support for SAS tape drives, SAS tape autoloaders, and SAS tape libraries. RAID controller features 512 MB or 2 GB cache RAID 0, 1, 10, 5, 50, 6, 60, 1 ADM, 10 ADM NOTE: RAID for Shared Storage is controlled by the controller of the array. Recovery ROM protects against ROM corruption. Smart Array PCIe mezzanine card Connects to a HP 6Gb/s SAS BL Switch through the c-class enclosure high-speed mid-plane supporting shared SAS storage. 4

5 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution 2 GB Flash-Backed Write Cache (FBWC) Provides read-ahead caching and write back caching with indefinite write cache data retention in the case of unexpected power outage. Software consistency on most current shipping Smart Array family products including: HP SmartStorage Administrator, HP Systems Insight Manager, and HP Intelligent Provisioning. The HP Smart Storage Administrator configures and manages a host of Smart Array controllers and other storage devices using a single interface. It replaces the HP Array Configuration Utility (ACU) with an updated design and configuration enhancements. HP SSD Smart Path feature Included in the Smart Array software stack, this feature improves SSD read performance. By bypassing the Smart Array firmware, it finds the optimal performance path to the SSD. HP SmartCache This licensed feature is a controller-based, read-caching solution in a DAS environment that caches the most frequently accessed data ( hot data) onto lower latency SSDs to dynamically accelerate application workloads. HP Smart Cache license is not required on P721m with 2 GB FBWC. HP Smart Cache requires a minimum of 1 GB FBWC. For more information on HP StoreFabric Storage Networking, visit: hp.com/go/storefabric Figure 3: HP Smart Array P721m Controller Solution description This solution brief provides best practices to help administrators implement an Exchange deployment with the following requirements. Targeted customer profile Availability Group (DAG) deployment with three mailbox servers in the primary site and three mailbox servers in the disaster recovery site, for a total of six mailbox servers across the entire solution. 4,000 mailbox users (1334 mailbox users per server) Exchange 2013 profile of 0.16 IOPS workload per user (200 messages sent and received per user, per day, with an additional 20% headroom added to base IOPS). 4GB mailbox size. Four copies of each mailbox database (one active / three passive). 32 databases per server (16 active and 16 passive) at the primary site and 96 passive database copies at the disaster recovery site. 27-3TB 7.2K SAS disks per Disk Enclosure for the Exchange databases, transaction logs and one extra disk per server as a spare. Designed for server and site resiliency. 5

6 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution Background Maintenance (BDM) enabled in testing. Storage deployed as RAID 0. See the Storage configuration section for more information. Note HP has developed the HP Sizer for Microsoft Exchange Server 2013 to assist customers with proper server and storage sizing for their Exchange Server 2013 deployments. The algorithms developed and implemented in this tool originate from extensive testing of Exchange Server 2013 server roles and technologies, HP s deep experience with Microsoft Exchange, and intricate knowledge of server CPU, memory, and subsystems. Designed to simplify configuration across a broad range of deployments, this tool is free to download and use. For more information, visit: hp.com/solutions/activeanswers/sizers. In addition to the HP Sizer, HP recommends that customers use the Microsoft Exchange Mailbox Server Role Requirements Calculator. If this Calculator is used as a starting point, it can be imported into the HP Sizer using the Load Exchange Calculator button. Figure 4 shows the database layout and failover from Exchange 2013 Mailbox Server Role Requirements Calculator. The diagram shows one server in the Primary site as offline, with the databases active on the two remaining servers. This illustrates that database failover planning is designed to support a maximum of 24 databases active on any single server. Figure 4: Details of database layout and failover from Exchange 2013 Mailbox Server Role Requirements Calculator 6

7 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution Storage configuration This solution uses the D6000 Disk Enclosure, paired with the Smart Array P721m controller configured as dual-domain SAS for storage performance and redundancy. This configuration uses independent SAS data paths from the servers to the storage. The dual-domain communication path prevents interruption of disk in the event of a data path failure. Create a single SAS Array X per physical drive, then create a single Logical Drive from all free space in the SAS Array using a 256 KB or higher strip size and RAID 0. Figure 5 shows the logical and physical view of the storage from the HP Array Configuration Utility (ACU). Multiple databases per volume is a new RAIDless JBOD scenario available in Exchange 2013 that allows for active and passive copies (including lagged copies) to be mixed on a single disk, enabling better disk utilization. See Microsoft s guidelines for JBOD considerations for multiple databases per volume at Figure 5: Array Configuration Utility (ACU) interface The recommended best practice for site and server resiliency and load balancing is three server blades with eight 3TB 7.2K LFF SAS Drives assigned to each server, in both the primary and disaster recovery sites. HP also recommends designating one extra disk per server as a spare in the case of disk failure. Refer to Microsoft Exchange 2013 Administrator Guide at to configuring spare volume for automatic reseed. This highly available load balanced configuration runs 16 active and 16 passive databases per server on the primary site during normal operations. Figure 6 illustrates the hardware used in this solution at the primary site. For more details on implementing a dual-domain SAS configuration with the D6000, see 7

8 Bay 1 Bay 9 DS Module PS 1 UID HP ProLiant BL460c Gen8 UID HP ProLiant BL460c Gen8 UID HP ProLiant BL460c Gen8 UID OK UID HP StorageWorks MDS600 HP BladeSystem c7000 Enclosure Bay 8 Bay 16 PS 6 UID UID X1 UID 2 UID 1 HP VC Flex-10 Enet Module X1 X2 X3 X4 X5 X6 HP 3GB SAS Switch Ensure HDD drawer is fully seated before removing this module. OUT IN Ensure HDD drawer is fully seated before removing this module. OUT OA/iLO SHARED Reset HP 6GB SAS Switch UID Active IN SHARED: UPLINK or X-LINK Enclosure UID Remove management modules before ejecting sleeve X7 X8 UID UID X1 Enclosure Interlink HP VC Flex-10 Enet Module X1 X2 X3 X4 X5 X6 SHARED HP 3GB SAS Switch HP 6GB SAS Switch Ensure HDD drawer is fully seated before removing this module. OUT IN Ensure HDD drawer is fully seated before removing this module. OUT IN SHARED: UPLINK or X-LINK X7 X8 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution Figure 6: Rack mounted view of the D6000 Disk Enclosure and the HP ProLiant BL460c Server Blades, cabled in a dual-domain configuration at the primary site. CAUTION CAUTION 1 CAUTION CAUTION 7.2K SAS 500 GB 7.2K SAS 500 GB 7.2K SAS 500 GB 3 7.2K SAS 500 GB 7.2K SAS 500 GB 7.2K SAS 500 GB 4 FAN FAN OA1 OA2 FAN 6 FAN 10 PS 6 PS 5 PS 4 PS 3 PS 2 PS 1 1. HP D6000 Drive Enclosure with (27) 3TB 7.2K LFF SAS Drives 2. HP BladeSystem c7000 Enclosure 3. HP ProLiant BL460c Gen8 Server Blades with P721m controller in each system 4. Primary path 5. Secondary path 6. HP 6Gb/s SAS BL Switches 8

9 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution Mailbox configuration primary site operational Figure 7 shows the mailbox layout for active and passive database copies when the primary site is operational. This solution brief provides best practices to help administrators implement the Exchange deployment, with the following requirements: 4,000 mailbox users IOPS workload per user (200 messages sent and received per user, per day, with an additional 20% headroom). 4GB mailbox size. Each mailbox server in the primary site hosts 32 database copies (16 active and 16 passive). Each mailbox server in the disaster recovery site hosts 32 database copies (32 passive). Background Maintenance (BDM) enabled. Figure 7: Normal operation with all mailboxes hosted at the primary site. Active Passive Primary Site Disaster Recovery Site MBX1 MBX2 MBX3 MBX4 MBX5 MBX6 DAG DB1 DB1 DB2 DB1 DB1 DB2 DB3 DB2 DB3 DB3 DB2 DB3 DB4 DB5 DB4 DB4 DB5 DB4 DB5 DB6 DB6 DB5 DB6 DB6 DB47 DB48 DB48 DB47 DB48 DB48 9

10 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution Mailbox configuration primary site unavailable (planned or unplanned) Figure 8 shows the mailbox layout for active and passive database copies when the primary site is unavailable due to a planned or unplanned event. This solution brief provides best practices to help administrators implement the Exchange deployment with the following requirements: 4,000 mailbox users IOPS workload per user (200 messages sent and received per user, per day, with an additional 20% headroom). 4GB mailbox size. Primary site s database copies are unavailable due to a planned or unplanned event. Each mailbox server in the disaster recovery site hosts 32 database copies (16 active and 16 passive). Background Maintenance (BDM) enabled. Figure 8: Primary site unavailable with all mailboxes hosted on disaster recovery site. Active Passive Primary Site Disaster Recovery Site DAG XMBX1 XMBX2 XMBX3 MBX4 MBX5 MBX6 DB1 DB1 DB2 DB1 DB1 DB2 DB3 DB2 DB3 DB3 DB2 DB3 DB4 DB5 DB4 DB4 DB5 DB4 DB5 DB6 DB6 DB5 DB6 DB6 DB47 DB48 DB48 DB47 DB48 DB48 10

11 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution Mailbox configuration primary site offline and single server failure at disaster recovery site (planned or unplanned) Figure 9 shows the tested mailbox layout for active database copies when the primary site and one of the servers in the disaster recovery site are unavailable due to a planned or unplanned event. This solution demonstrates the worst-case scenario, with each host in the disaster recovery site hosting twenty-four active databases, with the following requirements: 4,000 mailbox users IOPS workload per user (200 messages sent and received per user, per day, with an additional 20% headroom). 4 GB mailbox size. Primary site s database copies and one of the three servers at the recovery site are unavailable due to a planned or unplanned event. One server in the disaster recovery site is unavailable. Each mailbox server in the disaster recovery site hosting 32 database copies (24 active and 8 passive). Background Maintenance (BDM) enabled. Figure 9: Primary site unavailable and single server failure at disaster recovery site. Active Passive Primary Site Disaster Recovery Site DAG XMBX1 XMBX2 XMBX3 XMBX4 MBX5 MBX6 DB1 DB1 DB2 DB1 DB1 DB2 DB3 DB2 DB3 DB3 DB2 DB3 DB4 DB5 DB4 DB4 DB5 DB4 DB5 DB6 DB6 DB5 DB6 DB6 DB47 DB48 DB48 DB47 DB48 DB48 11

12 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution Tested deployment The following tables summarize the tested environment. Simulated Exchange configuration Number of Exchange mailboxes simulated 4,000 (4GB mailbox size) Number of Availability Groups (DAGs) 1 Number of active mailbox servers per DAG 2 Number of active mailboxes per server 2,000 Number of databases per server 24 Number of copies per database 4 (2 primary / 2 secondary) Number of active mailboxes per database 84 Simulated profile: operations per second per mailbox (IOPS, include 20% headroom) and Log LUN size Total database size for performance testing Background database maintenance (BDM) GB 15TB Tested with 24/7 BDM enabled % formatted storage capacity used by Exchange database 27% 15TB / 54.6TB (Total database size /Total formatted capacity) Storage hardware Storage Connectivity Storage model and firmware version Number of storage controllers per server 6 Gb/s SAS HP D6000 Dual Module Disk Enclosure Firmware Version: with 4 external SAS ports Number of SAS switches 2 Number of SAS ports on storage 8 Maximum bandwidth of storage connectivity to host Storage Controller and Firmware Version Controller Cache Host server type SAS Switch Model and Firmware Version 48 Gb/s (8 ports x 6 Gb/s) HP Smart Array P721m Firmware Version: GB Flash-Backed Write Cache (FBWC), (50/50 read/write) HP ProLiant BL460c Gen8 HP 6Gb SAS for HP BladeSystem c-class Firmware version: Total number of disks tested in solution 20* Maximum number of disks that can be hosted in the tested configuration 70 * Only 20 disks needed to host 48 active databases in test. 12

13 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution Storage software Smart Array P721m Driver HBA Queue Depth & target Setting Multipathing Host OS Not applicable Not applicable Microsoft Windows Server 2012 Datacenter ESE.dll file version Storage disk configuration (mailbox and log disks) Disk type, speed, and firmware version Raw capacity for each disk (TB) HP 3TB 6Gb SAS 7.2k RPM LFF 3.5 MDL Firmware Version: HPD4 3TB Number of physical disks in test 20 Total raw storage capacity (TB) RAID level & Set Size 60TB** RAIDless JBOD Step Size (KiB) 512 Total formatted capacity (TB) 54.6TB Storage capacity utilization 91% capacity utilization 25% 54.6TB/60TB (Formatted capacity/ Total Raw capacity) 15TB/60TB (Total database size / Total Raw capacity) ** Capacity of drives, and therefore the raw data capacity of the disk array, based on 1KB = 1000 bytes, not 1024 bytes. This means that 1 GB = 1,000,000,000 bytes, not (1,073,741,824 bytes) Replication configuration Replication Mechanism Number of links Simulated link distance Link type Link bandwidth Mailbox Resiliency Two (2) per server (base configuration) LAN IP Ethernet 1Gbps (base configuration) Best practices Exchange Server is a disk-intensive application. For Exchange 2013 best practices on storage design from Microsoft, see For Exchange Server solution guidance, visit HP at hp.com/solutions/microsoft/exchange. Based on the testing run using the ESRP framework, we recommend the following to improve the storage performance. Drivers and Firmware Ensure that firmware and drivers for all hardware in the solution are up to date. For more information, see: hp.com/go/support. 13

14 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution Drive Error Troubleshooting (media errors) A Microsoft 2013 Exchange deployment using RAID 0 volumes may see media error events in the Exchange servers Application and System event logs. Exchange 2013 will attempt the repair utilizing page patching while the drive and/or controller can designate the block as failed and redirect requests to an available spare block. HP strongly advises following Microsoft s recommendation to only replace a hard drive if it enters the fully-failed state (or is in predictive failure). A drive in this state will be marked failed in the available event logs and typically displays an amber or red LED on the drive. Physical Drive Write Cache Unlike controller-level cache, individual drive write caches are not protected. If the server loses power, or a hot plug drive is physically removed from the array during operations, unwritten data in the physical drives write caches will be lost. HP recommends disabling Physical Drive Write Cache on the HP Smart Array P721m Controller. Accelerator Ratio and Flash-Backed Write Cache (FBWC) Configure the controller cache to 50/50 read/write (instead of default 10/90 read/write.) Jetstress 2013 Before installing Exchange Server 2013, test your storage solution with Jetstress to ensure it is configured properly and meets your requirements. For more information on Jetstress, see: microsoft.com/en-us/download/details.aspx?id= Mailbox Copies When deploying a JBOD configuration, ensure that there are at least three copies for each Exchange mailbox database in the solution. Fault tolerance is achieved in this solution by having two copies at the primary site and two database copies at the disaster recovery site, ensuring business continuity through superior disaster recovery and Exchange availability options. Event Notification Use the HP ProLiant Smart Array SAS/SATA Event Notification Service so that storage events such as physical disk, array and logical disk status events are available in the Windows System Event Log. Visit hp.com/go/support for the latest support software. Strip Size For Exchange deployments, HP recommends using the largest strip size for your selected RAID level. The strip size was configured for 512 KB. For more information, see: RAID Level volumes used in this solution were configured as RAID 0. The largest set size possible is recommended for Exchange deployments with 7.2K disks. For more information, see: Windows Volumes For best performance, format the Windows volumes with 64KB allocation unit size. For more information on Exchange 2013 Storage Configuration Options and Best Practices recommendations, see: Test results summary This section provides a high-level summary of the test data from the ESRP and a reference to the detailed HTML reports, generated by the ESRP testing framework. For solutions that use a uniform building block style, in which each building block is independent and isolated, ESRP requires testing only one building block. For this solution, a building block consisted of three HP ProLiant BL460c Gen8 servers and one HP D6000 Disk Enclosure, with 24-3TB LFF SAS drives hosting all 4,000 mailboxes. HP recommends configuring three additional 3TB LFF SAS drives as spares for autoreseed. See Microsoft Exchange 2013 Administrator Guide at for instructions on Configuring AutoReseed for a Availability Group. 14

15 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution To prove that the solution can handle site and server resiliency, we tested with two HP ProLiant BL460c Gen8 servers and one HP D6000 Disk Enclosure, with 20 3TB LFF SAS drives hosting all 4,000 mailboxes distributed amongst 48 active databases and the tests continue to pass. However, for actual deployments of this solution, see the Storage configuration section for complete hardware requirements. For a complete solution, contact your HP Reseller or HP Sales Representative. Reliability Reliability testing exercises the storage with the maximum sustainable Exchange load for 24 hours. The goal is to verify that the storage can handle the maximum load for an extended amount of time. Both log and database files are analyzed for integrity after the stress test to confirm that no database or log corruption occurred. After running the 24-hour test, we found no relevant errors reported in the event log for the storage reliability testing, and no errors reported for the checksum process. Storage performance results Storage performance testing exercises the storage with the maximum sustainable Exchange load for two hours. The test measures storage response times under the maximum sustainable Exchange load. The following data is the sum of all logical disk s and an average of all logical disk latency in the two-hour test duration. This solution is designed around an Exchange deployment with a single DAG. The DAG has three mailbox servers in the primary site and three mailbox servers in the disaster recovery site. Ideally, the three servers in the disaster recovery site should host all active mailboxes when the primary site is offline (planned or unplanned.) Individual server metrics Tables below show the sum of across databases and the average latency across all databases on a per server basis. Server 1 Needed Disk Transfers/sec 320 Disk Transfers/sec 586 Disk /sec 377 Disk /sec 210 Disk Read (ms) 12 Disk Write (ms) 1.4 Transaction Log Log Disk /sec 150 Log Disk Write (ms) <1 ms Server 2 Needed Disk Transfers/sec 320 Disk Transfers/sec 587 Disk /sec 377 Disk /sec 211 Disk Read (ms) 12 Disk Write (ms)

16 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution Transaction Log Log Disk /sec 151 Log Disk Write (ms) <1 ms Aggregate performance across all servers The table below shows the sum of and the average latency across all servers in the solution. Needed Disk Transfers/sec* 640 Disk Transfers/sec 1173 Disk /sec 754 Disk /sec 421 Disk Read (ms) 12 Disk Write (ms) 1.5 Transaction Log Log Disk /sec 301 Log Disk Write (ms) <1 ms * This row represents the total IOPS necessary to satisfy the IOPS for all mailboxes in the solution backup / recovery performance This section contains two test reports. The first report measures the sequential read rate of the database files; the second report measures the recovery/replay performance (playing transaction logs into the database). The tests measure the read rates and log replay rates with all mailboxes under concurrent or simultaneous load. Read-Only Performance This test measures the maximum backup rate for Volume Shadow Copy Service (VSS) aware backup applications. The following table shows the average and sum of the database reads. MB read/sec per database 84 MB read/sec total 2022 Transaction Log Recovery / Replay Performance This test measures the maximum rate that log files can play against the databases. The following table shows the average rate for log files played. Each log file is 1 MB in size. number of log files played 425 time to play one log file (sec) 6 16

17 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution Conclusion The information discussed in this solution brief highlights the tested performance results and configuration best practices for an Exchange solution supporting 4,000 4GB mailboxes and a 0.16 IOPS workload for each user (200 messages sent and received per user, per day with additional 20% headroom). The tested solution included an HP D6000 Disk Enclosure with 20 3TB 7.2K LFF SAS drives for the mailbox database storage and transaction logs. Testing demonstrated that the storage subsystem supports the IOPS needed for 4,000 Exchange users and provides additional performance headroom. The test results and data presented in this document are from the ESRP test framework. Test results are only a guide; customers should test their specific solution to make sure it meets their requirements. The ESRP program is not a benchmarking program. The tests do not measure the maximum throughput for a given solution. Rather, program focus is on producing recommendations from vendors for Exchange Server storage solutions, not direct comparisons among solutions. Appendix A stress testing Microsoft Exchange Jetstress 2013 Stress Test Result Report Test Summary Overall Test Result Pass Machine Name Server 1 Test Description Test Start Time Test End Time 10/16/2013 9:19:15 AM 10/17/ :04:09 AM Collection Start Time 10/16/2013 9:32:56 AM Collection End Time 10/17/2013 9:32:50 AM Jetstress Version ESE Version Operating System Windows Server 2012 Datacenter ( ) Performance Log C:\Program Files\Exchange Jetstress\24_hr_test\Stress_2013_10_16_9_20_5.blg Sizing and Throughput Achieved Transactional per Second Target Transactional per Second 320 Initial Size (bytes) Final Size (bytes) Files (Count) 24 Jetstress System Parameters Thread Count 15 Minimum Cache Maximum Cache Insert Operations 40% Delete Operations 20% Replace Operations 5% Read Operations 35% Lazy Commits 70% Run Background Maintenance True Number of Copies per MB MB Configuration Log path: C:\dblg1 Instance : C:\dblg1\Jetstress edb Instance Log path: C:\dblg2 : C:\dblg2\Jetstress edb Instance Instance Log path: C:\dblg3 : C:\dblg3\Jetstress edb Log path: C:\dblg4 17

18 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution : C:\dblg4\Jetstress edb Instance Log path: C:\dblg5 : C:\dblg5\Jetstress edb Instance Log path: C:\dblg6 : C:\dblg6\Jetstress edb Instance Log path: C:\dblg7 : C:\dblg7\Jetstress edb Instance Log path: C:\dblg8 : C:\dblg8\Jetstress edb Instance Log path: C:\dblg9 : C:\dblg9\Jetstress edb Instance Log path: C:\dblg10 : C:\dblg10\Jetstress edb Instance Log path: C:\dblg11 : C:\dblg11\Jetstress edb Instance Log path: C:\dblg12 : C:\dblg12\Jetstress edb Instance Log path: C:\dblg13 : C:\dblg13\Jetstress edb Instance Log path: C:\dblg14 : C:\dblg14\Jetstress edb Instance Log path: C:\dblg15 : C:\dblg15\Jetstress edb Instance Log path: C:\dblg16 : C:\dblg16\Jetstress edb Instance Log path: C:\dblg17 : C:\dblg17\Jetstress edb Instance Log path: C:\dblg18 : C:\dblg18\Jetstress edb Instance Log path: C:\dblg19 : C:\dblg19\Jetstress edb Instance Log path: C:\dblg20 : C:\dblg20\Jetstress edb Instance Log path: C:\dblg21 : C:\dblg21\Jetstress edb Instance Log path: C:\dblg22 : C:\dblg22\Jetstress edb Instance Log path: C:\dblg23 : C:\dblg23\Jetstress edb Instance Log path: C:\dblg24 : C:\dblg24\Jetstress edb 18

19 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution Transactional Performance MSExchange ==> Instances /sec /sec Log Log Log /sec Log /sec Log Log Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Background Maintenance Performance MSExchange ==> Instances Maintenance IO /sec Maintenance IO Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance

20 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution Log Replication Performance MSExchange ==> Instances Log /sec Log Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Total Performance MSExchange ==> Instances /sec /sec Log Log Log Log Log /sec /sec Log Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance Instance

21 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 storage solution Host System Performance Counter Minimum Maximum % Processor Time Available M Free System Page Table Entries Transition Pages RePurposed/sec Pool Nonpaged Pool Paged Page Fault Stalls/sec Test Log 10/16/2013 9:19:15 AM -- Preparing for testing... 10/16/2013 9:19:40 AM -- Attaching databases... 10/16/2013 9:19:40 AM -- Preparations for testing are complete. 10/16/2013 9:19:40 AM -- Starting transaction dispatch.. 10/16/2013 9:19:40 AM -- cache settings: (minimum: MB, maximum: 6.0 GB) 10/16/2013 9:19:40 AM -- flush thresholds: (start: 61.4 MB, stop: MB) 10/16/2013 9:20:05 AM -- read latency thresholds: (average: 20 msec/read, maximum: 200 msec/read). 10/16/2013 9:20:05 AM -- Log write latency thresholds: (average: 10 msec/write, maximum: 200 msec/write). 10/16/2013 9:20:09 AM -- Operation mix: Sessions 15, Inserts 40%, Deletes 20%, Replaces 5%, 35%, Lazy Commits 70%. 10/16/2013 9:20:09 AM -- Performance logging started (interval: ms). 10/16/2013 9:20:09 AM -- Attaining prerequisites: 10/16/2013 9:32:56 AM -- \MSExchange (JetstressWin)\ Cache Size, Last: (lower bound: , upper bound: none) 10/17/2013 9:32:56 AM -- Performance logging has ended. 10/17/ :03:39 AM -- JetInterop batch transaction stats: 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60836, 60835, 60835, 60835, 60835, and /17/ :03:39 AM -- Dispatching transactions ends. 10/17/ :03:39 AM -- Shutting down databases... 10/17/ :04:09 AM -- Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete), Instance (complete) and Instance (complete) 10/17/ :04:09 AM -- C:\Program Files\Exchange Jetstress\24_hr_test\Stress_2013_10_16_9_20_5.blg has 5756 samples. 10/17/ :04:09 AM -- Creating test report... 10/17/ :07:26 AM -- Instance has 16.1 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 14.2 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 16.1 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 15.9 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 14.2 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 16.1 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 16.3 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 14.4 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 16.4 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 21

22 Technical white paper HP D6000 Disk Enclosure 4,000 Mailbox Resiliency Exchange 2013 Storage Solution 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 15.9 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 14.2 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 16.1 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.1 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.2 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.2 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.3 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.3 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.5 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.5 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.6 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.2 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.4 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.2 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 9.4 for. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Instance has 0.2 for Log. 10/17/ :07:26 AM -- Test has 0 Maximum Page Fault Stalls/sec. 10/17/ :07:26 AM -- The test has 0 Page Fault Stalls/sec samples higher than 0. 10/17/ :07:26 AM -- C:\Program Files\Exchange Jetstress\24_hr_test\Stress_2013_10_16_9_20_5.xml has 5705 samples queried. Microsoft Exchange Jetstress 2013 Stress Test Result Report Test Summary Overall Test Result Pass Machine Name Server 2 Test Description 22