HUAWEI OceanStor S5500T Exchange Server 2010 Solution with Users

Transcription

1 HUAWEI OceanStor S5500T Exchange Server 2010 Solution with Users

2 Contents Features 2 High performance and scalability 2 High reliability and availability 3 Efficient, easy, and flexible 3 Energy saving 3 Solution description 4 Storage configurations 5 Targeted customer profile 5 Tested deployment 6 Best practices 7 Mailbox servers 8 Core storage 9 Test results summary 14 Reliability 14 Storage performance 14 Individual server metrics 14 Performance across servers 15 Database backup and recovery performance 15 Database read-only performance 15 Transaction log recovery/replay performance 16 Detailed test results 16

3 Figures Figure 1-1 OceanStor S5000T Series Storage System 2 Figure 1-2 Solution architecture 4 Figure 1-3 Storage Configuration 5 Figure 1-4 Recommended SAN network for Exchange Server 9 Tables Table 1-1 Simulated Exchange configuration 6 Table 1-2 Storage hardware 6 Table 1-3 Storage software 6 Table 1-4 Storage disk configuration(mailbox store disks) 7 Table 1-5 Storage disk configuration(transactional log disks) 7 Table 1-6 Replication Configuration 7 Table 1-7 Database cache and estimated IOPS per mailbox under different loads 10 Table 1-8 Transitional IOPS of different types of disks 11 Table 1-9 Number of generated transaction logs under different loads 13 Table 1-10 Individual server metrics 15 Table 1-11 Performance across servers (Database) 15 Table 1-12 Performance across servers (Log) 15 Table 1-13 Database read-only performance 16 Table 1-14 Transaction log recovery/replay performance 16

4 Overview This document described HUAWEI OceanStor S5500T Exchange 2010 Solution with users and 1.5GB mailbox for each user, based on Microsoft Exchange Solution Reviewed Program (ESRP) storage program guidelines. Disclaimer This document has been produced independency with Microsoft Corporation. Microsoft Corporation expressly disclaims responsibly for, and makes no warranty-express or implied-with respect to the accuracy of the contents of this document. The information contained in this document represents the current view of HUAWEI on the issues discussed as of the date of publication. Due to changing market conditions, it should not be interpreted as a commitment on the part of HUAWEI. In addition, HUAWEI cannot guarantee the accuracy of any information presented after the date of publication. PROVIDED BY HUAWEI SOLUTION PROVEN CENTER 1

5 Features Figure 1-1 OceanStor S5000T Series Storage System Huawei OceanStor S5000T series (T series for short) is a new generation of mid-range storage systems. Based on its industry-leading hardware specifications, the T series integrates the advanced technologies of high-density disk design, TurboModule (high-density and hotswappable I/O modules), and TurboBoost (three-level performance boost), and multi-level data protection, satisfying the application requirements of large OLTP/OLAP databases, highperformance computing, digital media, Internet operation, centralized storage, backup, disaster recovery, and data migration. High performance and scalability Industry-leading hardware Equipped with 64-bit multi-core processors, high-speed and large-capacity cache, and, high internal bandwidth, improving overall performance by 50% Diversified disk types FC, SAS, NL SAS, SATA, and SSD, allowing customers to choose in various scenarios Outstanding scalability and flexibility Up to 12 I/O modules, 48 front-end and back-end I/O ports that support 4 or 8 Gbps FC, GE or 10GE, and 6 Gbps SAS 2.0 wide ports. The unique TurboModule technology enables the density of I/O modules to reach the maximum and the number of front-end and back-end I/O modules to be adjustable in proportions, cutting maintenance cost significantly. TurboBoost technology Industry-leading hardware empowers the T series to deliver solid system performance, the SmartCache boosts read performance notably by continuously monitoring hotspot data and caching it to SSDs, and the RAID groups consisting of only SSDs further improve the performance to a new high. The three-level performance boost technology greatly contributes to the proven system performance, lowering the total cost of ownership (TCO). 2

6 High reliability and availability TurboModule technology TurboModule technology allows customers to swap the controller, hard disk, I/O module, fan module, power module, or built-in battery unit (BBU) without powering off the system or interrupting services. Online expansion is applicable to hot-swappable I/O modules. Cache data protection The BBUs ensure that cache data is written to the data coffer in case of a power failure, increasing overall reliability. Disk pre-copy technology The systems detects failing disks and copies data on these disks to their spare disks to prevent from RAID degradation and data loss. Advanced data protection technologies HyperImage and HostAgent provide consistent snapshots and instant data recovery from snapshots for application systems. HyperClone and HyperCopy implement local data backup and recovery, and HyperCopy across storage platforms enables data protection between heterogeneous storage. HyperMirror achieves synchronous and asynchronous remote replication for backup and disaster recovery. Efficient, easy, and flexible Unified IO modules The T series products use the same I/O modules, reducing the TCO to the maximum. High disk density A 2 U or 4 U disk enclosure can house up to twenty-four 2.5-inch or 3.5-inch disks, decreasing the expansion cost by 60% compared with low-density disk enclosures Thin Provisioning HyperThin extends storage capacity automatically and increases disk usage, allowing customers to purchase storage on demand and maximize the return on investment (ROI). User-friendly OAM tool The T series supports the OceanStor Integrated System Manager (ISM) for unified management and maintenance, remarkably reducing the OAM costs. The ISM enables you to complete initial configuration in five steps and upgrade dual controllers' firmware online just in one click. In addition, the ISM supports audible and visual alarms, SMS alarms, and alarms. Energy saving Disk spin-down Hard disks automatically spin down in off-peak hours, lowering power consumption by 40%. 16-step intelligent fan speed control The fans automatically adjust rotational speed according to system temperature. This helps reduce power consumption and noise. CPU frequency control The CPU intelligently adjusts working frequency according to the workload. In off-peak hours, it lowers working frequency to reduce power consumption. 3

7 Solution description This solution is intended for medium-to-large enterprise that is planning to deploy Microsoft Exchange Server 2010 on HUAWEI storage and would like to leverage the Exchange Server 2010 mailbox resiliency Database Availability Group (DAG) feature. The solution design represents an Exchange Server 2010 environment supporting users in a mailbox resiliency configuration across two OceanStor S5500T storage arrays. The solution is designed to provide outstanding performance and flexibility for today s and tomorrow s Exchange users. In this solution, users are deployed across a DAG with four Exchange mailbox servers, two servers in the primary site and two in the secondary site. Each database has two copies, one RAID-protected by a S5500T array and the other replicated to a different server on the secondary array. Each Exchange mailbox server hosts 5000 active users and 5000 passive users during normal operation, and supports active users should a database switchover from another mailbox server due to a server failure or for maintenance purposes. The user mailboxes have a 1.5 GB capacity and 0.10 IOPS profile with a 20 percent throughput reserve. The solution is designed to eliminate a single point of failure and handle the loss of an array, mailbox server, database volume, HBA. Figure 1-2 Solution architecture 4

8 Storage configurations Figure 1-3 Storage Configuration Figure 1-3 illustrates how the disks in a S5500T were organized into RAID Groups used by databases, logs. Each set of numbered disks represents a RAID Group. There were 68 2TB 7.2K RPM NL SAS disks used in there tests, configured into 9 RAID Groups. RAID Groups 0-7 are used for database, 4 LUNs each with 1800GB created in each Group. RAID Group 8 is used for log, 32 LUNs each with 100GB are created in the RAID Group. One S5500T is used by two servers. LUNs from RAID Group 0, 1, 4, 7 are mapped to server 1 used for database, 16 LUNs from RAID Group 8 are mapped to server 1 used for log. LUNs from RAID Group 2, 3, 5, 6 are mapped to server 2 used for database, 16 LUNs from RAID Group 8 are mapped to server 2 used for log. Targeted customer profile This solution is designed for any medium-to-large enterprise that is planning to consolidate its Exchange Server 2010 storage environments and requires a highly reliable, scalable storage solution. The design of this solution has the following characteristics: active mailboxes 100 messages sent/received per user per day,0.10 IOPS throughput with a 20 percent overhead(0.12 IOPS tested) 1.5 GB mailbox size Two Exchange Servers with a 5000 active/5000 passive configuration (tested by simulating active users per server) Native DAG replication mechanism for mailbox resiliency and high availability with two database copies maintained 7*24 background databases maintenance (BDM) enabled Two HUAWEI OceanStor S5500T storage arrays(the solution is tested with two mailbox servers connected to a single array) 5

9 Tested deployment The following tables summarize the testing environment Table 1-1 Simulated Exchange configuration Attribute Value Number of Exchange mailboxes Number of Databases Availability Groups(DAGs) 1 Number of servers/dag Number of active mailboxes/server 4(2 tested) 5000(10000 simulated in Jetstress testing) Number of databases/mailbox server 16 Number of copies/database 2 Number of mailboxes/database 625 User profile Database LUN size Log LUN size Total database size for performance testing 0.10(0.12 tested in Jetstress) 1800GB 100GB 30042GB(15021GB for each server) Percentage of storage capacity used by Exchange database 52%(30042/57600) Table 1-2 Storage hardware Component Storage Connectivity(FC or iscsi) Storage model and OS/firmware revision Storage cache Description FC OceanStor S5500T, V100R002C00SPC008 32GB(16GB for each controller) Number of storage controllers 2 Number of storage ports 4(Two for each controller ) Maximum bandwidth of storage connectivity to host HBA model 16Gbps(2*8 Gbps FC Ports) Qlogic QLE2562 PCI Express to 8Gb FC Dual Channel Number of HBAs/host 1 Host server type IBM X3850 with Intel(R) Xeon(R) X7542,2.67 GHz,256 GB RAM Number of disks used in solution 68 Maximum number of disks supported by OceanStor S5500T storage array 288 Table 1-3 Storage software Component Description HBA driver HBA Queue Depth Setting 32 Host OS Microsoft Windows Server 2008 R2 Enterprise 6

10 Table 1-4 Storage disk configuration(mailbox store disks) Attribute Description Disk type, speed and firmware revision 2 TB NL SAS 7.2K RPM, Firmware 0002 and 0006 Raw capacity for each disk (GB) 1862 GB Number of physical disks in test 64 Total raw storage capacity (GB) GB RAID level RAID 10 Total formatted capacity 57600GB(28800GB for each server) Storage capacity utilization(percentage) 48%(57600/119168) Database capacity utilization(percentage) 25%(30042/119168) Table 1-5 Storage disk configuration(transactional log disks) Attribute Attribute Description Description Disk type, speed and firmware revision 2 TB NL SAS 7.2K RPM, Firmware 0002 Raw capacity for each disk (GB) 1862 GB Number of physical disks in test 4 Total raw storage capacity(gb) 7448 GB RAID level RAID 10 Total formatted capacity 3200 GB(1600 GB for each server) Table 1-6 Replication Configuration Attribute Replication mechanism Description Exchange 2010 DAG Mailbox Resiliency Number of links 2 Simulated link distance Link type Link bandwidth LAN IP Gigabit Ethernet(1 Gbps) 7

11 Best practices Microsoft Exchange Server 2010 is a disk-intensive application. It represents two distinct workload patterns to the storage, with 32KB random read/write operations to the database, and sequential write operations of varying size (between 512B up to the log buffer size) to the transaction logs. For this reason, designing an optimal storage configuration can prove challenging in practice. Exchange 2010 is designed to reduce the amount of I/O traffic to storage subsystem that was necessary in previous Exchange versions. Microsoft accomplished this by implementing a larger page size (32KB) and more effectively using mailbox server cache through the use of larger log checkpoint depth. Due to this I/O reduction, customers can now deploy Exchange Server 2010 on low cost NL SAS disks in addiction to FC, SAS, and SSD drives. Based on the testing run using ESRP framework, HUAWEI recommends these best practices to improve the performance of HUAWEI OceanStor S5500T running Exchange Mailbox servers Partition alignment Setting 512KB alignment is recommended when partitioning MBR disk. NOTE Partition alignment can only be set by using DISKPART tool in windows. For example: DISKPART> SELECT DISK 1 DISKPART> CREATE PARTITION PRIMARY ALIGN=512 NTFS Allocation Unit Size The page size of Exchange 2010 is 32KB. The NTFS allocation unit size should bigger than 32KB. In fact, 64KB is a better choice in any scenario. HBA driver and parameters Newest StorPort driver from HBA vendors is recommended. The default setting of HBA parameters is recommended by HUAWEI. But the command queue depth of some HBA is too small for Exchange Server is the recommended HBA queue depth. Multiple possible fail points Multiple HBAs, switches, and path, and install OceanStor Ultrapath for storage multi-path. The recommended SAN network is illustrated in the following figure. 8

12 Figure 1-4 Recommended SAN network for Exchange Server Core storage Isolate workload For performance consideration, keep the Exchange workload isolated from other applications. For reliability and performance consideration, physically isolate the Exchange database from the log groups. The database and log groups have different I/O patterns, putting them together makes the performance degrade. Disk types S5500T supports SAS, NL SAS, SATA, and SSD disk types, allowing customers to choose in various scenarios. SAS and NL SAS are both suitable for Exchange database and log. SATA is not recommended for Exchange database and log. RAID level RAID10 is recommended for log groups. RAID10 is recommended for database when using NL SAS disks. RAID5 is recommended for database when using SAS disks. Number of disks in RAID group For RAID5, the recommended number of disks is 5, 7, 9, or 11. For RAID10, the recommended number of disks is 4, 6, 8, 10, or 12. Hot spare Configuring hot spare disk in S5500T is strongly recommended. The vault disks are not suitable for host spare. Configure approximately the same number of hot spares as the number of disk enclosures. 9

13 LUN configuration Owning controller S5500T has two active-active storage controllers. To keep the load balance of the two storage controllers, average distributing LUNs on the two controllers is recommended. Stripe unit The recommended stripe unit of RAID5 is 64KB or 128KB, and keeping the stripe size less or equal to 1MB is recommended. The recommended stripe unit of RAID10 is 512KB. Pre-fetch strategy For most scenarios, set pre-fetch strategy to 'intelligent pre-fetch'. Setting constant pre-fetch is helpful for Exchange log for replication. Setting the asynchronous pre-fetch parameters may helpful. Write strategy Write back with cache mirroring is recommended in all scenarios. Write through can be used in high reliability requirement and low performance requirement scenarios. Planning Transactional I/O performance Transaction I/O indicates the I/O operations related to mailbox databases and transaction logs. The I/O type of mailbox databases is small-block random read/write, and that of transaction logs is small-block sequential write. The transactional I/O is mainly determined by the random IOPS of the mailbox databases. The transactional IOPS is subject to the amount of database cache per user (D) and the messages each user sends and receives per day (M). The read IOPS of Exchange 2010 approximates to ( x M) x (D ^ -0.65), and the write IOPS approximates to ( x M). The following table lists the estimated IOPS values under different loads. Table 1-7 Database cache and estimated IOPS per mailbox under different loads Messages Per Mailbox Per Day Database Cache Per Mailbox Estimated IOPS Per Mailbox

14 Table 1-8 Transitional IOPS of different types of disks Disk Type Transactional IOPS NL SAS 50 SAS 200 After the IOPS is determined, estimate the required number of disks according to the IOPS per disk and RAID level. Table 1-8 lists the estimated transactional IOPS of disks (IOPSDISK). Use the following formula to calculate the required number of disks (N) with the number of mailboxes (U) and the load (M). RAID 10 creates stripes for multiple mirror groups. Each mirror group usually has two disks. Therefore, one host write I/O generates two disk write I/Os. The formula is as follows: U x (( x M) x (D ^ -0.65) + 2 x ( x M)) = IOPSDISK x N RAID 5 uses striped parity. Each stripe has a disk that stores redundant parity data. The parity data is obtained by executing XOR operations over the data on other disks in the stripe. Therefore, one host write I/O generates four disk write I/Os. The formula is as follows: U x (( x M) x (D ^-0.65) + 4 x ( x M)) = IOPSDISK x N Transaction logs are small sequential I/Os. When S5500T storage is used, the small I/Os are converted into large I/Os in the storage cache and then written to the disks, eliminating the possibility of performance bottlenecks. Planning Storage Capacity Having sufficient capacity is critical. When a database disk runs out of space, the database goes offline. All mailboxes in the database cannot be accessed. When a transaction log disk runs out of space, it causes all of the databases in that storage group to go offline. All databases in the storage group cannot be accessed. The following describes the planning of the database and log LUN capacity. Planning Database LUN Capacity A mailbox database contains mailbox quota, white space, internal data structure, dumpster, and content indexing. You must consider these factors when planning the database LUN capacity. Mailbox quota: Each mailbox has a specific size, which is called mailbox quota. The total capacity of all mailboxes equals to the number of mailboxes multiplied by the mailbox quota. White space: The database itself will always have free pages, or white space, spread throughout. The size of white space is correlated with the average mailbox load, and it can be approximated by the amount of mail sent and received by the users with mailboxes in the database. Dumpster: Each database has a dumpster that stores hard-deleted items. By default, items are stored for 14 days in the dumpster, and after 14 days, the items will be deleted within a maintenance period which lasts for 24 hours. The size of the dumpster can be approximated by the amount of mail sent and received per user in two weeks with mailboxes in the database. 11

15 Content indexing: Content indexing creates an index, or catalog, that allows users to easily and quickly search through their mail items. Exchange 2007 creates an index that is about 5% of the total database size. Internal data structure: The internal data structure, including tables, views, and internal indexes, requires additional 20% space. Maintenance: The database maintenance operations include online database defragmentation, database repair, and offline database defragmentation. Database recovery and offline database defragmentation will need capacity equal to the size of the target database plus 10%. Replication or backup: When continuous replication or backup is used, the capacity of the target database LUN is equivalent to that of the source LUN. For example, if the database has 1000 mailboxes, the quota for each mailbox is 2 GB, and each mailbox sends/receives averagely 52 MB of mail per week (10 MB per work day and 2 MB for weekends), the database capacity is calculated as follows: Considering white space and dumpster: Mailbox size = 1000 x (2048 MB + 10 MB + 52 MB) = MB Considering content indexing: Mailbox size = MB/95% = MB Considering internal data structure: Mailbox size = MB x 120% = MB Considering maintenance operations: Mailbox size = MB/90% = MB Considering replication or backup: Total size = x 2 = MB Planning Log LUN Capacity The transaction log files are a record of every transaction performed by the database engine. Each transaction log file in Exchange 2010 is 1 MB in size. Table 1-9 lists the number of transaction logs that are generated on an Exchange 2010 Mailbox server under different loads every day. 12

16 Table 1-9 Number of generated transaction logs under different loads Messages Per Mailbox Per Day Logs Generated Per Mailbox When you set the LUN size, consider the following factors: Backup period: Logs record all the data changes in a database. You can replay the logs to restore the database to an accurate checkpoint. To ensure that the data is recoverable, you must save all the logs between the previous backup and the current point in time. After you back up the database, you can delete or move the logs before the backup. The log LUN size must be larger than the total size of all logs that are generated during the backup period. Mailbox moving: If there s mailbox moving operations, leave enough capacity in log LUN. Database copy count: The log capacity of the entire system is increased based on the number of database copies chosen in the high availability deployment. If you have three database copies spread across three servers, you need to provision log capacity for each copy on each server. Log truncation mechanism: High availability in Exchange 2010, with the ability to have up to 16 copies of each mailbox database, provides the foundation to use continuous replication circular logging as the log truncation/deletion mechanism as opposed to running Full/Incremental backups to truncate/delete the older logs. Database copy replay lag: Leave enough space for database copy replay lag. Business growth: Considering the potential fluctuation or increase of the business, reserve 20% of the space for logs. Using Advanced OceanStor Features Using OceanStor HyperImage, HyperClone, HyperCopy, or HyperMirror to protect Exchange data is efficient and easy. Also, the server CPU and memory resources are released to application. 13

17 Test results summary This section provides a high-level summary of the ESRP test results for this solution and the detailed test result reports are attached to this PDF in HTML format. The solution was validated using Microsoft Jeststress 2010 to ensure that the storage design satisfies the disk I/O and capacity requirements for the target profile. Jetstress simulates Exchange Server 2010 I/O types at the database level by interacting directly with the database technology of the ESE on which Exchange is built. Jetstress can be configured to test the maximum I/O throughput available to the disk subsystem within the performance constraints of Exchange. Alternatively, Jetstress can be configured to accept a specific user profile (user count, IOPS per user) and validate that the disk subsystem is capable of maintaining an acceptable performance level with such a profile. Both configuration options produce similar results. Reliability The reliability test runs for 24 hours. The goal is to validate that the storage can handle high I/O load for a long period of time. Both log and database files are analyzed for integrity after the stress tests to ensure that there is no database or log corruption. The 24-hour stress test results reveal: No errors in the saved event log file. No errors for the database and log checksum processes. Storage performance The primary storage performance testing is designed to exercise the storage with maximum sustainable Exchange I/O activity for two hours. The test shows how long it takes for the storage to respond to an I/O under load. The following data summarizes all of the logical disk I/ Os and average of all the logical disks I/O latency in the two-hour test duration. Each server is listed separately and the aggregate numbers across all servers is listed as well. Individual server metrics The following table shows the sum of transactional I/O across all databases (database read operations per second and database write operations per second) and the average latencies across all databases and logs for each server. The configuration is designed to achieve a target of 1200 IOPS for users on each server. This includes 20 percent overhead, above the 0.10 IOPS user profile, for reserve throughput capability. 14

18 Table 1-10 Individual server metrics Database I/O Server1 Server2 Achieved transactional I/Os I/O database reads/sec I/O database writes/sec I/O database average read latency (ms) I/O database average write latency (ms) Transaction log I/O Server1 Server2 I/O log writes/sec I/O log average write latency (ms) Performance across servers The following tables show the sum of transactional I/O and the average latency across all servers in the solution. Table 1-11 Performance across servers (Database) Database I/O Value Target transactional I/Os 2400 Achieved transactional I/Os I/O database reads/sec I/O database writes/sec I/O database average read latency (ms) I/O database average write latency (ms) 3.27 Table 1-12 Performance across servers (Log) Transaction log I/O Value I/O log writes/sec I/O log average write latency (ms) 3.13 Database backup and recovery performance This section has two tests: The first measures the sequential read rate of the database files and the second measures recovery/replay performance (playing transaction logs in to the database). Database read-only performance This test measures the maximum rate at which databases can be backed up via VSS. The following table shows the average rate for a single database file across two servers. 15

19 Table 1-13 Database read-only performance Metric Server1 Server2 MBs read per second per database MBs read per second per server Metric Value Total MBs read per second for all two servers Transaction log recovery/replay performance This test measures the maximum rate at which log files can be played against the databases. The following table shows the average rate for 500 log files played against a single database. Each log file is 1MB. Table 1-14 Transaction log recovery/replay performance Metric Value Average time to play one log file (in seconds) 3.83 Performance across servers The detailed test results for both mailbox servers are attached to this PDF in the form of Jeststress HTML reports. These reports include: 2-hour performance test Performance_Server1.html Performance_Server2.html Checksum for 2-hour performance test Performance_DBChecksum_Server1.html Performance_DBChecksum_Server2.html 24-hour performance test Stress_Server1.html Stress_Server2.html Checksum for 24-hour performance test Stress_DBChecksum_Server1.html Stress_DBChecksum_Server2.html Database backup test DatabaseBackup_Server1.html DatabaseBackup_Server2.html Soft recovery test SoftRecovery_Server1.html SoftRecovery_Server2.html 16

20 Copyright Huawei Technologies Co., Ltd All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademark Notice, HUAWEI, and are trademarks or registered trademarks of Huawei Technologies Co., Ltd. Other trademarks, product, service and company names mentioned are the property of their respective owners. General Disclaimer The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice. HUAWEI TECHNOLOGIES CO., LTD. Huawei Industrial Base Bantian Longgang Shenzhen , P.R. China Tel: Version No.: M C-1.0