Revision 1.1 A Dell Technical Report. Page 1

Transcription

1 Dell Reference Architecture Sizing and Best Practices for Microsoft Exchange Server 2007 in a VMware ESX Server Environment using EqualLogic PS Series Storage Revision 1.1 A Dell Technical Report Page 1

2 TABLE OF CONTENTS TABLE OF CONTENTS... 2 EXECUTIVE SUMMARY... 3 AUDIENCE... 4 UNDERSTANDING EXCHANGE SERVER 2007 STORAGE REQUIREMENTS... 4 INPUTS FOR EXCHANGE SERVER 2007 STORAGE DESIGN... 5 USER COMMUNITY INFORMATION... 5 MICROSOFT IOPS GUIDELINES FOR VARIOUS USER PROFILES... 6 DESIGNING A SERVER INFRASTRUCTURE... 6 DESIGNING A STORAGE INFRASTRUCTURE... 8 CALCULATING THE IOPS REQUIREMENTS... 8 CALCULATING THE CAPACITY REQUIREMENT Database Capacity Log Capacity STORAGE GROUP SIZING CONSIDERATIONS LARGE MAILBOX CONSIDERATIONS DATABASE AND LOG VOLUME LAYOUT CONSIDERATIONS CHOOSING A RAID TYPE Comparing RAID 10 to RAID Selecting RAID Type for Log Volumes SELECTING A PS SERIES MODEL AND DRIVE CAPACITY SIZING EXAMPLE OPTIMAL PS SERIES MODEL, DRIVE CAPACITY AND RAID TYPE ADDITIONAL CONSIDERATIONS FOR THE PRODUCTION ENVIRONMENT DESIGNING A NETWORK INFRASTRUCTURE DELL REFERENCE ARCHITECTURE FOR 5000 USERS HARDWARE SETUP SUMMARY OF TEST RESULTS (WITHOUT SNAPSHOT OR REPLICATION) BEST PRACTICES A STARTING POINT OR A BUILDING BLOCK ASM/ME AND EXCHANGE 2007 SMART COPY ADDITIONAL INFRASTRUCTURE CONSIDERATIONS PUTTING IT ALL TOGETHER: VALIDATE THE DESIGN APPENDIX A: REFERENCES AND LINKS DELL MICROSOFT VMWARE OTHER USEFUL LINKS APPENDIX B: DATA COLLECTION FROM A CURRENT EXCHANGE SERVER APPENDIX C: RAID10 AND RAID50 TEST RESULTS RAID 10 TESTING RAID 50 TESTING Page 2

3 EXECUTIVE SUMMARY message volume and mailbox sizes are experiencing explosive growth due to increased reliance on for communication and collaboration services. To keep pace with the changing user profile characteristics Microsoft has also been continuously evolving the Exchange architecture. The latest in server, storage and network technology can offer many enhanced capabilities and benefits for implementing Microsoft Exchange 2007 in enterprises of all sizes. For example, Internet SCSI (iscsi) technology combined with server virtualization software using hypervisor-based virtual machines (VMs) can significantly reduce total cost for implementing the Exchange Server, and connecting, managing and protecting the Exchange data. As a result, when you implement Exchange 2007 your messaging IT infrastructure architecture is likely to be considerably different than it was when you implemented a previous version of Exchange. The basic principles for designing server, network and storage as part of a robust, reliable messaging system, however, remain largely the same. It s still important to design the Exchange Server 2007 to meet your peak load requirements, and leave sufficient headroom in your design to accommodate growth and seasonal fluctuations. This paper describes the Dell Reference Architecture, a blueprint or a starting configuration, for Microsoft Exchange Server 2007 in a virtualized server environment and provides guidelines for designing and sizing the storage for the same using Dell EqualLogic PS Series storage systems. It also offers the latest best practice recommendations and other considerations along the way. The approach taken is to design a storage layout that meets the following goals: Optimal performance User response times are acceptable and there is no buildup of mail queues even during peak periods. Efficient backup and rapid recovery Backups complete within the allotted window, with an acceptable impact on the production environment. Local recovery meets the service level agreement (SLA) requirement. Simplicity of design Configuration is based on a building block approach so that it can be easily scaled up or down. Many Dell Consultants use the Dell Exchange Advisor tool to build supported Microsoft Exchange 2007 environments. The Microsoft Exchange Team Blog also provides an Exchange 2007 Mailbox Server Role Storage Requirements Calculator. However, EqualLogic Exchange 2007 Sizing Tool must be used as a primary resource of information when designing Exchange 2007 configurations with EqualLogic storage. NOTE: Please note that a full list of considerations and best practices for configuring backup and local recovery of Exchange Server 2007 using Dell EqualLogic storage snapshots, and disaster recovery will be included in a future version of this paper. Page 3

4 AUDIENCE This white paper is intended for Enterprise Technologists, Solution Architects, System Consultants and Implementation Engineers who are involved in working with customers and prospects on Microsoft Exchange Server 2007 opportunities. The reader should have a general knowledge of Microsoft Exchange Server 2007 and Windows technology including backup, as well as an advanced understanding of Dell EqualLogic PS Series features, functions and terminology. UNDERSTANDING EXCHANGE SERVER 2007 STORAGE REQUIREMENTS Under the Windows Server Virtualization Validation Program (SVVP) Microsoft now provides Mainstream Support for Microsoft Exchange Server 2007 in a hypervisorbased virtual server environment. VMware ESX Server and Microsoft Windows 2008 Hyper-V are two virtual machine environments that are typically used to implement Exchange Server This paper uses design examples based on VMware ESX Server virtual machine (VM) environment. The best practices and requirements presented here are applicable if you are implementing Exchange 2007 in a physical server environment or a VMware environment. In order to design and size storage configuration for an Exchange 2007 deployment in a virtualized server environment a solution architect must understand: 1. The server infrastructure, 2. The storage requirements, and 3. The network infrastructure. Microsoft provides comprehensive guides for understanding Exchange Server 2007 I/O improvements over Exchange Server 2003 and for planning Storage and Server Memory configurations. Moreover, Microsoft Exchange Team Blog provides a comprehensive spreadsheet based Exchange 2007 Mailbox Server Role Storage Requirements tool and Dell provides an online Dell Exchange Advisor tool to help you design and configure the storage for your Exchange Server While these tools prescribe one or more storage configuration options to meet your needs, they don t go behind the scene to help you understand why these storage options are the most appropriate storage options for your Exchange 2007 environment. This technical report describes the design parameters for planning a: 1. VMware-based virtualized server environment, 2. EqualLogic PS Series-based storage environment, and 3. iscsi based Storage and Server Network environment. These design parameters are calculated based on a set of simple inputs obtained from the customer for their environment. Using an example, this technical report then shows how to use the calculated design parameters to create EqualLogic PS Series storage reference architecture a blueprint or a starting configuration, with specific PS Series model, drive size and type, quantity of arrays, and volume layout for a 5000 user Exchange 2007 environment. This example reference architecture is also tested and verified against using Microsoft recommended tools like JetStress and LoadGen. While designing and configuring storage for your customer s Exchange 2007 environment, use this Reference Architecture as a baseline and scale it appropriately to create a starting storage configuration and a layout. However, always consult the EqualLogic Exchange 2007 Sizing Tool as the primary resource of information for designing your customer s Exchange environment using EqualLogic PS Series storage arrays. Page 4

5 INPUTS FOR EXCHANGE SERVER 2007 STORAGE DESIGN This section describes data items to gather for the target Microsoft Exchange 2007 environment. If an older version of Microsoft Exchange is being used in this environment, such information can also be obtained from a Microsoft Exchange Server Profile Analyzer. Whenever possible, it is always valuable to have this data supported by empirical measurements (concurrent users, messages sent/received, log files/day, etc.) from the current environment. Appropriate tools and counters for empirical measurement of an existing Exchange environment are included in Appendix B. USER COMMUNITY INFORMATION The information gathered in this category should lead to a good estimate of the I/O profile for a set of users. Mailbox Count and Size How many total numbers of mailboxes today? (Including non-user mailboxes such as central help desk, and conference room mailboxes.) Anticipated growth in number of mailboxes over the solution lifecycle? For example you have 1000 users and you expect this number to grow by 20% over solution s lifecycle. What are the mailbox size limits? Small - <512 MB, Average 512 MB to <1 GB, large 1GB-2GB, Very Large >2GB User Activity to Predict IOPS activity How many messages are sent and received per user per day and what is the average message size? Typical Message size is 50KB How many concurrent users during the peak period? Typically it is 80 to 100% unless there are users in different time zones. What is the Exchange activity level of the users? Large attachments, small messages etc. This helps us determine the additional mailbox size or IOPS. When are the peak activity periods? So that you don t schedule maintenance, or backup activity during the peak period. Typically Monday morning 9-11am Is there geographic dispersal of the users across time zones? So that you can determine user concurrency, and periods of less activity to determine backup window and schedule backup. Categorization of the user types leads to estimated base IOPS demand (see Table 1 below) Measured I/O in the existing environment gives the best starting point. Other User Profile Characteristics What mail clients are used and in what proportion? Both Outlook Web Access (OWA) and Outlook in Online Mode store indexes on and search against the server s copy of the data. This results in approximately double the IOPS/mailbox. Each Mobile devise (for example, Blackberry) is equal to two to three times the normal user IOPS. Other System Characteristics What is the retention period for items deleted by users but retained by server before permanent deletion? Typically 14 days. Page 5

6 Are you using Content Indexing to speed up searches? By default it is enabled in Exchange 2007 and requires an additional 5% capacity per Storage Group. Do you want to use a dedicated restore volume during maintenance or data restoration? Typically the answer to this question is yes because, if a restore volume is selected then additional restore capacity will not be needed in each exchange mailbox database volume. Are there special category users with different security, performance, or backup/recovery requirements? So that you can create a special Storage Group for them. MICROSOFT IOPS GUIDELINES FOR VARIOUS USER PROFILES Once you have the Exchange user community information, the next step is to categorize each user in one of the five Exchange User Profiles outlined in Table 1. This table has five Exchange user categories and provides an estimate of the IOPS demand and recommended server cache (memory) per user for each profile. Based on the Exchange User Profiles and the Mailbox size limits you will be able to estimate the EqualLogic Storage requirements. Table 1 Exchange User Profiles for 50KB Message Size and 250MB mailbox size in Cached Mode Mailbox profile Message profile Logs generated / mailbox Database Cache / Mailbox IOPS / Mailbox Light 5 sent/20 received 6 2 MB 0.11 Average 10 sent/40 received MB 0.18 Heavy 20 sent/80 received 24 5 MB 0.32 Very Heavy 30 sent/120 received 36 5 MB 0.48 Extra Heavy 40 sent/160 received 48 5 MB 0.64 DESIGNING A SERVER INFRASTRUCTURE Dell Labs has tested various configurations of Exchange 2007 in physical and virtual environments to evaluate the performance scalability and understand the sizing requirements for Exchange 2007 in virtualized environments. Moreover, VMware also has extensive documentation on designing Exchange Server 2007 using VMware infrastructure. Exchange Server 2007 design in a VMware environment follows the same rules that generally apply in a physical environment. For example, mailbox servers must be separated from other peripheral server roles, like CAS, and Hub Transport servers. Figure 1 below demonstrates what such an installation might look like conceptually. In this diagram, which has been taken from a VMware whitepaper on Designing and Sizing Examples of Microsoft Exchange Solutions on VMware, separate pools of VMs Page 6

7 have been used to logically separate production, test and development workloads. Similarly, distinct physical servers have been used to separate production environment from test and development environment. (The top two physical servers represent the production environment and the bottom two servers represent the test and development environment respectively.) VMware uses the terms virtual CPU (vcpu) and physical CPU to distinguish between the processors within the virtual machine and the underlying physical x86 based processors. The results of the tests described in a joint Dell-VMware whitepaper reveal the following best practices: 1) Ensure that the total number of vcpus assigned to all the virtual machines is equal to or less than the total number of core on the VMware ESX Server host machine. 2) The optimal number of vcpus in one VM is determined by the number of users that VM needs to support. 3) Performance of many small VMs is similar to that of a few large VMs. Figure 1 VMware Infrastructure Landscape for Exchange 2007 According to the tests in this whitepaper, VMware recommends that for best response time and most efficient processor utilization, it is best to focus on one of the four mailbox VM building blocks: 500, 1000, 2000 or 4000 users; and then apply the Microsoft guidelines in sizing CPU and Memory. (Larger building blocks are possible for larger environments.) Table 2 gives an example of CPU and Memory sizing for Exchange Server 2007 with average User Profile using Microsoft guidelines. Page 7

8 Table 2 Building block CPU and RAM sizing for mailboxes with average profiles Building Block Profile Average Average Average Average vcpu Base RAM (GB) Variable(MB/Mailbox) 3.5MB/Mailbox 3.5MB/Mailbox 3.5MB/Mailbox 3.5MB/Mailbox Total (GB) In addition to requiring licenses for the additional instances of OS and Exchange Mailboxes, the CPU and memory requirements for dividing the users across smaller VMs will also result in a slightly higher CPU usage and memory requirement. However, the advantage of spreading users across smaller VMs is that it will provide more flexibility to the administrator if VMs need to be migrated for load balancing. Similarly, if a failure occurs on a VM the number of users impacted will be smaller as opposed to a larger VM with higher number of users. Table 3 gives an example of CPU and Memory sizing for Exchange Server 2007 with Heavy User Profile using Microsoft guidelines. Table 3 CPU and RAM sizing for mailboxes with Heavy User Profiles Building Block Profile Heavy Heavy Heavy Heavy vcpu Base RAM (GB) Variable (MB/Mailbox) 5MB/Mailbox 5MB/Mailbox 5MB/Mailbox 5MB/Mailbox Total (GB) Please note that it is important to keep in mind Microsoft support policies & recommendations when running Exchange Server 2007 in VMware environments. DESIGNING A STORAGE INFRASTRUCTURE This section provides guidelines for calculating design parameters for configuring storage to handle the production Exchange data. When designing a storage configuration for Exchange, the two key design parameters to consider are: 1. The I/O operations per second (IOPS) requirement to handle the peak load 2. The storage capacity requirement Undersized storage solutions can cause a poor user experience, and running out of disk space can cause unexpected and unwanted outages. It s important that the resulting design meets both IOPS and capacity requirements. CALCULATING THE IOPS REQUIREMENTS It is a best practice to design for the peak I/O period typically mid-morning on Mondays. With an ever-increasing amount of data to backup and a shrinking backup window, the peak I/O period for an Exchange environment may take place during backup (at night time). Total IOPS required is the sum of the following three types of IOPS: 1) User Activity IOPS: Messages sent and received. 2) Background Activity IOPS: For example backup, and Online Maintenance. Page 8

9 User Activity IOPS Amount of database cache per user and the number of messages each user sends and receives per day are the two most important determinants for predicting User Activity IOPS requirement for an Exchange environment. Table 1 above provides the estimated User Activity IOPS for typical User Profiles with 50KB message size and 250MB mailbox size in cached mode. Unlike cached mode, online mode operations occur against the database and not the server memory. As a result read IOPS increase as much as 50%, increasing total IOPS by as little as 25%, for mailboxes greater than 250MB. Background Activity IOPS Background activities should be scheduled to take place only during off-peak times. However, several such activities cannot be scheduled during off-peak times. To estimate the amount of additional IOPS from these activities, it is best to perform tests in an environment that matches the target production environment as closely as possible. Typically such IOPS are estimated as a fraction of the User Activity IOPS. Log shipping Exchange LCR and CCR include ongoing shipment of log files. It typically places little I/O impact on a production Exchange server, but on a heavily loaded server, log shipping can have an impact on I/O latency. To support failover of two active sites in both directions requires configuring combined IOPS capacity of both sites on each site. Remote replication If a storage based remote replication solution is used for disaster recovery, its impact on IOPS and I/O latency must be factored in. Snapshots Snapshots are used for keeping a disk-based local copy for fast recovery and for doing off-host backup. In EqualLogic storage based Exchange designs additional IOPS due to snapshots are negligible. However, if there is any concurrent user activity during snapshot based off-host backup, it s important to pay attention to response times during this period. Exchange online maintenance often represents a high I/O demand on the system. So make sure that Exchange online maintenance is scheduled during off-peak hours. When using snapshots to meet recovery SLA, it is not uncommon to schedule Exchange online maintenance once a week during the weekend. Other IOPS considerations Large Mailboxes: Read IOPS increases as mailbox size increases. So design for additional IOPS if there are large mailboxes. Typically 10% additional IOPS. Mobile Users: Count each Blackberry user as the equivalent of 2 to 3 typical users. Page 9

10 Read/Write Ratio: If you follow the recommended server memory guidelines and use Cached Exchange mode, typical read/write ratio for Exchange 2007 is 1:1 or 50% reads. When using Outlook in Online mode, or when using large mailboxes, the read/write ratio will increase slightly on the Read side. Note Example showing baseline IOPS calculation From Table 1, for the following user population: Mailboxes = 5000 Mailbox size = 512 GB Mode = Cached User Profile = Very Heavy Message Size = 50KB Read/Write Ratio = 1:1 = 50% read and 50% write Server memory required = 2GB * 5 MB = 27 GB User Activity IOPS = 5000 * 0.48 = 2400 Large mailboxes add = 10% = 240 Spikes, Mailbox moves add = 20% = 480 Total Database IOPS = 3120 (130% of User Activity IOPS) Total Log IOPS = 75% of (2400 * 0.5) = 900 = 900 (38% of User Activity IOPS) Note: Log Shipping, Mobile Users, Online Mode Users may require additional IOPS headroom. Mailboxes not associated with a user: Organizations typically use a mailbox in Exchange for reserving a resource like conference room or equipment. Sometimes a mailbox also serves as the central contact point for a group. Depending on the number of these unassociated mailboxes (it can be significant 10 percent or more) and their activity level (which can vary significantly), it may be appropriate to factor these in to the IOPS calculation. By default, treat these mailboxes as equivalent to the typical user mailboxes, and always include them in the capacity calculation. Log to Database Write Ratio: In Exchange 2007, the log to database write ratio is approximately 3:4, for example for every 100 database writes there are 75 log writes. A detailed discussion of IOPS requirement using snapshot for fast local recovery and off-host backup will be provided in a future version of this guide. Please validate your IOPS requirements from the EqualLogic Exchange Sizing Tool and note that all options presented by this tool meet or exceed the I/O demand. CALCULATING THE CAPACITY REQUIREMENT In Exchange Server 2007 information is organized in Storage Groups. A Storage Group is a logical container for mailbox databases and their associated system and transaction log files. Storage Groups are the basic unit for backing up and restoring data in Microsoft Exchange (although you can restore a single database, mailbox or mail item). All databases in a Storage Group share a single backup schedule and a single set of transaction log files. Total capacity required is the sum of the following two capacities: 1) Database Capacity: Capacity of Exchange database volumes. Page 10

11 2) Log Capacity: Capacity of Log volumes. Database Capacity Add the following factors together to estimate the size of the Exchange database: User Count x Mailbox Size. For each category of users in the storage group, multiply the maximum mailbox size by the number of users in that category. Deleted item retention. You can roughly estimate the space taken up by these items by calculating the total size of the messages received and messages sent by users in the storage group in a day, and multiplying that by the desired deleted item retention period. White space. When online maintenance runs, it removes deleted items that have gone past the retention period and the emptied space is consolidated. The amount of white space will vary, but a rule-of-thumb estimate is to use 10% of the Exchange database size. Content index. If content indexing is enabled, the index is maintained on the same volume as the Exchange database. Estimate the size of the content index to be 5% of the database size. Free space. Allow an additional buffer of 20% free space on the volume for growth and to assure that you don t run out of capacity. Planned growth. If you plan to add users to the storage group or to increase the mailbox maximum at a later date, factor that in. Other database capacity considerations Moving mailboxes to a new storage group for load balancing: Gradually moving mailboxes from one storage group to a new one is potentially an alternative method to renew a database, with lower user downtime. It can also be used to re-balance the activity among storage groups. Don t forget to allocate sufficient capacity in the log volume of the target storage group (see the section Log Capacity below). Recovery Storage Group: If you have allocated space for a recovery storage group, it s possible to share this space for offline defragmentation. Log Capacity An Example showing baseline database capacity calculations 5000 Very Heavy mailboxes with 512MB/Mailbox Total Usable Mailbox Capacity requirement 14 days (10 work days) deleted item retention (10 work (120 received+30 sent)x50 KB/ message = 7.5 MB/user/day) 10 x 7.5 MB x 5000 = White space (10% of mailbox total space) Total database size Content index (5% of database size) Volume free space for growth (~20% of database size) Final database size (3991 GB is 156% of Usable Mailbox Capacity) Any change to an Exchange database is first recorded in a transaction log file. This includes not only user activity but also administrative activity such as online Page GB 2560 GB 375 GB 256 GB 3191 GB 160 GB 640 GB 3991 GB

12 maintenance, and mailbox moves. You can estimate storage capacity requirements for a log volume by considering: The number of log files generated per user per day. For example, from Table 1 for a very heavy user with 250MB mailbox size there are roughly 36MB of logs generated per user each day, including online maintenance activity. The number of days worth of logs to maintain. Typical Exchange full online backups are run nightly and prune the log files on each run. Differential backups do not prune the log files. It s important to have sufficient capacity for several days of log files in the event of problems with the backup process, or for protection with some disaster recovery solutions. Typically this number is 14. Number of mailbox moves. When a mailbox is moved from one Storage Group to another, transaction logs are generated in the target log volume that are roughly the same size as the mailbox being moved. Typically 10% of mailboxes are moved per week and they are moved all at once. An example showing baseline log capacity calculations 5000 very heavy users with mailboxes of 512 MB = 2560 Usable Mailbox Capacity 36 MB of logs per user per day (including online maintenance) Need to maintain 10 working days (2 weeks) of log files Need to be able to move up to 500 mailboxes at a time In this case the log capacity requirement would be: 5000 users x 36 MB x 10 days mailbox moves x 512MB Other Log Capacity Considerations Note With the exception of recovery operations and CCR, log I/Os are 100 percent writes. This makes the overall read/write ratio to be more writes and less reads. Circular logging is an Exchange feature that causes log files to be deleted after their transactions have been committed to the database. Only a handful of the logs are maintained at any time to save space. However, this sacrifices the capability to recover a database up to the minute. The feature is off by default and should not be turned on. To avoid running out of space on the log volume, it is important to confirm that the Exchange backup processes is deleting the committed log files. A detailed discussion of capacity requirement using snapshot for fast local recovery and off-host backup will be provided in a future version of this guide. Please validate your capacity requirements from the EqualLogic Exchange Sizing Tool and note that all options presented by this tool meet or exceed the capacity required. Page 12

13 STORAGE GROUP SIZING CONSIDERATIONS Smaller Storage Groups are generally better because they can be backed up and restored more quickly than larger Storage Groups. However, Storage Group size should be balanced against other factors, especially management complexity. Immediately deploying the maximum number of Storage Groups may add unnecessary management complexity to your system. For manageability and flexibility, Microsoft recommends that each new database be placed in its own storage group until the maximum number of storage groups is reached. Microsoft also recommends a maximum database size of 100 GB without continuous replication, and 200 GB with continuous replication. To meet recovery SLA of your organization if you require small databases, deploying large mailboxes could restrict a server to a smaller number of users than it could otherwise handle. For example, 1000 users with a 1 GB mailbox may require 14 storage groups (1400 GB total, including an estimated 40% additional space) because without a rapid recovery solution in place, the recommended database maximum is 100 GB. (See Table 4 for an estimated number of Storage groups for 1000 users for varying mailbox sizes.) Table 4 Storage Groups and Large Mailboxes SLA Database Max 250 MB Mailbox (350 GB Total) Number of Storage Groups for 1,000 Users 500 MB Mailbox (660 GB Total) 1 GB Mailbox (1,400 GB Total) 2 GB Mailbox (2,500 GB Total) 50 GB GB GB LARGE MAILBOX CONSIDERATIONS Exchange mailbox sizes have been trending larger. More than the decreasing cost per gigabyte of storage, this is being caused by: Larger messages the use of HTML and increasing use of bitmap files in signatures increases the average message size by three or more times. Larger attachments due to higher resolution graphical content and increasing use of rich media. Better search capability because Exchange 2007 includes content indexing that is significantly improved, users will be willing to maintain larger mailboxes because they can search for their older mail more efficiently. Removal of personal folders (PSTs) This may be the largest factor of all. Stored on users PCs and in various file shares, personal folders present problems for protecting the organization s intellectual property and for supporting compliance requirements. Organizations are increasingly absorbing the content of personal folders into the Exchange database and limiting their use. Exchange 2007 improves handling of very large mailboxes 2 GB and more. However, there are some side effects to consider before deploying very large mailboxes. Page 13

14 Increased I/O. Increase in I/O activity with large mailboxes may be significant, for example, when moving from a 200 MB mailbox to a 2 GB mailbox. Impact on the backup process. By increasing the maximum mailbox size significantly it s quite possible that you will force a change in the backup process to remain within the nightly backup window. Server de-consolidation. With the 50 database maximum per server and the recommended maximum database size of 200 GB (with a rapid recovery solution), there is potential for very large mailboxes to limit the number of users you can place on a server. DATABASE AND LOG VOLUME LAYOUT CONSIDERATIONS There are some important considerations in designing the disk layout for the Exchange database and log volumes: Each database volume must have its own transaction log volume. To avoid added recovery complications in the unlikely event of a drive hardware failure, it is a Microsoft best practice that the transaction log volume should never share the same spindles as the database volume for the same Storage Group. However, it is alright to combine the database volume of one Storage Group with the log volume of a different Storage Group. This is for protection rather than performance. If something should happen to the database, the log files are essential to recover transactions since the last backup. If those log files reside on the same physical disk and that disk is damaged, this option is lost. In very small Exchange 2007 deployments due to limited number of drives, the potential exists for both database and log volumes for the same Storage Group to share physical disk resources. In such deployments, due to performance and cost considerations one may be unable to accommodate Microsoft recommendation that is designed for recoverability. CHOOSING A RAID TYPE One of the most important decisions that you have to make when deploying a new Exchange storage solution is what RAID and Disk type(s) to use. Considerations to keep in mind are: usable capacity, performance, and protection. Exchange 2007 usable capacity and peak performance requirements (both normal operations including backup, and rebuild time) must be met for the selected RAID and Disk type. Once these two criteria are met, choosing a RAID and Disk type comes down to two factors: cost in terms of capacity utilization efficiency and protection level. Please see Table 2 below for RAID comparisons taken from the How to select the correct RAID for an EqualLogic SAN whitepaper. Table 2 RAID Comparison RAID Reads (Random/ Sequential) Writes (Random/ Sequential) Relative Cost Relative Protection Rebuild Performance 10 / / $$$$ 50 / / $$$ 5 / / $ 6 / / $$ Comparing RAID 10 to RAID 50 Depending on cost and performance considerations, the two most appropriate RAID options for production Exchange database volume are: Page 14

15 RAID 10 This offers the best performance with high protection, but only 50 percent of the raw capacity is usable in the RAID group. It is frequently recommended because, with today s large capacity disk drives, it provides sufficient space across the number of spindles required for handling peak I/O load. On a RAID 10 volume, there are two write operations for each write requested (a write to each mirrored disk group), described as a write penalty of 2 (two). Please note that all the writes to the drives in the stripe will be done in parallel. RAID 50 This configuration offers a higher usable capacity per RAID group than RAID 10. It can be effective for environments with very large mailboxes and/or lower IOPS requirements. However, in a RAID 50 group there are four read/write operations for each write requested (two reads to calculate parity, one write for data, and one write for parity), described as a write penalty of 4 (four). Please note that all the reads/writes to the drives in the stripe will be done in parallel. It is an accepted notion that RAID 10 is a better choice for random-write environments like Exchange. The effect is somewhat subtle; since all writes hit the cache, RAID 10 and RAID 50 RAID groups perform equally well until the storage system is sufficiently busy and the storage system has to rebuild. The advantage of using RAID 10 rather than RAID 50 with Exchange is that rebuild times and rebuild impact is reduced with RAID 10 in the event of disk replacement. The downside may be cost. RAID 10 requires more raw-disk capacity for a given usable-capacity. Microsoft recommends that we use 35% rebuild overhead for RAID10 and 100% rebuild overhead for RAID50. Selecting RAID Type for Log Volumes When choosing a RAID type for log file volumes, I/O performance and data protection are the overriding factors rather than capacity. RAID10 provides better response time than RAID 50 in degraded situations. In the case of a disk replacement, rebuilding of a RAID 10 set completes faster than that of a RAID 50 set. The longer the rebuild period, the more vulnerability there is to data loss. Data loss generally occurs if a second drive is lost during rebuild. So RAID10 is the best RAID type to use for log volumes, however it costs more. So, if cost is a concern, in large environments, a RAID50 set can also be used for log volumes. SELECTING A PS SERIES MODEL AND DRIVE CAPACITY EqualLogic PS Series storage arrays include storage virtualization technology where multiple PS Series members are combined in a group and virtualized into one or more pools of storage spanning one or more members. A volume represents a portion of a storage pool with a specific size and spreads across all the drives in the pool members. In an EqualLogic PS Series storage pool, performance and capacity scale linearly with the quantity (number) of members in a pool. Regardless of RAID type, a physical disk type can handle a certain number of Exchange style IOPS. I/O performance improvements on new disk models have not kept pace with the increase in storage capacity. Consequently, in traditional storage architectures the decision about RAID and drive type comes down to individual disk characteristics. However, in case of EqualLogic PS Series with virtualized pooled storage architecture, for a simple starting configuration, RAID and Disk type decisions can be as simple as iterating on the following two steps: 1) Selecting a Model (5000E, 5000XV, 5000X and 5500E) and quantity of members in a pool to meet the performance requirements and then, Page 15

16 2) Selecting the type of disk drives in the model and the quantity of members in the pool to meet the capacity requirements. Relative performance capability in terms of Exchange IOPS with typical Exchange read/write mix of various EqualLogic PS5000 models for a given RAID type are shown in Figures 2 and 3 and Table 6. We expect these charts for a given RAID type to be straight lines with fixed slope because in EqualLogic performance scales linearly when new members you added in a pool. Similarly, for a given RAID type usable capacity charts of various EqualLogic PS5000 models with specific drive capacities are also shown in Figures 4 to 5 and Table 7. Once again we expect these charts for a given RAID type to be straight lines with fixed slope because in EqualLogic usable capacity scales linearly when you add members in a pool. Using Figures 2 to 5, you can estimate the number of members of specific PS5000 model to meet your IOPS and capacity requirements for 5000 Very Heavy Users with 512MB mailboxes. Table 6 Relative Performance (Exchange 2007 IOPS with typical Read/Write mix) of PS Series Model Type for a given RAID Type Model RAID50 Relative Performance RAID10 Relative Performance PS5000XV (15K RPM drives) PS5000X (10K RPM drives) PS5000E (7200 RPM drives) PS5500E (7200 RPM drives) Table 7 Usable Capacity of a PS Series Model with a specific drive capacity for a given RAID Type Model RAID50 Usable Capacity RAID10 Usable Capacity PS5000XV (450GB 15K RPM drives) 4680 GB 2750 GB PS5000XV (300GB 15K RPM drives) 3120 GB 1830 GB PS5000X (400GB 10K RPM drives) 4160 GB 2440 GB PS5000E (750GB 7200 RPM drives) 7800 GB 4580 GB PS5500E (500GB 7200 RPM drives) GB GB Page 16

17 Relative Performance Relative Performance 7000 Performance Sizing for 5000 Very Heavy Users with 512MB Mailbox (Using RAID 50 PS5000 Models) PS5000XV PS5000X PS5000E PS5500E Required Performance 0 1 Member 2 Members 3 Members 4 Members No. of Members in a Group Figure 2 Relative Performance of PS5000 Models using RAID Performance Sizing for 5000 Very Heavy Users with 512MB Mailbox (Using RAID10 with PS5000XV) PS5000XV PS5500E PS5000X PS5000E Required Performance 0 1 Member 2 Members 3 Members 4 Members No. of Members in a Group Figure 3 Relative Performance of PS5000 Models using RAID 10 Page 17

18 Usable Capacity GB Usable Capacity GB Capacity Sizing for 5000 Very Heavy Users with 512MB Mailbox (Using RAID 50 PS5000 Models) 1 Member 2 Members 3 Members 4 Members No. of Members in a Group PS5500E-500GB disks PS5000E-750GB disks PS5000XV-450GB disks PS5000X-400GB disks PS5000XV-300GB disks Required Capacity Figure 4 Usable Capacity of PS5000 Models using RAID Capacity Sizing for 5000 Very Heavy Users with 512MB Mailbox (Using RAID 10 PS5000 Models) 1 Member 2 Members 3 Members 4 Members No. of Members in a Group PS5000E-750GB disks PS5000XV-450GB disks PS5000X-400GB disks PS5000XV-300GB disks Required Capacity Figure 5 Usable Capacity of PS5000 Models using RAID10 Page 18

19 SIZING EXAMPLE Consider the following calculation to determine the number of users that ONE PS5000XV WITH 450GB 15K RPM DRIVES IN RAID10 can handle: From Figure 6, 1000 Very Heavy User Profile (0.5 IOPS) requires 522 Relative Performance (including database and log IOPS AND overhead for potential growth and mailbox moves). Keeping 20% performance headroom (spikes, On-line users and Mobile users), 1000 users will require 1.2 x 522 = 626 relative performance. Microsoft recommends using a RAID10 Rebuild overhead of 35%, however, due to firmware optimizations the RAID10 Rebuild overhead for EqualLogic is in the range of 10% to 20%. Using a 20% RAID10 Rebuild overhead, 1000 users require (626*1.2) = 751 IOPS. From Table 6, PS5000XV can handle 2625 RAID10 Relative Performance. Dividing 2635 into 751 we can determine that 1 PS5000XV with RAID10 can meet performance requirement of approximately 3500 users. From Figure 7, 1000 users with mailbox size of 512MB requires 791 GB (including Database and Log volumes and some headroom for whitespace). Keeping additional 20% headroom (for growth), 1000 users will require approximately (791 * 1.2) = 950GB. From Table 7, one PS5000XV with 450GB drives in RAID10 has a usable capacity of 2750 GB. Dividing 2750GB into 950, we can determine that 1 PS5000XV with 450GB drives using RAID10 can meet capacity requirement for approximately 2900 users. Note that in this example, although PS5000XV with 450GB 15K RPM drives in RAID10 configuration can support a performance load of up to 3500 users with Very Heavy User Profile and 512 MB mailbox (Table 1), it has usable storage capacity to support only up to 2900 users. The additional performance can be useful when using snapshots for recovery and off-host backup. Following similar logic as above, from Table 7, a PS5000XV with 450GB 15K RPM drives in RAID50 has a usable capacity of 4.68 TB, and Relative Performance of 1500 IOPS. Microsoft recommends a RAID50 Rebuild overhead of 100%, however, due to firmware optimizations the RAID50 Rebuild overhead for EqualLogic is in the range of 20% to 40%. Using a RAID50 Rebuild overhead of 40%, 1000 users require (626*1.4) = 876 IOPS. As a result, a PS5000XV with 450GB 15K RPM drives in RAID50 can support a performance load of approximately (1500/876)=1700 users. With 512MB mailboxes 1700 users need (950/1000) * 1700) = 1615GB of usable capacity. Thus, a PS5000XV with 450GB drives in RAID50 has left-over usable capacity of ( ) = 3065GB. We can either keep this additional capacity as reserve for snapshots for recovery purposes or distribute it among users by increasing their mailbox size. However, cannot put any significant additional production performance load on this array. Similar logic can be followed to determine the number of users a specific PS5xxx Series model can support with RAID10 or RAID50 with other drive capacity types. Page 19

20 Total Relative Performance Total Performance by Load (1000 users at 1GB Mailbox Size Total Capacity-1.48TB) Total Performance 0.78, , , , Load - IOPS per User Figure 6 Performance Requirements by Load/User Profile Please note that the above example tries to establish a simple starting configuration for one unit of PS5xxx array based on performance and capacity considerations. However, there may be other important considerations (like cost and power consumption) to factor in before making a final decision. OPTIMAL PS SERIES MODEL, DRIVE CAPACITY AND RAID TYPE Using calculations like the one shown above, and factoring in increased I/O for larger mailboxes and other considerations, you can estimate the model number and quantity of PS Series model required to provide sufficient capacity and I/O performance for various configurations. Here are some best practices to follow: Leave at least 20% performance headroom for spikes. Leave at least 20% capacity headroom for growth in the design. Reasons for choosing a configuration with fewer (potentially more expensive) members include overall savings on power and floor space, and possibly staying within the Group limits of an EqualLogic SAN (limited to 12 members). Reasons for choosing a configuration with higher performance drives include getting better performance with fewer members such that backup windows can be shortened. Activities such as snapshots, Exchange online maintenance, and backup to disk/tape will complete more quickly. Page 20

21 Total Capacity in GB Total Capacity by Mailbox Size (1000 users at 0.26 IOPS/User LoadTotal IOPS=261) Total Capacity 10240, , , , , Mailbox Size in MB Figure 7 Capacity Requirements by Mailbox Size ADDITIONAL CONSIDERATIONS FOR THE PRODUCTION ENVIRONMENT This section describes some additional considerations for Hub Transport and Client Access server roles. Server roles divide functions in the Exchange 2007 architecture. This provides administrators some flexibility on how to configure the various functions on one or more servers depending on the size and activity of the environment. Two other required server roles in Exchange 2007 are: 1. Hub Transport All internal mail even within a server passes through the hub transport server. In a virtualized implementation of Exchange 2007, it is a best practice to implement the Hub Transport Server in a VM on a non-mailbox ESX server. It is a best practice to implement antivirus software on this Hub server. 2. Client Access Mail from all clients Outlook, Outlook Web Access, mobile devices, etc. passes through the client access server before being passed on to the appropriate mailbox server. If users access their inbox by using any client other than Microsoft Outlook, you must then install the CAS role in your Exchange Environment. In a virtualized implementation of Exchange 2007, it is a best practice to implement the CAS in a VM on a non-mailbox ESX server. (Hub Transport and CAS VMs can be implemented on the same physical ESX Server.) Depending upon the size of the Exchange environment multiple HUB and CAS servers may be required to provide load balancing. In a virtualized implementation this may require additional ESX servers. Similarly, one VM could also do both the HUB & CAS roles if required. For a detailed discussion on processor and memory configuration of non-mailbox server roles in Exchange 2007, please see VMware and Microsoft links in Appendix A. Page 21

22 DESIGNING A NETWORK INFRASTRUCTURE Dell Labs has done extensive testing for designing the iscsi SAN that connects the Exchange Server 2007 and the EqualLogic storage in a virtualized environment. Dell EqualLogic Configuration Guide provides a comprehensive discussion building an iscsi based SAN solution using EqualLogic PS Series Arrays. In addition to building the physical connectivity, VMware ESX Server includes VMnics and vswitches. Figure 8 provides a conceptual network configuration view of connecting VMnics to vswitches to physical switches. Design and Sizing Examples for Microsoft Exchange Solutions on VMware provides a detailed description of designing the virtual network using VMnics and vswitches. Figure 8: Conceptual view of ESX Server VMnics and vswitches Layout ESX server network configuration best practices: 1. Configure the ESX Server network for performance and High Availability and for connecting the Exchange Server VMs directly to the PS-array volumes/mailstores utilizing Microsoft s iscsi initiator versus using ESX s software initiator. 2. Each vswitch consists of two redundant VMnics (VMnics translate to physical NICs that are connected to 2 different physical iscsi Network Switches and are spread across two different network adapters in the ESX Servers). This gives us our redundancy should a NIC port/adapter, cable and or switch port/switch fail. Page 22

23 3. Create a separate VMotion network (vlan) to isolate VMotion traffic from the regular iscsi data traffic, thus potentially providing a better performance for both VMotion and VM/ESX iscsi data access. 4. Utilizing the Stacked switches also allows you to create a NIC team for the iscsi VMkernel, thus allowing you to load balance (not exactly true MPIO) volumes across both of the VMnics serving iscsi traffic, thus providing some additional performance gains within the ESX Software initiator. Utilization of Stacked switches provides us with ease of management as the stacking feature treats the two switches as one unit, along with providing more than adequate bandwidth for iscsi traffic and array communication across the stacking cables. Stacked switches also provide for full use of all GbE ports, where as non-stacking switches require ISL channels be used. This method consumes valuable GbE ports from switches leaving fewer ports available for initiator and target devices. Typically non-stacked switches don t scale well for use in larger environments and require you to manage the switches as two separate entities. DELL REFERENCE ARCHITECTURE FOR 5000 USERS HARDWARE SETUP To validate the Dell Reference Architecture for 5000 Exchange users, we tested the hardware setup in Figure 9 with RAID10 and RAID50 configurations with various user profiles. Our storage configuration was as follows: 5000 Exchange User Environment 20 Storage Groups each with 1 Database volume and 1 Log volume 250 users with 512 MB mailboxes in each Storage Group 20 database volumes with a size of 147GB each 20 Log volumes with a size of 30GB each We arrived at the above number of Storage Group and volume sizes based on the following reasoning: 1) We chose the number of SG for ease of recovery; didn t want large storage groups with large number of users. 2) Estimated 10-15% headroom for DB volumes 3) Estimated log volumes to be15% of database volumes. 250 users * 512MB * 10-15% overhead = 147GB for DB Volumes Log Volumes were 15% of DB volumes = ~30GB for Log Volumes Page 23

24 Figure 9: Tested Hardware configuration SUMMARY OF TEST RESULTS (WITHOUT SNAPSHOT OR REPLICATION) We used both JetStress and LoadGen to test the above configurations. LoadGen tests passed on all configurations. Table 8 provides a summary of JetStress test results. RAID10 Based on the JetStress testing performed (not including snapshots and not including RAID rebuild requirement), 2 x PS5000XV will support 5000 users with 512MB mailboxes with very heavy (0.5 IOPS per user) user profiles with very good performance. However, this configuration will leave little additional space on the array for snapshots, and replication. RAID50 Based on the JetStress testing performed (not including snapshots and not including RAID rebuild requirement), 2 x PS5000XV will support 5000 users with 512MB mailboxes with very heavy (0.5 IOPS per user) user profiles with plenty of additional space on the array for snapshots and replication but leave little additional IOPS for growth. Results of RAID10 and RAID50 tests conducted by Dell Labs, including SAN HQ and JetStress details, are shown in Appendix C. BEST PRACTICES DELL recommends RAID10 as the default choice for Exchange database and log volumes because: RAID10 group provides better IOPS than a RAID50 group with the same number of members. Page 24

25 Rebuilds of a disk in a RAID10 group will complete more quickly than RAID50 The smaller usable capacity of a RAID 10 group does not matter in the many cases where the member count must be determined by the IOPS requirements RAID50 will be appropriate for some Exchange deployments depending on IOPS load, mailbox size, and cost considerations. Table 8: 5000 mailboxes - JetStress Results for Various RAID and User Profiles Run # RAID Format JetStress Results for MB Mailboxes 2 x PS5000XV with 450GB 15K RPM Drives (Raw Capacity 14.4 TB) Max Formatted Capacity IOPS per User Desired IOPS Desired Capacity Achieved # of IOPS Pass/Fail TB TB 5241 Pass TB TB 4339 Pass The decision to choose between 10K RPM or 15K RPM drives will likely come down to cost. When IOPS are the determining factor in deciding the number of PS Series members you need, the number of PS Series members with 15K RPM drives required will usually be same or less than the number of PS Series members with 10K RPM drives. Balance the additional cost per PS Series member against savings on power required. A STARTING POINT OR A BUILDING BLOCK Since Exchange Mailboxes are generally grouped by tiers of user profile/load, and are fixed in size, Exchange implementations perform well and are easier to maintain and manage if you configure them from a limited number of proven building blocks consisting of PS Series pools with same RAID types and disk sizes. These building blocks can be easily replicated to create a complete Exchange storage design. In a building block the elements must be small and flexible such that the recovery of a mailbox or a mail store is quick; it must allow for growth, and it must have been proven to perform well. Figure 10 represents the Dell Reference Architecture, a starting point or a building block, for 5000 Exchange users with 512MB mailboxes and very heavy (0.5 IOPS per user) user profiles. It provides storage configuration and volume layout using 2 PS5000XV and 1 PS5000E. The 2 units of PS5000XV with 450GB 15K RPM drives in RAID10 are used for database volumes, and the one unit of PS5000E with 500GB 7.2K RPM drives in RAID50 is used for log volumes. This configuration will leave sufficient IOPS and capacity headroom for growth, snapshots and backup-to-disk. Note: Use of the sizing tools with Snapshot and Replication will dictate additional capacity and IOPS requirements as these functions will increase these requirements in conjunction with SCR/LCR/CCR. The results from such testing will be included in the next revision of this technical report. Page 25

26 Figure 10: Storage Layout and Configuration for Tested Hardware Setup ASM/ME AND EXCHANGE 2007 SMART COPY For most production environments, backing up data is essential. Unless you perform successful backups regularly, data restored from backup copies may not be usable. As a result, some backup solutions require you to close files or shut down applications while you run a backup. In a production Exchange environment that runs 24X7X365, such backup opportunities are fewer and shorter in duration. EqualLogic Auto-Snapshot Manager/Microsoft Edition (ASM/ME) uses Microsoft Volume Shadow Copy Service (VSS) to provide a framework for backing up and restoring Microsoft applications like SQL Server 2008 and Exchange 2007 data in a production environment. ASM/ME enables you to easily create Smart Copies of application data, which are point-in-time, consistent copies of application data in a PS Series Group. While you create Smart Copies, the applications remain online and there is little impact on performance and application availability. The time required for copying is minimized, and the application data is always consistent and usable. You use ASM/ME to create fast, coordinated copies of Exchange 2007 volumes on your PS Series group, ensuring that the backed-up data is easy to restore and available for fast recovery. Once a smart copy is made using ASM/ME, it can be backed up using traditional methods. When you use ASM/ME the underlying Microsoft VSS operations are transparent. Note: ASM/ME is not a replacement for a regular and complete backup of your data to long-term media, but rather can enhance and supplement your regular backup regimen by providing fast and efficient data recovery. You can also use your backup software to transfer the data from Smart Copies to long-term backup media. Page 26

27 Because the applications in your production environment remain online during such transfers, ASM/ME significantly reduces your planned computer downtime. In testing smaller sized Exchange 2007 environments (less than 1000 users), we found that in most cases ASM/ME can be quickly and easily setup to accommodate an RPO/RTO of say, up to 1 hour. However, in a large Exchange 2007 environment, you must take into account the size of storage groups/mailboxes, and data change rate, to determine the desired storage capacity when you set up ASM/ME to accommodate your RPO/RTO objectives. In such large Exchange 2007 environments, you must carefully set up ASM/ME Smart Copy schedule and retention count for each storage group to accommodate the RPO/RTO objectives. Depending upon the data change rate, your Smart Copies may potentially require a higher snapshot reserve space. In any case, it is a best practice to use an off-host verification server for soft recovery and checksum validation when using ASM/ME with Exchange as the offhost server will mount the Smart Copies and do the soft recovery and checksum verification thus off-loading the production Exchange Server from these tasks. ADDITIONAL INFRASTRUCTURE CONSIDERATIONS Unlike other storage systems, the PS Series storage arrays are self tuning and high performing out of the box. There is little customization outside of choosing the appropriate RAID type and volume sizes for your exchange environment and planning for snapshot and replication space. There is no internal tuning to be done on the PS-series arrays. However, there are a few server-based tuning and environment recommendations included here because problems that they can avoid are often mistakenly attributed to storage issues. The location and speed of domain controllers, DNS servers and the global catalog are vital to Exchange performance. The supporting servers should be local enough to the Exchange servers to provide a fast connection. There should also be enough of them to support the Exchange environment. Use Diskpart.exe, a disk partition tool, for Windows File-System alignment. Microsoft Exchange Server 2007 writes data in multiples of 8-kilobyte (KB) I/O operations, and I/O operation to a database can be from 8KB to 1MB. Therefore, make sure that the starting offset is a multiple of 8 KB. Failure to do so may cause a single I/O operation spanning two tracks, causing performance degradation. PUTTING IT ALL TOGETHER: VALIDATE THE DESIGN Dell constructed and tested the Exchange 2007 storage Reference Architecture for 5000 users with 512MB mailboxes and very heavy (0.5 IOPS per user) user profile. This Reference Architecture provides a starting point or a building block for sizing and designing Exchange 2007 configurations using EqualLogic PS Series storage. Before committing to a particular design you must validate it against the EqualLogic Exchange 2007 Sizing Tool, and if possible, compare it with known good configurations, such as the building block designs reported above. Whenever possible, the configuration should also be built and tested with performance tools such as Microsoft JetStress, LoadGen, Performance Monitor, and EqualLogic SAN Headquarter. Anticipate unforeseen issues to arise early in a rollout and be prepared to address them. Page 27

28 APPENDIX A: REFERENCES AND LINKS DELL Comprehensive Data Protection for Microsoft Exchange Application Integration and Data Protection (Microsoft Exchange and SQL) Deploying Microsoft Exchange Server 2007 in an iscsi SAN Microsoft Exchange Server 2007 on Dell EqualLogic Storage Best Practices MICROSOFT Microsoft Exchange Solution Reviewed Program (ESRP) for Exchange Exchange 2007 Planning Disk Storage Mailbox Server Storage Design VMWARE Deploying Exchange Server on a VMware infrastructure This is a very rich website including references, whitepapers, podcasts, blogs and kits Design and Sizing Examples: Microsoft Exchange Solutions on VMware OTHER USEFUL LINKS Windows Server Virtualization Validation Program (SVVP) Microsoft server software and supported virtualization environments Exchange Support in VMware ESX Server Exchange Support in Microsoft Windows 2008 Hyper-V Dell Exchange Advisor tool ge_2007?c=us&cs=555&l=en&s=biz&~ck=mn Page 28

29 EqualLogic Exchange 2007 Sizing Tool ( Exchange 2007 Mailbox Storage Calculator at the Microsoft Exchange Team Blog Microsoft Guide for Planning Storage Configurations Microsoft Guide for Planning Memory Configurations Understanding Exchange Server 2007 I/O improvements Exchange User Profiles Microsoft Exchange Server Profile Analyzer A389-69C281B2ABDA&displaylang=en Microsoft Exchange Best Practices Analyzer The tool scans Exchange servers and identifies items that do not conform to Microsoft best practices. This tool determines the overall health of Exchange servers and topology. AC22-E2DDBD258DF3&displaylang=en Dell EqualLogic Configuration Guide for designing an iscsi SAN APPENDIX B: DATA COLLECTION FROM A CURRENT EXCHANGE SERVER Here is the sequence of steps that can be used to collect Perfmon data from an Exchange Server 1. Click Start > Run > Perfmon. 2. From the left-hand menu, click Performance Logs and Alerts > Counter Logs 3. Right-click Counter Logs, and click New Log Settings 4. Use MS Exchange Collection-x as the name for the counter log. 5. Add the counters to the counter log: 6. Set the data collection interval to 60 seconds. 7. Set the log file type to Binary. 8. Specify the file location. 9. Configure the job to collect data Attached below the list of counters we can use to collect perfmon data for Exchange runs. We should use binary mode to collect the data and make sure that the collection interval time does not impact the disk space and system performance. 1. Physical Disk All Counters All Instances (Write down which disks contain Exchange Data) Page 29

30 2. Network Interface All Counters All Instances 3. SMTP Server 4. MSExchangeIS 5. MSExchangeISMailbox 6. Processor All Counters All Instances 7. Memory All Counters APPENDIX C: RAID10 AND RAID50 TEST RESULTS RAID 10 TESTING For RAID10 Configurations the following table summarizes the performance results that we achieved as compared to the maximum suggested IOPS for this configuration from the EqualLogic sizing tool. Table 9: RAID10 Test Results using JetStress Users 1000 IOPS/user 5000 IOPS/user RAID type Mailbox size # of threads IOPS Achieved Suggested IOPS from EqualLogic Exchange 2007 Sizing Tool RAID RAID As you can see from the above table, using JetStress on 2 PS5000XV units with RAID10, for 1000 users with 0.5 IOPS/user with 14 threads, we achieved 4,031 IOPS and passed the performance requirements comfortably. (Headroom of 1219 IOPS) Note: Further JetStress testing must be conducted to find the IOPS threshold (saturation point) by increasing the number of threads in the above testing. For 5000 users with 0.5 IOPS/user with 8 threads we achieved 5241 IOPS and passed the performance requirements. (Headroom of 9 IOPS only) However, when we increased the number of threads in these tests with the same storage configuration one or more data sets failed with high latency (>20 ms). (For example, when we tested the same storage configuration with 5000 users with 0.5 IOPS/users with 14 threads, we achieved 7149 IOPS, however 1 storage group failed with high latency (>19ms). Note: Further JetStress testing must be conducted to find the IOPS threshold (saturation point) by increasing the number of threads. SAN HQ and JetStress results for 1000 and IOPS/user on RAID10 configurations are shown below. Page 30

31 RAID 50 TESTING For RAID50 configurations the following table summarizes the performance results that we achieved as compared to the maximum suggested IOPS for this configuration from the EqualLogic sizing tool. Table 10: RAID50 Test Results using JetStress Users 1000 IOPS/User 5000 IOPS/User RAID type Mailbox size # of threads IOPS Achieved Suggested IOPS from EqualLogic Exchange 2007 Sizing Tool RAID RAID As you can see from the above table, using JetStress on 2 PS5000XV units with RAID50: For 1000 users with 0.5 IOPS/user with 14 threads we achieved 3918 IOPS and passed the performance requirements. For 5000 users with 0.5 IOPS/user with 8 threads we achieved 4339 IOPS and passed the performance requirements. In the above RAID50 tests we achieved more IOPS than the max IOPS suggested by the EqualLogic sizing tool and still passed. This may be for two reasons: 1) JetStress recommends that we test with databases that utilize 80% of the usable capacity to get the most realistic performance numbers. However, our resting used only 16% of usable capacity for 1000 users and 47% of usable capacity for 5000 users. As a result, high IOPS achieved in RAID50 testing may have been caused by short stroking of drives. 2) The EqualLogic Exchange 2007 sizing tool has some built-in headroom for deteriorated conditions. For example, in a failure scenario when a RAID rebuild is required, the test may not pass with such high IOPS. So it is desirable to stay within the limits of the suggested max IOPS by the EqualLogic Exchange 2007 Sizing Tool. SAN HQ and JetStress results for 1000 and IOPS/user on RAID50 configurations are shown below. Note: Further JetStress testing must be conducted to find the IOPS threshold (saturation point) by increasing the number of threads. LoadGen tests conducted using 2 PS5000XV and 1 PS5000E for 1000 and 5000 users under various user profiles also passed. Page 31

32 For 1000 users RAID10 JetStress 0.5 IOPS/user, the SAN HQ and JetStress results are shown below. Figure 11: SAN HQ results from 1000-user 0.5 IOPS/user Page 32

33 Figure 12: JetStress Results from 1000-user 0.5 IOPS/user Page 33

34 For 5000 users RAID10 JetStress 0.5 IOPS/user, the SAN HQ and JetStress results are shown below. Figure 13: SAN HQ results for 5000 users 0.5 IOPS/user Page 34

35 Figure 14: JetStress results for 5000 users 0.5 IOPS/user Page 35

36 For 1000 users RAID50 JetStress 0.5 IOPS/user, the SAN HQ and JetStress results are shown below. Figure 15: 15: SAN HQ results for 1000 users 0.5 IOPS/user Page 36

37 Microsoft Exchange Server Jetstress Performance Test Result Report Test Summary Overall Test Result Pass Machine Name Test Description Test Start Time Test End Time EXCHSRV1-DR 2/28/ :09:22 AM 2/28/2009 2:13:13 AM Jetstress Version Ese Version Operating System Windows Server (R) 2008 Enterprise Service Pack 1 ( ) Performance Log Database Sizing and Throughput Achieved I/O per Second Target I/O per Second 500 C:\Program Files\Exchange Jetstress\Performance_2009_2_28_0_9_31.blg C:\Program Files\Exchange Jetstress\DBChecksum_2009_2_28_2_13_13.blg Initial database size Final database size Database files (count) 4 Jetstress System Parameters Thread count Log buffers 9000 Minimum database cache MB Maximum database cache MB Insert operations 40% Delete operations 30% 14 (per-storage group) Figure 16: 15: JetStress results for 1000 users 0.5 IOPS/user Page 37

38 For 5000 users RAID50 JetStress 0.5 IOPS/user, the SAN HQ and JetStress results are shown below. Figure 17: SAN HQ results for 5000 users 0.5 IOPS/user Page 38