1 Archiving Microsoft Exchange Mailboxes on Hitachi Content Platform using Storage Adapter for Symantec Enterprise Vault Lab Validation Report By Praveen Javehrani March 2012 Month Year
Feedback Hitachi Data Systems welcomes your feedback. Please share your thoughts by sending an email message to SolutionLab@hds.com. Be sure to include the title of this white paper in your email message. 1
Table of Contents Product Features... 3 Hitachi Content Platform... 3 Storage Adapter for Symantec Enterprise Vault... 3 Symantec Enterprise Vault... 4 Test Environment Configuration... 4 Storage Configuration... 5 Test Methodology... 6 Workload Generation: Load Generator... 7 Analysis... 7 Observations... 7 Recommendations... 8 Appendix Test Results... 9 Test Results for Performance Baselines... 9 Test Results for Determining Optimal Archival Rate... 10 Additional Information... 12 For More Information... 13 2
Archiving Microsoft Exchange Mailboxes on Hitachi Content Platform using Storage Adapter for Symantec Enterprise Vault Lab Validation Report This document presents performance baseline results conducted at Hitachi Data Systems labs. It establishes the performance of Symantec Corporation Enterprise Vault to archive Microsoft Exchange mailboxes on Hitachi Content Platform. The archive process used Storage Adapter for Symantec Enterprise Vault. Hitachi Data Systems worked with Symantec to produce a new storage partition interface. This new interface, Storage Adapter for Symantec Enterprise Vault (storage adapter), helps you to take advantage of your investment in Hitachi Content Platform and Symantec Enterprise Vault. Exchange mailbox archiving tests determined the items archival rate attained using storage adapter. The results demonstrated the performance capability of applications using storage adapter, such as how Enterprise Vault uses a REST/HTTP interface that delivers enhanced functionality and better performance than the legacy CIFS/NFS protocol. Product Features Hitachi Content Platform Hitachi Content Platform is a distributed, fixed-content, data storage system from Hitachi Data Systems. It provides a scalable repository to accommodate all types of fixed-content data. This includes simple text files, medical images, and very large database images. A fixed-content storage system is well suited for non-transactional data. This type of data seldom changes, if ever, and must be preserved for extended periods. Content Platform uses write-once, read-many (WORM) storage technology. It also uses a variety of policies and services to ensure the integrity of the stored data and the efficient use of storage capacity. It provides easy access to the repository for adding, retrieving, deleting, or shredding the stored data. For more information, see Hitachi Content Platform. Storage Adapter for Symantec Enterprise Vault Hitachi Content Platform and Symantec Enterprise Vault offer a complete, proven solution for email and file archiving. The solution is highly scalable and provides policy-based retention, quick search and retrieval, and data destruction. At the core of this solution is Storage Adapter for Symantec Enterprise Vault by Hitachi (storage adapter). Hitachi Data Systems developed the adapter to use the new storage streamer API in Enterprise Vault when used with Content Platform. 3
Storage adapter connects to Hitachi Content Platform over its enhanced REST/HTTP interface to provide a high level of function and performance. The high performance means archiving more objects per hour. This means a single Hitachi Content Platform can support a very large environment. If replicating Content Platform, storage adapter verifies objects are present on both systems before marking an object safe for placeholder creation. Symantec Enterprise Vault Symantec Enterprise Vault is an application that operates on Microsoft Windows. It enables storing messaging and file system data automatically in centrally held archives. Using Enterprise Vault clients, you can retrieve selected items when required. Enterprise Vault archives many types of data. This ranges from items in Microsoft Exchange user and journal mailboxes, items in IBM Lotus Domino mail files and databases, files held on network file servers, and Microsoft SharePoint servers, to name a few. For more information, see Symantec Enterprise Vault. Test Environment Configuration Table 1 and Table 2 show the hardware and software configuration used for testing. Table 1. Hardware Configuration Hardware Dell PowerEdge R610 Rack Server 3 separate servers Microsoft Windows Server 2008 R2 Enterprise SP1 x64 used on all three servers Specification and Use Microsoft Exchanged Server Dual Quad core Intel Xeon E5420 at 2.5 GHz, 8 GB RAM Enterprise Vault and SQL Server Dual Quad core Intel Xeon E5504 @ 2.00 GHz, 12 GB RAM Load Generator Dual Quad core Intel Xeon E5420 @ 2.5 GHz, 8 GB RAM Hitachi Adaptable Modular Storage 2500 Hitachi Content Platform 300 Dual Controller RAID 5 4 nodes Version 4.1.0.105 4
Table 2. Software Configuration Description Version Microsoft Exchange Server 2010 x64 Microsoft Exchange Outlook Client 2010 Symantec Enterprise Vault 9.0.2 Microsoft SQL Server 2008 Microsoft Workload Generator 2010 Storage Configuration Figure 1 shows the high-level system architecture. Figure 1 5
Test Methodology Executing two distinct set of performance tests determined the Exchange archival rate using storage adapter with Symantec Enterprise Vault and Hitachi Content Platform. Table 3. Test Methodology Steps Test Set Description Comments 1 Determine the baseline numbers for vault store settings of immediate after archiving and after backup with the preservation of safety copies for 1000 mailboxes with 5 concurrent Exchange connections (default setting) and storage archive process set to 25. 2 Determine the impact on the archival rate by increasing the Exchange concurrent connection setting in Enterprise Vault to the 15 (maximum setting). These tests created a stub file for the original safety copy upon deletion, which is a reference link to the original item. The Enterprise Vault creates this stub asynchronously to the archive process, when specifying the After Backup vault store property. There are intermediate test results for stepping the number of connections from 5 to 15. The tests used the following tasks: User mailboxes creation Enabling and provisioning of mailboxes Running mailbox archiving task VBS script that gives the archival rate per minute Collecting, monitoring, and analyzing performance counters using perfmon Note The Microsoft SQL Server and Enterprise Vault reside on the same server. Most users run these programs on separate physical servers. 6
Workload Generation: Load Generator Microsoft Load Generator benchmarking tool created and simulated the user mailboxes on the Exchange server. This tool creates varied types of workloads for these tests, including the following: Number of email messages Email message size Number of attachments to an email message and the attachment size Appointments Contacts Deletions There were baseline tests conducted. These tests were for 1000 mailboxes with the configuration in Table 4 using different scenarios. Table 4. Microsoft Load Generator Settings Total Number of Messages per Inbox 50 Average Maibox Size 72KB Frequency of Attachments 0.05 attachments/second Analysis This analysis includes observations from the test results and recommendations based on those results. Observations These are observations from the testing: There is a significant increase in the archival rate after scaling up the number of Exchange connections. There is a significant increase in the CPU utilization on the Enterprise Vault server after scaling the Microsoft Exchange sever connections up from 5 to 15. Determine the number of Exchange connections to set based on the number of physical server cores. There were few spikes of CPU utilization spikes of 100% on the Enterprise Vault server when the setting for Exchange connections was 10 or more. This could bog down the system momentarily. The high utilization on Exchange server should not affect users by scheduling the archival task at a time of low user usage. Users could experience some slowness if the Microsoft Message Queue Server queue value goes beyond 1500. Using the after backup property, the most common property for removal of safety copies, has a higher archived items per hour than the immediately after archive property. 7
Recommendations These are recommendations resulting from these tests: If the queue for Microsoft Message Queue Server increases over 1500, change the registry value for each Value in Table 5. Table 5. Registry Changes Value Key Content ThrottleLowerThreshold ThrottleUpperThreshold HKEY_LOCAL_MACHINE\SOFTWARE\ KVS\Enterprise Vault\Storage HKEY_LOCAL_MACHINE\SOFTWARE\ KVS \Enterprise Vault\Storage DWORD value set to an integer value. Default is 1500. Raise this to 5000. DWORD value set to an integer value. Default is 4500. Raise this to 10000. Have Microsoft Message Queue reside on a different physical drive than the system drive. (Symantec also recommends this practice.) Increase the number of threads for the storage archive process to achieve a higher archival rate. Increase the number of connections to the Microsoft Exchange server used by the archive process to achieve a higher archival rate. Only make these changes during periods of low user usage, since this change may affect user response time. Even though Symantec Enterprise Vault server and Microsoft SQL Server resided on the same server for these tests, use two separate physical servers in a production environment for Enterprise Vault and SQL Server to free some cycles for EV to improve performance. Follow the network bandwidth guidelines in the Symantec Performance Manual. 8
Appendix Test Results The test scenarios described in this paper do not cover varied Exchange workloads. They only give a good insight on what to anticipate with a similar workload and hardware configuration. Test Results for Performance Baselines Table 6 has the baseline results for 1000 mailboxes using a configuration of five concurrent Microsoft Exchange connections and storage archive process set to 25: For the Immediately After Archiving setting, Symantec Enterprise Vault removes all original items, also known as safety copies, immediately after archiving each item to Hitachi Content Platform. However, before deleting the safety copies, Enterprise Vault creates a placeholder with a link to the original item. Depending on configuration, there may not be a replica ( secondary copy). Content Platform reports back to Enterprise Vault the item is safe based on the existence of the file on the primary (only) Content Platform. For the After Backup setting, Symantec Enterprise Vault does not remove safety copies until fully replicating the partition that contains the archived items, presuming there is a replica Hitachi Content Platform. In the absence of a replica Content Platform, items are considered safe upon successful archive to Content Platform. For more information, see Changes to Backup procedures in Enterprise Vault 8 on the Symantec website. The immediately after archive rate was five concurrent Exchange connections with an average archival rate of 504 items/minute (around 30,000 items/hour). Table 6 has the baseline test results for the archival rate per hour and CPU utilization. Table 6. Baseline for 1000 Mailboxes Timing Avg Items/Minute Total Items/Hour CPU Utilization Average Maximum Immediately After Archiving 516 30,982 46 81 After Backup 704 42,226 51 61 A graph that compares the after backup scenario archival rate to the immediately after archive policy scenario for the removal of safety copies is graph in Figure 2. This main difference in the archival rate results from when Enterprise Vault deletes the messages. 9
Figure 2 Test Results for Determining Optimal Archival Rate These tests gauged the performance effect on Enterprise Vault server and Exchange server by changing the number of concurrent Exchange connections. Table 7 shows the affect the number of concurrent Exchange connections has on the archival rate per minute and per hour. Table 7. Archival Rate for 1000 Mailboxes at 50 Messages Each with 72KB Average Exchange Concurrent Connections Immediate After Archiving Archival Rate/Minute Archival Rate/Hour After Backup Archival Rate/Minute Archival Rate/Hour 5 504 30,247 704 42,240 7 695 41,700 1,021 61,260 8 862 51,720 1,042 62,520 10 910 54,600 1,163 69,780 15 1,064 63,840 1,388 83,280 10
Figure 3 and Figure 4 show the affect of the number of Exchange connections on the archival rate per hour and per minute. Figure 3 Figure 4 Table 8 shows the affect the number of concurrent Exchange connections has on the percentage CPU utilization for immediate after archive and after backup. 11
Table 8. Enterprise Vault CPU Utilization Concurrent Exchange Connections EV Percent CPU Utilization: Immediate after Archive Percent CPU Utilization: After Backup 5 45 49 7 60 69 9 72 81 10 74 83 15 84 92 Figure 5 graphs the affect the number of Exchange connections has on CPU utilization. Figure 5 Additional Information Symantec Symantec Enterprise Vault 9.0 Installing and Configuring Enterprise Vault 9.0 Performance Guide Symantec Enterprise Vault 9.0 Setting up Exchange Server Archiving A5 MSMQ consistently has a number of items left in the queue and the number of items in the queue grows at a steady rate Enterprise Vault 9.0 Manuals Hitachi Data Systems Hitachi Content Platform and Symantec Enterprise Vault 12
For More Information Hitachi Data Systems Global Services offers experienced storage consultants, proven methodologies and a comprehensive services portfolio to assist you in implementing Hitachi products and solutions in your environment. For more information, see the Hitachi Data Systems Global Services website. Live and recorded product demonstrations are available for many Hitachi products. To schedule a live demonstration, contact a sales representative. To view a recorded demonstration, see the Hitachi Data Systems Corporate Resources website. Click the Product Demos tab for a list of available recorded demonstrations. Hitachi Data Systems Academy provides best in class training on Hitachi products, technology, solutions, and certifications. Hitachi Data Systems Academy delivers on-demand web-based training (WBT), classroom-based instructor-led training (ILT), and virtual instructor-led training (vilt) courses. For more information, see the Hitachi Data Systems Academy website. For more information about Hitachi products and services, contact your sales representative or channel partner. Otherwise, visit the Hitachi Data Systems website. 13
Corporate Headquarters 750 Central Expressway, Santa Clara, California 95050-2627 USA www.hds.com Regional Contact Information Americas: +1 408 970 1000 or info@hds.com Europe, Middle East and Africa: +44 (0) 1753 618000 or info.emea@hds.com Asia Pacific: +852 3189 7900 or hds.marketing.apac@hds.com Hitachi is a registered trademark of Hitachi, Ltd., in the United States and other countries. Hitachi Data Systems is a registered trademark and service mark of Hitachi, Ltd., in the United States and other countries. All other trademarks, service marks and company names mentioned in this document are properties of their respective owners. Notice: This document is for informational purposes only, and does not set forth any warranty, expressed or implied, concerning any equipment or service offered or to be offered by Hitachi Data Systems Corporation Hitachi Data Systems Corporation 2012. All Rights Reserved. AS-123-02 March 2012 14