ACCELERATING MICROSOFT EXCHANGE 2010 PERFORMANCE WITH EMC XTREMCACHE

Transcription

1 White paper ACCELERATING MICROSOFT EXCHANGE 2010 PERFORMANCE WITH EMC XTREMCACHE EMC XtremSF, EMC XtremCache, EMC VNX, VMware vsphere, Microsoft Exchange 2010 EMC Solutions Abstract This white paper demonstrates how EMC XtremCache can accelerate Microsoft Exchange 2010 performance on EMC VNX storage and VMware virtual infrastructure. The paper describes the XtremCache design, deployment, best practices, and performance results. December 2013

2 Copyright 2013 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All other trademarks used herein are the property of their respective owners. Part Number: H Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 2

3 Table of contents Executive Summary... 5 Business case... 5 Solution overview... 5 Key findings... 5 Introduction... 6 Purpose... 6 Scope... 6 Audience... 6 Terminology... 6 Technology overview... 8 EMC XtremSF and EMC XtremCache... 8 Server-side flash caching for maximum speed... 8 Write-through caching to the array for total protection... 8 Application agnostic... 9 Integrated with VMware vsphere... 9 Minimum impact on system resources... 9 EMC VNX family of unified storage platforms... 9 VMware vsphere EMC PowerPath/VE... 9 EMC Virtual Storage Integrator plug-in for vsphere Microsoft Exchange Server Solution architecture Architecture overview Solution hardware Solution software Solution design Simulated Exchange workload Exchange storage design Exchange 2010 building block design method Mailbox server virtual machine building block details Exchange storage pool details Exchange virtual machine disks VSphere server design for Exchange HA Configuring XtremCache with Exchange 2010 and VMware vsphere Planning the XtremCache design deployment Determining the XtremSF card size Determining the Exchange workload characteristics Configuring XtremCache with VMware Configuring XtremCache Configuring the XtremCache device for Exchange virtual machines Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 3

4 Viewing the XtremCache disk in Windows Disk Manager Configuring the XtremCache device with data deduplication Migrating Exchange virtual machine with XtremCache device Performing an automated migration Recovering Exchange data from a snapshot Monitoring XtremCache performance Solution validation and performance results About performance results Exchange performance validation tools Exchange Jetstress Exchange Load Generator Monitoring Exchange performance Monitoring the Exchange server s health Solution validation with Exchange Jetstress Performance results with Jetstress servers XtremCache acceleration test results Storage array performance results with Jetstress Solution validation with Exchange LoadGen Analyzing the LoadGen test results LoadGen test results summary Performance with XtremCache data deduplication Deduplication test results summary Conclusion Summary Findings References White papers Product documentation Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 4

5 Executive Summary Business case IT organizations face application performance challenges caused by an imbalance between the processing power of servers and the access time of storage disks. Server processing power continues to advance, doubling every 18 months, yet disk drive throughput remains the same. This causes a bottleneck in the input/output (I/O) stack whereby the server and the application have the capacity to process more I/O than disk drives can deliver. This is referred to as the I/O gap. Adding flash drives to the array has helped to close this gap by increasing performance by an order-of-magnitude. Now, server-based PCIe flash technology is accelerating I/O performance beyond flash drives by yet another order of magnitude. EMC XtremSF is a server flash-caching solution that reduces latency and increases throughput to improve application performance by leveraging intelligent software and XtremSF flash technology. EMC XtremCache is the software that accelerates reads and protects data by using a write-through cache to the networked storage to deliver persistent high availability and disaster recovery. Coupled with array-based EMC Fully Automated Storage Tiering (FAST ) software, XtremCache creates the most efficient and intelligent I/O path from the application to the datastore. The result is a networked infrastructure that is dynamically optimized for performance, intelligence, and protection for both physical and virtual environments. Solution overview Key findings This solution evaluates and demonstrates how XtremCache can be deployed with Microsoft Exchange 2010 in a VMware virtual infrastructure to accelerate Exchange 2010 performance for extreme user workloads. This solution provides: Design recommendations for XtremCache with Exchange 2010 on VMware vsphere Benefits of using XtremCache for virtualized Exchange deployments on vsphere with EMC VNX storage Performance benefits for very heavy Exchange workloads The key findings in this solution show that XtremCache: Improves Exchange performance by reducing read latencies and offloading reads to the back-end storage Helps to maximize I/O throughput for Exchange workloads without changing or adding any additional storage resources Reduces bandwidth requirements, using deduplication features, offloading write processing from the Exchange back-end storage Has little impact on system resources such as CPU and memory Is easy to manage and monitor in a virtualized environment through its integration with the EMC Virtual Storage Integrator (VSI) plug-in for VMware. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 5

6 Introduction Purpose The purpose of this white paper is to describe the ability of XtremCache to accelerate the performance of Exchange Server 2010 deployed on a vsphere virtual infrastructure with VNX storage. Note: Any mention of VFCache or XtremSW Cache in this white paper refers to XtremCache. Scope This white paper: Provides an overview of XtremCache technology Describes how to design and use XtremCache with Exchange 2010 workloads Showcases the ability of XtremCache to accelerate the performance of heavy Exchange 2010 workloads with little impact on the system resources Demonstrates how easy it is to configure and manage XtremCache Note: This document was accurate as of the time of publication. As new software is released and information is added, we may release new versions of this document to the EMC website. Check the website to ensure that you are using the latest version of this document. Audience Terminology This document is intended for EMC customers, partners, and employees who are interested in increasing the performance of their heavy user workload applications such as Exchange We assume that you are familiar with EMC unified storage systems and the Microsoft Exchange messaging application. Table 1 defines several terms used in this document. Table 1. Terminology used in this white paper Term Background Database Maintenance (BDM) Building-block Database Availability Group (DAG) Service-level agreement (SLA) Definition The process of the Exchange 2010 database maintenance that involves performing a checksum procedure on both the active and passive database copies. A design method that represents the required amount of disk and server resources required to support a specified number of Exchange 2010 users. An Exchange 2010 base component that provides high availability and site resilience. A DAG can contain up to 16 Mailbox servers. The servers host a set of databases that provide automatic database-level recovery from failures affecting individual databases. Any server in a DAG can host a copy of a mailbox database from any other server in the DAG. Policies defined by data center administrators that set the levels of availability, serviceability, performance, and operation of an organization s environment. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 6

7 Term Storage pool Thick LUN Thin LUN XtremSF (aka VFCache PCI flash card) XtremCache Warm-up time Definition A collection of drives with a specified RAID type. A storage pool can be single tier (homogeneous with one drive type) or multitier (heterogeneous with mixed drive types) type. A type of pool LUN in which the physical space allocated is equal to the user capacity seen by the host server A type of pool LUN where the physical space allocated can be less than the user capacity seen by the host server EMC XtremSF is a PCIe flash card deployed in the server to dramatically improve application performance by reducing latency and accelerating throughput. XtremSF can be used as a local storage device to accelerate read and write performance, or can be used in conjunction with server flash caching software, XtremCache, for accelerated read performance with data protection. XtremCache is the software that can be used with server flash hardware, XtremSF, to accelerate read performance with data protection. Warm-up time consists mostly of promotion operations into XtremCache. This occurs if the XtremSF PCIe card was recently installed and empty. This also happens when the working data set of the application has significantly changed and the current XtremCache data is no longer being referenced. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 7

8 Technology overview This section provides an overview of the technologies used in this solution: EMC XtremSF and EMC XtremCache EMC VNX family of unified storage platforms VMware vsphere 5 EMC PowerPath/VE EMC Virtual Storage Integrator plug-in for vsphere Microsoft Exchange Server 2010 EMC XtremSF and EMC XtremCache EMC XtremSF is PCIe flash hardware deployed in the server to dramatically improve application performance by reducing latency and accelerating throughput. You can use XtremSF as a local storage device to accelerate read and write performance for high-transactional, high-performance applications. XtremSF PCIe flash is available in 550 GB, 700 GB, 1.4 TB, and 2.2 TB emlc flash capacities, and 350 GB and 700 GB SLC flash capacities. You can use XtremSF in conjunction with EMC XtremCache, intelligent server flashcaching software. Combining XtremSF with XtremCache provides accelerated read performance with protection. While accelerating performance, XtremCache simultaneously delivers enterprise-class protection of mission-critical application data. Data in the cache is simply a copy of data that is already stored on the array, while the master copy is maintained by the advanced data services that EMC networked storage provides, including high availability, end-to-end integrity, reliability, and disaster recovery. XtremCache uses a write-through algorithm that ensures newly written data persists to the networked storage array, such as EMC Symmetrix VMAX or VNX. If the server fails, the data remains protected and accessible on the array. Server-side flash caching for maximum speed The XtremCache software caches the most frequently referenced data on the serverbased XtremSF card, thereby putting the data closer to the application. The XtremCache caching optimization automatically adapts to changing workloads by determining which data is most frequently referenced and promoting it to the server flash cache. This means that the hottest data (most active data) is automatically cached on the XtremSF card in the server for faster access. As an added benefit, XtremCache offloads the read traffic from the storage array, which allows it to allocate greater processing power to other applications. While one application is accelerated with XtremCache, the array s performance for other applications is maintained or even slightly enhanced. Write-through caching to the array for total protection By configuring write-through caching to the array for total protection, XtremCache accelerates reads and protects data by using write-through caching to the storage to deliver persistent high availability, integrity, and disaster recovery. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 8

9 Application agnostic XtremCache is transparent to applications, so no rewriting, retesting, or recertification is required to deploy XtremCache in the environment. Integrated with VMware vsphere XtremCache enhances both virtualized and physical environments. When it is integrated with the VSI plug-in, you can easily manage and monitor XtremCache. Minimum impact on system resources XtremCache is designed to minimize CPU overhead in the server by offloading flash management operations from the host CPU to the XtremSF card. XtremCache creates the most efficient and intelligent I/O path from the application to the datastore. This results in an infrastructure that is dynamically optimized for performance and protection in both physical and virtual environments. EMC VNX family of unified storage platforms The EMC VNX family delivers innovation and enterprise capabilities for file, block, and object storage in a scalable, easy-to-use solution. This next-generation storage platform combines powerful and flexible hardware with advanced efficiency, management, and protection software to meet the demanding needs of today s enterprises. All of this is available in a choice of systems ranging from entry-level solutions to high-performance, petabyte-capacity configurations servicing the most demanding application requirements. The VNX family includes the EMC VNXe series, purposebuilt for the IT manager in entry-level environments, and the VNX series, designed to meet the high-performance, high-scalability requirements of midsize and large enterprises. VMware vsphere 5 EMC PowerPath/VE VMware vsphere 5 is the virtual data center operating system. It transforms the IT infrastructure into the most efficient, shared, and on-demand utility, with built-in availability, scalability, and security services for all applications and simple, proactive automated management. EMC PowerPath /VE provides intelligent, high-performance path management with path failover and load balancing optimized for EMC and selected third-party storage systems. PowerPath/VE supports multiple paths between a vsphere host and an external storage device. Having multiple paths enables the vsphere host to access a storage device even if a specific path is unavailable. Multiple paths can also share the I/O traffic to a storage device. PowerPath/VE is particularly beneficial in highly available environments because it can prevent operational interruptions and downtime. The PowerPath/VE path failover capability avoids host failure by maintaining uninterrupted application support on the host in the event of a path failure (if another path is available). PowerPath/VE works with VMware ESX/ESXi as a multipath plug-in (MPP) that provides path management to hosts. It is installed as a kernel module on the vsphere host and plugs in to the vsphere I/O stack framework to provide the advanced multipathing capabilities of PowerPath/VE including dynamic load balancing and automatic failover to the vsphere hosts. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 9

10 For this solution, we installed PowerPath/VE on all ESXi hosts that house Exchange virtual machines. EMC Virtual Storage Integrator plug-in for vsphere Microsoft Exchange Server 2010 EMC Virtual Storage Integrator (VSI) for VMware vsphere is a free plug-in for VMware vsphere Client that provides a single management interface for managing EMC storage within the vsphere environment. You can add and remove features from VSI independently, which provides flexibility for customizing VSI user environments. You manage VSI features by using the VSI Feature Manager. VSI provides a central location from which to manage the EMC infrastructure within VMware vcenter, allowing you to update each feature independently and introduce new features rapidly in response to changing requirements. Examples of features available for VSI include XtremCache, Storage Viewer, Path Management, Storage Pool Management, Symmetrix storage replication adapter utilities, and Unified Storage Management. Microsoft Exchange is an enterprise mail and collaboration system that can be deployed on-premises, in the cloud, or in hybrid configurations. It provides , a calendar and contacts on your PC, and a phone. Exchange 2010 also supports a variety of browsers, including Internet Explorer, Firefox, Safari, and Chrome, and allows you to work and collaborate no matter where you are. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 10

11 Solution architecture This section describes the solution s architecture and its hardware and software components: Architecture overview Solution hardware Solution software Architecture overview To validate Exchange 2010 performance with XtremCache, we virtualized Exchange 2010 on vsphere and used VNX storage for hosting all Exchange data. We deployed Exchange 2010 in a Database Availability Group (DAG) configuration with two database copies. We used the EMC Proven building-block design methodology to size and deploy Exchange 2010 on EMC storage. We also used EMC, Microsoft, and VMware best practices and guidelines to deploy Exchange 2010 in virtual environments. For performance acceleration, XtremSF cards were installed into vsphere servers hosting Exchange Mailbox server virtual machines. Figure 1 shows a high-level configuration Exchange 2010 solution with XtremCache on vsphere and VNX storage that we deployed and validated. Figure 1. Virtualized Exchange 2010 solution with EMC XtremCache Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 11

12 Solution hardware Table 2 lists the hardware components used in the test environment. Table 2. Hardware components Item Storage platform Storage connectivity to host Description EMC VNX5700 block (36 GB system cache) 8 Gb/s Fibre Channel (FC) Physical hypervisor hosts 2 Quad eight-core Intel Xeon x7560 processors with 2.27 GHz CPU, 192 GB RAM (Hyper threading enabled) for hosting Exchange 2010 Mailbox server virtual machines 2 Quad six-core Intel Xeon x7460 processors with 2.66 GHz CPU, 192 GB RAM (Hyper threading enabled) for hosting CAS and HUB Exchange 2010 virtual machines and other infrastructure virtual machines Host bus adapter (HBA) type XtremSF PCIe card 8 GB dual port HBAs per vsphere host (2 per ESXi host) Model Number: PCIEHHS-3XXM2 300 GB Solution software Table 3 lists software components used in the test environment. Table 3. Software components Item EMC VNX5700 operating environment Multipathing software Description Operating environment for block PowerPath /VE 5.7 for VMware vsphere Hypervisor operating system VMware vsphere 5.0 Virtual machines operating system Exchange Server Exchange performance validation tools Windows Server 2008 R2 Enterprise SP1 Exchange Server 2010 Enterprise SP2 Version 14.2 (Build 247.5) Exchange Jetstress 2010 (64 bit), version Exchange Load Generator 2010 (64 bit) version EMC XtremCache Version Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 12

13 Solution design This section presents the configuration details of the solution s test environment including: Simulated Exchange workload Exchange storage design Exchange 2010 building block design method Mailbox server virtual machine building block details Exchange storage pool details Exchange virtual machine disks VSphere server design for Exchange HA For this solution deployment we have created a simulated customer Exchange 2010 environment that represents a medium to large-enterprise configuration with heavy user workload and large mailboxes. Simulated Exchange workload Table 4 details the simulated Exchange 2010 workload profile created in this solution. Item Table 4. Simulated Exchange 2010 profile used in the test environment Number of Exchange 2010 users simulated Value 15,000 users User profile (in Mailbox Resiliency configuration) Mailbox size Number of DAGs and database copies Number of hypervisor hosts hosting Exchange Mailbox server virtual machines Number of Exchange Mailbox server virtual machines Number of Exchange users per Mailbox server virtual machine Database read/write ratio (in Mailbox Resiliency configuration) 150 messages/user/day 1.5 GB 1 DAG with 2 copies 2 6 5,000 users in a switchover condition (2,500 active/2,500 passive) 3:2 Background Database Maintenance (BDM) Enabled 24 x 7 Number of databases per Exchange Server 6 Number of users per database 834 LUN sizes Database LUNs TB, Log LUNs 90 GB Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 13

14 Item Total Exchange dataset used for performance testing Value 23.4 TB Exchange storage design To support the Exchange Server 2010 user profile and mailbox size requirements for this solution (see Table 4 for details), we used an EMC VNX5700 storage array with 2 TB 7.2 k rpm NL-SAS drives to house the databases and logs. To simplify design and deployment, we designed and configured a Mailbox server building block with the appropriate number of drives to support the required performance and capacity for each Mailbox server virtual machine in this solution. Exchange 2010 building block design method EMC provides methodologies to easily and efficiently deploy Microsoft Exchange on the EMC VNX family of storage arrays. For this solution, we used an Exchange 2010 building block design method. A Mailbox server building block represents the amount of storage (I/O and capacity) and server compute (CPU and memory) resources required to support a specific number of Exchange Server 2010 users. The amount of required resources is derived from a specific user profile type (messages sent/received, IOPS per user, and mailbox size). Using the building block approach simplifies the design and implementation of Exchange Server With the initial building block designed, we can easily reproduce it to support the required number of users in your enterprise. By using this approach, EMC customers can now create their own building blocks based on their company s specific Exchange environment requirements. This approach is helpful when future growth is expected because it makes Exchange environment expansion simple and straightforward. EMC s best practices involving the building block approach for Exchange Server design have proven to be successful in many customer implementations. For more details on using the EMC building-block methodology for Exchange 2010 deployments, refer to Microsoft Exchange 2010 Storage Best Practices and Design Guidelines for EMC Storage. Mailbox server virtual machine building block details For this solution, the Exchange Mailbox server virtual machine design is based on a building block for 5,000 users. The storage, CPU, and memory resources for this building block are derived from the design attributes presented in Table 5. For additional Microsoft guidelines considered for this solution s design, refer to the Microsoft TechNet Library topic Mailbox Server Processor Capacity Planning. Table 5 summarizes the Exchange Mailbox server virtual machine building block for 5,000 users with a 150-message profile and a 1.5 GB mailbox quota. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 14

15 Table 5. Mailbox server building block summary Users per Exchange Mailbox server virtual machine Disks per Exchange Mailbox server virtual machine 5, (16 for databases and 2 for logs) vcpus per Exchange Mailbox server virtual machine 4 CPUs 32 GB Memory per Exchange Mailbox server virtual machine Table 6 shows storage design details for a Mailbox server building block with 5,000 users. Table 6. Storage design details for a 5,000-user Mailbox server building block Attribute Number of users per server User profile Mailbox size Disk type Storage configuration/raid type Databases per server/users per database LUNs/LUN sizes Details 5,000 (in a switchover condition) 150 messages sent/received (0.15 IOPS) 1.5 GB 2 TB NL-SAS, 7.2 k rpm 18 disks per building block for Exchange data: 16 disks in RAID 1/0 for databases 2 disks in a RAID 1/0 for logs 6 databases per server, 834 users per database 13 LUNs per Mailbox server virtual machine: 6 database LUNs (1.6 TB) (RDM/P) 6 log LUNs (90 GB) (RDM/P) 1 LUN for virtual machine OS (200 GB) (VMFS) Exchange storage pool details To simplify the Exchange storage design and provide optimal performance for the Exchange application and end user experience, we configured the VNX storage using storage pools. We also designed each storage pool to ensure that it will not contain two copies of the same database to provide high availability for Exchange virtual machines. As best practice, we separated the Exchange databases from their transactional logs onto separate disks and deployed four storage pools for the Exchange data two pools for databases and two pools for logs. For the Exchange databases, we deployed two RAID 1/0 storage pools with 48 2 TB 7,200 rpm NL-SAS disks per pool for each vsphere server hosting three Exchange Mailbox Server Role virtual machines, that is three building blocks per pool. Each storage pool provides almost 43 TB of usable capacity and meets the IOPS and mailbox capacity requirements to support three mailbox server building blocks 15,000 users at 0.15 IOPS and a 1.5 GB mailbox. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 15

16 Figure 2 shows VNX storage pool information used for this solution from EMC Unisphere. It provides details about each storage pool capacity, disk type, and other information. Figure 2. EMC Unisphere Storage pools created for this solution From each database storage pool, we configured TB database LUNs six LUNs per mailbox server to accommodate the database space requirements. All LUNs from each database pool were distributed between three Exchange Mailbox server virtual machines running on the vsphere host. We used a similar design for configuring storage pools for Exchange transactional logs by deploying two RAID 1/0 storage pools, each with four 2 TB 7,200 rpm NL-SAS disks, with each pool supporting three Mailbox server building blocks. From each logs storage pool, we configured GB LUNs six LUNs per mailbox server. The LUNs are distributed between three Exchange Mailbox server virtual machines configured on the vsphere host. For Exchange virtual machine OS volumes, we created a separate RAID5 storage pool with 10 k rpm SAS drives. Figure 3 shows the storage configuration for Exchange databases used for the Mailbox server virtual machines in this solution. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 16

17 Figure 3. Storage design for virtualized Exchange Mailbox servers Exchange virtual machine disks We configured the disk volumes for the database and log LUNs on each Exchange virtual machine as RDMs in physical compatibility mode (RDM/P) for optimal performance and replication ability. Note: RDMs are required if you want the ability to use the hardware that VSS provides. For example, EMC AppSync uses and extends the VSS framework to initiate application-level protection and recovery using VNX Snapshot s replication technology. Figure 4 shows the disks configured for one of the Exchange mailbox server virtual machines. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 17

18 Figure 4. Exchange virtual machine disk configuration VSphere server design for Exchange HA To provide high availability for Exchange virtual machines, we configured two vsphere hosts as a high-availability (HA) cluster. Each physical server had enough CPU and memory resources to sustain failure or maintenance of another cluster node. During vsphere server maintenance, all Exchange virtual machines hosted on this server can be migrated to another vsphere cluster node if required. We provisioned client access and hub transport role virtual machines on another vsphere host. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 18

19 Configuring XtremCache with Exchange 2010 and VMware vsphere This section describes how to configure XtremCache with Exchange 2010 deployed on VMware: Planning the XtremCache design deployment Determining the XtremSF card size Determining the Exchange workload characteristics Configuring XtremCache with VMware Configuring XtremCache Configuring the XtremCache device for Exchange virtual machines Viewing the XtremCache disk in Windows Disk Manager Configuring the XtremCache device with data deduplication Planning the XtremCache design deployment When you are planning to implement XtremCache into an existing or new Exchange 2010 deployment on vsphere, consider the following: Size of the XtremSF PCIe card to deploy Number of Exchange virtual machines deployed on each vsphere host that will be using XtremCache Exchange workload characteristics (for example, the read:write ratio and user profile type) All of the above elements contribute to your final XtremCache design. Determining the XtremSF card size In this solution, we designed and configured each vsphere server to host three Exchange Mailbox server virtual machines to operate during normal conditions. We also provisioned enough CPU and memory capacity on each vsphere server to host another three Mailbox server virtual machines from the other cluster node to handle failures or maintenance events for the other vsphere server. So, in total, each vsphere server could potentially host six Exchange Mailbox server virtual machines. To provide the same performance for the Exchange virtual machines that are migrated from other vsphere cluster nodes, as the Exchange virtual machines already hosted on the server, we need to provision enough XtremCache capacity for each virtual machine. For that, we used a 350 GB XtremSF PCIe card (with 326 GB usable capacity) on each vsphere server hosting the Exchange Mailbox servers. To efficiently use the entire capacity of the card, we created a 300 GB datastore on each vsphere host and then created 50 GB XtremCache caching devices for each of the virtual machines on this vsphere host. The remaining 150 GB of XtremCache capacity we reserved for Exchange virtual machines that can be migrated from the other cluster node in the event of a failure or for required maintenance. To effectively use XtremCache, consider using a 1,000:10 ratio when configuring your XtremCache device. That is, for each 1 TB of Exchange data on the virtual machine, Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 19

20 configure approximately 10 GB of XtremCache. In our solution, for each Exchange virtual machine with 5 TB of storage, we configured 50 GB of XtremCache caching device. Figure 5 details the virtualized Exchange 2010 design enhanced with XtremCache deployed and configured on each vsphere server. Figure 5. XtremCache design for Exchange 2010 on vsphere Determining the Exchange workload characteristics To properly configure the XtremCache device for maximum performance and efficiency for Exchange 2010, you first need to be familiar with Exchange 2010 I/O workload characteristics. Exchange 2010 workload is characterized by a large 32 KB mostly random I/O for the databases, and small 4 KB sequential I/O for the transactional log files. A large 256 KB sequential read I/O is also associated with the Exchange background database maintenance process (BDM). When the Exchange 2010 database s 32 KB I/O is mixed with 256 KB BDM I/O and coalesced on the host, it produces large random I/O activity to the storage subsystems. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 20

21 Figure 6 shows the Exchange 2010 I/O characterization that we observed during the LoadGen simulated workload. It shows approximately 48 KB read/write random I/O to the database LUNs when BDM was not running for some databases and 128 KB random I/O to the database LUNs when BDM was running. Figure 6. Exchange 2010 I/O characteristics Based on the Exchange 2010 sequential I/O workload to the log volumes and random I/O workload to the database volumes we recommend that you enable XtremCache acceleration for the database volumes only, and not the log volumes. In Exchange 2010, the larger database cache decreases the number of reads to the database on disk causing the reads to shrink as a percentage of total I/O. Under medium to high workloads ( messages/user/day) the database read:write ratio is usually 3:2 (60 percent reads and 40 percent writes). But when user workload increases (300 messages/user/day and above), you may see the read:write ratio changing to 1:1 (50 percent reads and 50 percent writes). Configuring XtremCache with VMware In a VMware environment, the XtremSF card resides on the ESXi server, while the XtremCache software is installed on each of the virtual machines that is accelerated by XtremCache. You use the XtremCache VSI plug-in, which resides on the vcenter client, to manage XtremCache. XtremCache can accelerate performance for either RDM or VMFS LUNs in a VMware environment. In our solution, we configured XtremCache to accelerate the performance of Exchange database volumes configured as RDM devices. The XtremCache installation is distributed over various vsphere system components. Figure 7 shows the location of these installed components, with the XtremCache software installed on the guest machines and the XtremCache VSI plug-in installed on a vsphere client. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 21

22 Figure 7. EMC XtremCache components Configuring XtremCache Configuring XtremCache is a straightforward process using the integrated XtremCache VSI plug-in wizard. Alternatively, you can also use the XtremCache CLI from the Windows virtual machine to configure XtremCache on each Exchange server. We followed these steps to configure the XtremCache for Exchange virtual machines: 1. From the vsphere vcenter client, we created a virtual machine file system (VMFS) datastore on the cache device. 2. From the XtremCache VSI plug-in, we created XtremCache caching devices for each Exchange virtual machine on the ESXi server in the form of virtual disks to each virtual machine. 3. Using the XtremCache CLI on the Windows virtual machines, we added the XtremCache caching devices to each Exchange virtual machine using the following parameters: page_size 64 and max_io_size 64 Note: In the XtremCache software release used in this solution (1.5.1), the XtremCache VSI plug-in does not provide the option of configuring a specific page size. This capability will be available in the next revision of the XtremCache VSI plug-in software. 4. Using the XtremCache VSI Plug-in on a vcenter client or XtremCache CLI on the Windows virtual machine, we added Exchange LUNs as the source devices to the enabled XtremCache device to accelerate performance. Note: Any source device can be stopped temporarily for the caching operation with XtremCache or removed from caching without affecting other source devices. In summary, after installing a 350 GB XtremSF PCIe card in each of the ESXi servers hosting Exchange virtual machines, we used the entire 326 GB usable capacity of the card and configured a 300 GB VMFS datastore. Then, using the VSI with XtremCache plug-in configuration wizard, we created 50-GB cache devices for each of the three Exchange virtual machines configured on the host. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 22

23 We repeated the same process to create cache devices on the vsphere cluster server hosting the other three members of the Exchange DAG. If you prefer, you can also use the XtremCache CLI from the Windows virtual machine to perform same function. Figure 8 shows the VSI with XtremCache plug-in configuration wizard for creating XtremCache devices. Figure 8. XtremCache configuration using EMC VSI plug-in Configuring the XtremCache device for Exchange virtual machines Using the XtremCache CLI on each Exchange virtual machine, we added a 50 GB cache device to each of the virtual machines using specific XtremCache parameters for page size and maximum I/O size. We set the XtremCache page size to 64 KB and block I/O maximum size to 64 KB to optimize the XtremCache performance for the Exchange 2010 workload. This setting enables more efficient cache read requests based on the Exchange 2010 application I/O size. The following CLI command adds an XtremCache device to the Exchange virtual machines: vfcmt add -cache_dev harddisk13 set_page_size 64 set_max_io_size 64 In the above command, harddisk13 is a configured operating-system cache device for the virtual machine. Note: In the XtremCache software release used in this solution, the XtremCache VSI plug-in does not provide the option of configuring a specific page size. This capability will be available in the next revision of the XtremCache VSI plug-in software. Figure 9 shows the VSI with XtremCache plug-in configuration wizard after the cache device and Exchange LUNs (source devices) were configured for cache acceleration. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 23

24 Figure 9. XtremCache configuration for Exchange Mailbox server virtual machine Viewing the XtremCache disk in Windows Disk Manager After configuring the XtremCache device for the virtual machine, the device is displayed in the Virtual Machine Properties dialog box as Thick Provision Lazy Zeroed under Disk Provisioning (shown in Figure 10) and in Windows Disk Manager as the Original Equipment Manufacturer (OEM) partition (shown in Figure 11). Figure 10. XtremCache device in Virtual Machine Properties Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 24

25 Figure 11. XtremCache disk on Windows virtual machine Configuring the XtremCache device with data deduplication You can also configure the XtremCache data deduplication feature for Exchange Mailbox server virtual machines. Data deduplication eliminates redundant data by storing only a single copy of identical chunks of data, while at the same time, providing access to the data from the cache. Deduplication also helps to reduce storage and bandwidth requirements and extend cache device life expectancy. To enable data deduplication for the XtremCache device, select the checkbox for Use Data Deduplication under Advanced configuration in the Add VFCache Device wizard while adding the XtremCache device to a virtual machine. Select the deduplication percentage gain based on your Exchange workload type as shown in Figure 12. You can also enable data deduplication using the XtremCache CLI on the Windows client machine by executing the following command: vfcmt add -cache_dev harddisk13 set_page_size 64 set_max_io_size 64 enable_ddup ddup_gain 20 In the preceding command: harddisk13 is a configured operating-system cache device for the virtual machine. ddup_gain 20 is the deduplication gain percentage for the system cache device on the virtual machine. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 25

26 Figure 12. Enabling data deduplication on the XtremCache device with the XtremCache VSI plug-in After adding the deduplication-enabled XtremCache device, add the Exchange database LUNs as source devices to the XtremCache device for performance acceleration. To determine the appropriate data deduplication gain for your Exchange workload, review the XtremCache statistics information in the XtremCache VSI plug-in or use the CLI on the Windows server, after the cache warm-up, and follow these recommendations: Calculate the observed deduplication hit ratio, and compare it with the configured ratio. Calculate the observed deduplication hit ratio by dividing the Write Hits by the Writes Received. This is the amount of duplicated data in the cache. If the observed ratio is less than 10 percent, turn off deduplication, or to reconfigure the deduplication gain to zero percent. Keeping deduplication enabled will allow you to benefit from the extended cache device life. If the observed ratio is over 35 percent, raise the deduplication gain to match the observed deduplication. If the observed ratio is between 10 and 35 percent, leave the deduplication gain as it is. To change the configured ratio, remove the XtremCache device from the Exchange Mailbox server virtual machine, and re-add the device with a new deduplication percent value using the VSI plug-in or the CLI command (vfcmt add -cache_dev), as described earlier in this section. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 26

27 The results of the deduplication tests with Exchange 2010 are described in Solution validation and performance results. Migrating Exchange virtual machine with XtremCache device Moving an Exchange virtual machine that has an XtremCache disk from one vsphere host to another is possible. Under a typical scenario, without an XtremCache device, you can use the native vsphere migrate command to move a virtual machine from one host to another. This is possible because in a typical scenario, the virtual machine s datastores and RDMs are shared resources. In the XtremCache environment, however, the XtremCache datastore is mapped to its local host flash drive. Consequently, this datastore is accessible only to that host; thus the native vsphere migrate command is not supported. For this you should use the XtremCache VSI plug-in to perform the virtual machine migration that has an XtremCache device attached. Multiple forms of migration are available. The form of migration that you choose determines the steps you perform to complete the migration. Before you begin, ensure that your system meets the following prerequisites: The target datastore has enough available capacity for the new device. There are no additional DAS flash-based devices for the host virtual machine. Only one XtremCache device is configured on the host virtual machine. The virtual machine you want to migrate is not currently being migrated. The source host and the target host must be able to communicate with each other (ensure that the IP address and DNS have been properly configured). Performing an automated migration An automated migration does not require disrupting the virtual machine; however, the cache will be cleared, resulting in a cold cache start. A warning message indicating that a cold cache start will occur after the migration displays during the migration. To perform the migration, complete the following steps: 1. From the XtremCache window of a virtual machine, click Migrate Virtual Machine. 5. Select the target host and target datastore. Only hosts within your datastores that are available to the host XtremCache device will be listed. 6. Click OK. The warning message is displayed stating that the cold cache start will take place after the migration as shown in Figure 13. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 27

28 Figure 13. Cold cache start post-migration warning 7. Click Yes to start migration. 8. Follow the task progress in the Task window. Figure 14 details the virtual machine migration wizard. Figure 14. EMC XtremCache VSI-plug-in for virtual machine migration wizard After migration is completed, the successful migration message appears in the VSI management window. Recovering Exchange data from a snapshot If you are using backup software that performs snapshots of Exchange LUNs accelerated by XtremCache, you need to follow specific procedures when restoring data from those snapshots to ensure data integrity. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 28

29 If an Exchange LUN snapshot is taken on the array, and then later used to roll back changes on the source LUN, the server will not be updated with the changes made. This could result in the cache supplying data that has not been updated with the contents of the snapshot. To prevent this from occurring, when recovering from snapshot, perform the following steps: 1. Quiesce the application that is accessing the source volume using application-specific tools, such as EMC Replication Manager. 9. Flush the data in the host buffers using an appropriate command, such as admsnap flush, and unmount the file system. 10. Invalidate the contents of the source device by using the purge - source_dev command. 11. Perform the snapshot operations on the array. 12. After the restore is complete, remount the file system, as necessary. Monitoring XtremCache performance You can monitor XtremCache performance using the VSI XtremCache plug-in for vcenter or using CLI commands on Exchange virtual machines. For example, to review XtremCache statistics for the cache device on the virtual machine select Statistics under Device Configuration as shown in Figure 15. Figure 15. XtremCache device statistics information with XtremCache VSI plug-in You can also review the statistics by issuing the vfcmt display all command on the Windows virtual machine. The summary of the cache device is displayed as shown in Figure 16. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 29

30 Figure 16. Displaying the XtremCache device statistics using the CLI command Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 30

31 Solution validation and performance results This section details the solution validation and performance test results: About performance results Exchange performance validation tools Monitoring Exchange performance Monitoring the Exchange server s health Solution validation with Exchange Jetstress Performance results with Jetstress servers XtremCache acceleration test results Storage array performance results with Jetstress Solution validation with Exchange LoadGen Analyzing the LoadGen test results LoadGen test results summary Performance with XtremCache data deduplication Deduplication test results summary About performance results Performance results are highly dependent on workload, specific application requirements, and system design and implementation. Relative system performance will vary as a result of these and other factors. Therefore, we suggest that you do not use this workload as a substitute for a specific customer application benchmark when you are planning for capacity and evaluating product decisions. All performance data contained in this document was obtained in a rigorously controlled environment. Results obtained in other operating environments may vary significantly. EMC Corporation does not warrant or represent that a user can or will achieve similar performance expressed in transactions per minute. Exchange performance validation tools To evaluate how XtremCache improves Exchange 2010 performance, we performed a full end-to-end solution validation using standard tools for Exchange performance validation, including both Jetstress and LoadGen. Each tool is described briefly below. Exchange Jetstress Jetstress 2010 works with the Microsoft Exchange Server 2010 database engine to simulate the Exchange database and log disk I/O load. It simulates the Exchange database and log file loads produced by a specific number of users. Jetstress is primarily used to verify the performance and stability of a disk subsystem prior to putting an Exchange 2010 server into production. Jetstress helps verify that your disk subsystem meets or exceeds the performance criteria you establish. After successful completion of the Jetstress disk performance and stress tests in a nonproduction environment, you will have verified that your Exchange 2010 disk Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 31

32 subsystem is adequately sized (in terms of performance criteria you establish) for the user count and user profiles you have established. Jetstress is also helpful in identifying any issues or bottlenecks in your I/O stack, from Windows STORport drivers to the SAN and hypervisor infrastructure, prior to production deployment. Note: You will never see the workload that Jetstress creates in your production environment. Jetstress runs at 100 percent user concurrency with the maximum load to the storage subsystem. It is highly unlikely for this condition to occur in your production environment. EMC strongly recommends that you use Jetstress to validate storage reliability and performance prior to deploying your Exchange servers in a production environment. You can download Jetstress from the Microsoft Download Center: Jetstress documentation describes how to configure and execute an I/O validation or evaluation on your server hardware. In this solution, we used Jetstress to evaluate how the VNX storage subsystem was affected by the Jetstress workload and how XtremCache acceleration on the database LUNs helped to reduce the read I/O to the back-end storage. We also used Jetstress to identify the best option for XtremCache page and max_io parameters for the XtremCache device. Exchange Load Generator While the Jetstress tool tests the performance of the Exchange storage subsystem before placing it in the production environment, it does not test the impact of the server CPU and memory configuration of MAPI user activity on the entire Exchange Infrastructure. For this purpose, we used the Exchange Load Generator (LoadGen) tool. LoadGen is designed to produce a simulated client workload against a test Exchange deployment. Although this workload is different from the workload you will see in your production environment, it can be used to evaluate how Exchange performs and to test the overall solution. It can also be used to analyze the effect of various configuration changes on Exchange behavior and performance while the system is under load. LoadGen is a useful tool for administrators when they are sizing servers and validating a deployment plan. Note: LoadGen requires full deployment of the Exchange environment for validation testing. You should perform all LoadGen validation testing in an isolated lab environment where there is no connectivity to production data. Also, ensure that you plan an adequate time to prepare your test environment as the database initialization process can take a long time depending on the mailbox size and number of users in your test Exchange deployment. In our lab environment, for example, populating 15,000 users with 1.5 TB of Mailbox data took three weeks. For additional information about using Jetstress and LoadGen, refer to the Microsoft TechNet Library topic Tools for Performance and Scalability Evaluation. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 32

33 Monitoring Exchange performance When validating whether a Mailbox server is properly sized you should focus on processor, memory, storage, and Exchange application health. When validating performance on Exchange 2010 Mailbox servers, EMC recommends that you use the key counters with the target benchmark values shown in Table 7. For a full list of all Exchange 2010 performance counters, review the Microsoft TechNet Library topic Performance and Scalability Counters and Thresholds. Table 7. Key Exchange performance counters Counter MSExchange Database\I/O Database Reads Average Latency MSExchange Database\I/O Database Writes Average Latency MSExchange Database\IO Log Writes Average Latency MSExchangeIS\RPC Averaged Latency MSExchangeIS\RPC Requests Target <20 ms <20 ms <10 ms <10 ms on average <70 at all times Processor(_Total)\% Processor Time <80% For Exchange Jetstress, you should concentrate on the disk latencies and IOPS you achieved from your storage subsystems, while with LoadGen you consider CPU, memory, and RPC latencies on the Exchange Mailbox servers and other Exchange roles. Note: In this solution, we focused only on the performance of the Mailbox server role and not the Hub Transport and Client Access roles. Monitoring the Exchange server s health Even if there are no obvious issues with processor, memory, or disk, EMC recommends that you monitor the standard application health counters to ensure that the Exchange Mailbox server is in a healthy state. The MSExchangeIS\RPC Averaged Latency counter provides the best indication of whether other counters with high database latencies are actually impacting Exchange s health and client experience. Often, high RPC-averaged latencies are associated with a high number of RPC requests, which should be less than 70 at all times. Solution validation with Exchange Jetstress To validate Exchange performance with XtremCache, we configured Exchange Jetstress to run on three exchange virtual machines with all databases configured in a single VNX storage pool simulating a 15,000-user workload (similar to a DAG database switchover condition). In the first test, we established our baseline, where XtremCache was not enabled on the database LUNs. In the second test we, enabled XtremCache on the database LUNs to accelerate exchange performance. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 33

34 Performance results with Jetstress servers XtremCache acceleration test results After both tests completed, we analyzed the results and compared the I/O and disk latency performance between the two tests on the Exchange servers and the backend storage array. For performance comparison we only measured achieved IOPS for the Exchange database LUNs. These IOPS include: Database IOPS (user IOPS) database reads and database writes BDM IOPS IOPS associated with Background Database Maintenance process Figure 17 and Figure 18 display the results of the tests. We observed the following: IOPS aggregate from three Mailbox server improved by 26 percent from 2,812 IOPS to 3,545 IOPS. Read IOPS increased by 34 percent from 1,388 IOPS to 1,862 IOPS. Write IOPS increased by 33 percent from 851 IOPS to 1,118 IOPS. Read latencies decreased by 3.2 ms. The marginal decrease in read latencies occurred because Jetstress is not fully utilizing server memory. Jetstress is designed to stress the back-end storage system by bypassing local server resources. When Exchange LoadGen workload is used, as described later in this white paper, the beneficial effect of XtremCache is more apparent. Figure 17. Exchange performance with XtremCache Jetstress results (IOPS) Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 34

35 Figure 18. Exchange performance with XtremCache Jetstress results (latencies) Storage array performance results with Jetstress On the VNX storage array, we observed the following results when XtremCache was enabled on Exchange database LUNs in the virtual machines. While the read IOPS decreased, because of XtremCache acceleration on the host, the write IOPS increased resulting in almost unchanged total IOPS produced by the VNX R1/0 NL-SAS storage pool. There was a slight increase in bandwidth with increased disk utilization as a result of the array processing 50 percent more writes. The following results summarize the data presented in Figure 19. Sixteen-and-a-half percent decrease in read IOPS to the back-end storage array because XtremCache offloads the reads from the array to the server. Fifty percent increase in write IOPS to the back-end storage array because XtremCache offloads the reads from the array to the server, allowing more write activity to be processed by write-through cache. Fifteen percent increase in disk utilization due to the array processing more writes. Six-and-a-half percent increase in bandwidth (MB/s) due to the increased write activity processed by the array. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 35

36 Figure 19. VNX storage performance with Exchange Jetstress Solution validation with Exchange LoadGen To validate the effectiveness of XtremCache with Exchange 2010 under MAPI workload, we performed multiple tests with different LoadGen workload profiles. We installed Exchange 2010 and initialized 15,000 users with up to 1 GB mailboxes that resulted in the 15 TB total working set (5 TB per each Exchange server virtual machine). For testing purposes, we performed a DAG database switchover that resulted in three Exchange virtual machines hosting 5,000 active users configured from a single VNX storage pool with 48 2-TB NL-SAS drives in RAID 1/0 configuration. We ran each LoadGen workload simultaneously on three Mailbox servers with 5,000 users per Exchange Mailbox server and with the Outlook profile in cached mode. Each LoadGen test ran for 8 hours with a full workload. As stated earlier, the baseline profile for this solution is 150-messages/user/day. With each test, we increased the workload to 250 messages/user/day and then to 300 messages/user/day. We conducted these tests without changing the virtual machines configuration or the back-end VNX storage. After establishing a baseline for each workload profile, we reran the tests with XtremCache enabled on the Exchange database LUNs. Note: To simplify the comparison analysis between workloads, we configured each Exchange virtual machine with six vcpus and 32 GB of RAM. Although only four vcpus were required for a 150-message/user/day workload profile, eight vcpus were required for 250- and 300- message/user/day workloads. Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 36

37 Analyzing the LoadGen test results LoadGen test results summary To ensure accuracy in our analysis, we used the following values for results comparison: IOPS value We calculated the results by adding database read I/O, database write I/O, and BDM I/O for all three Exchange virtual machines, each running a 5,000-user workload. The BDM process ran periodically on each database during the eight-hour LoadGen runs. Latencies We used database read and write latency performance counters from each Exchange Mailbox server. After performing all tests, we observed the consistent reduction in read latencies and increased user IOPS with all workload types when XtremCache was enabled to accelerate performance for database LUNs. Even 300-message workloads that experienced over 20 ms read latencies without XtremCache, became a normal steady workload with reduced latencies and increased IOPS when XtremCache was enabled for the performance acceleration of database LUNs. This failing workload was expected, because the storage and Exchange virtual machine resources were originally designed for 150-message workloads. Figure 20 provides additional details for each test performed. Figure 20. Exchange 2010 performance with XtremCache and LoadGen workload The observed test results highlights are as follows: A 150-message/user/day workload achieved a 51 percent reduction in read latencies (by 6.4 ms) and a 14.6 percent increase in user IOPS (by 224 IOPS). A 250-message/user/day workload achieved a 69.3 percent reduction in read latencies (by 11.1 ms) and a 12.8 percent increase in user IOPS (by 275 IOPS). Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 37

38 A 300-message/user/day workload achieved a 56.8 percent reduction in read latencies (by 12.5 ms) and a 12 percent increase in user IOPS (by 346 IOPS). Performance with XtremCache data deduplication To validate Exchange performance with XtremCache inline data deduplication, we performed validation on one Exchange virtual machine with 5,000 users. We performed a series of LoadGen tests, with each test for 8 hours, and with multiple workload profiles to see the effect of data deduplication. We monitored the XtremCache statistics to determine the appropriate deduplication ratio for each workload. With LoadGen workloads we generated, we observed that a 30 percent deduplication ratio would be more effective than the default 20 percent. Figure 21 shows the deduplication ratio observed during testing. Note: As stated earlier, the LoadGen workload does not represent the actual workload that will be seen in your production environment. The results observed and recommendations provided here are based on lab configuration and results only. Ensure that you configure your environment based on your workload requirements and characteristics. Figure 21. XtremCache statistics with data deduplication Deduplication test results summary Figure 22 and Figure 23 show the XtremCache data deduplication test results with multiple workload profiles for the Exchange 2010 Mailbox server, demonstrating: Decreased Exchange server CPU utilization with each workload Slightly increased write latencies due to XtremCache analysis and processing of duplicate data Accelerating Microsoft Exchange 2010 Performance with EMC XtremCache 38