Virtualizing Microsoft Exchange Server 2007 on Dell Systems Running VMware ESX

Transcription

1 Virtualizing Microsoft Exchange Server 2007 on Dell Systems Running VMware ESX A Dell Technical White Paper End to End Solutions Engineering By Ramesh Radhakrishnan and Tom King Dell Product Group May 2009

2 THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND Dell Inc. All rights reserved. Reproduction of this material in any manner whatsoever without the express written permission of Dell Inc. is strictly forbidden. For more information, contact Dell. Dell, the DELL logo, EqualLogic, and the DELL badge are trademarks of Dell Inc.; Microsoft, Outlook, and Windows are either trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Page ii

3 Table of Contents Virtualizing Microsoft Exchange Server 2007 on Dell Systems Running VMware ESX... i Executive Summary... 2 Introduction... 2 Virtualized Exchange Server 2007 Building Block... 3 Solution Architecture... 3 Deployment Best Practices... 4 Network Setup and best practices... 5 Storage Best Practices... 5 Performance Evaluation of Virtualized Mailbox Role... 5 Test Methodology... 5 Scalability Testing... 6 Conclusion... 9 References... 9 Appendix A Page 1

4 Executive Summary This paper provides an overview of deploying Exchange 2007 Server in a virtualized environment using VMware 3.5 Update 3 on Dell PowerEdge Servers and EqualLogic iscsi storage array. In this paper a solution building block is proposed for 4000 mailbox users to deploy in an ESX environment running on Dell hardware. Various mailbox configurations of Exchange Server 2007 are tested to understand the sizing requirements for Exchange Server 2007 in a virtualized environment and validate the building block. Introduction There is a growing interest in deploying Exchange Server 2007 in virtualized environments with the recent announcement regarding change in support policies for virtualized Exchange deployments 1. In a virtualized environment, Exchange users can consolidate multiple Exchange Server Mailbox roles on a single physical server to take advantage of unused CPU cycles or available memory. Additional Exchange Server roles (Hub Transport or Client Access Server) can be hosted on virtual machines (VM) and consolidated on the same hardware to reduce costs. Benefits of running Exchange in a virtualized environment will depend on the customer environment. Some of the advantages of virtualizing Exchange 2007 Server are: Cost savings thru server hardware consolidation Reducing performance bottlenecks by moving Mailbox VMs to systems with available CPU cycles Easier management of Exchange systems and infrastructure In this paper a sample configuration is presented which is tested and validated to meet the performance and stability requirements of an enterprise Exchange deployment. This sample configuration is scalable and can be used as a building block for larger scale deployments. By deploying Exchange 2007 Server on ESX environment hosted on Dell Servers and EqualLogic iscsi SAN, organizations can reduce the management complexity and improve utilization of hardware resources by consolidating on fewer hardware resources. To provide sizing guidelines for Exchange Server 2007 the Mailbox role was deployed in a Virtualized environment and tested using multiple Mailbox configurations. The trending data was used to understand the CPU and memory requirements for Exchange Server 2007 running within a VM and how it differed compared to a physical environment. For consolidation purposes, the performance of deploying Exchange Server 2007 Mailbox roles in multiple VMs on a single server is also evaluated. This white paper is intended for customers, IT architects, storage administrators and IT administrators who are evaluating, planning, or deploying Exchange 2007 Server on VMware ESX 3.5 platform and require guidance on available options and best practices. The goals of this white paper are to introduce customers to Dell s building block for deploying Exchange 2007 Server on VMware ESX and simplify the decision making process for the customer when deploying Exchange 2007 in Dell hardware running ESX hypervisors. 1 Microsoft Support Policies and Recommendations for Exchange Servers in Hardware Virtualization Environments, us/library/cc aspx Page 2

5 Virtualized Exchange Server 2007 Building Block This section presents an overview of the building block configuration for an Exchange Server 2007 deployment which uses VMware ESX to create a virtualized environment. The building block presented in Figure 1 was tested using LoadGen tool to simulate a 4000 heavy user workload and it comfortably met the performance requirements recommended by Microsoft for good quality of service. The Performance Evaluation section describes how this building block configuration performed at the server, storage and application level when LoadGen was used to simulate up to 4000 users. Networking Legend Exchange Mailbox VM CAS/Hub Transport VM Active Directory VM Workload LAN iscsi Fabric W8OS W8OS W8OS Clients Vmware ESX Dell PowerEdge R710 PC6248 PC6248 EqualLogic PS6000XV EqualLogic PS6000XV Virtualized Storage Environment Solution Architecture Figure 1 Architecture of Building Block for Exchange 2007 on ESX 3.5 In this section the core components that make up the Dell VMware Virtualized Exchange Building Block are detailed. The architecture can be broken into four primary layers, each coexisting and working together to provide an optimized platform for messaging deployment targeting small to mid size organizations or businesses. The individual layers of the building block are Layer 1: Dell PowerEdge Servers Layer 2: Dell EqualLogic iscsi SAN Storage Layer 3: VMware Infrastructure 3 Page 3

6 Layer 4: Windows Server 2008 and Microsoft Exchange Server 2007 Layer 1: Selecting the right server for a virtualized Exchange environment requires careful consideration since it could negatively impact the cost and performance of the overall solution. The proposed building block uses the Dell PowerEdge R710 server with dual quad core processors (Intel Xeon 5500 series). The R710 server is optimized for virtualization solutions with its enhanced memory and I/O features. Microsoft recommended guideline should be followed when determining the CPU and memory resources being assigned to the individual machines running the Exchange 2007 Roles. [1] Layer 2: The storage solution used for the virtualization building block is based on the Dell EqualLogic 6000 Series iscsi SAN. EqualLogic storage arrays provide a cost effective virtualized storage solution that offers excellent performance for messaging environments. The number and size of volumes for Exchange 2007 mailbox databases and logs were based on best practices recommended by Dell and Microsoft and depends on the number and size of user mailboxes. When planning storage architecture for Exchange 2007 the Exchange 2007 Storage Requirements calculator is a valuable tool. [2] Layer 3: VMware Virtualization Infrastructure 3 (VI3) was used as the virtualization foundation to create the virtualized messaging environment. VI3 abstracts hardware resources (CPU, memory, storage and network) to components that can serve multiple virtual machines providing greater hardware utilization. Using a virtualized environment allows the hardware to run multiple virtual machines that host the individual pieces of the Exchange 2007 solution. Multiple virtual machines can be used to run the Exchange 2007 Mailbox role, each hosting a small number of users to allow more flexibility to the administrator in terms of migrating VMs across physical servers. Some solution may use a larger VM configuration and host a larger number of users per VM while still taking advantage of virtualization features like decoupling from hardware and increasing hardware utilization levels. Layer 4: The final layer consists of the guest OS running in the VMs and the application itself. The virtual machines run Windows Server 2008 OS editions and Exchange Server 2007 roles are installed on the guest VMs. Additional infrastructure server such as Active Directory server may be virtualized as well to increase server consolidation. Exchange 2007 server best practices should be followed when implemented in a virtualized environment as well to maximize performance and solution stability [3]. Additional consideration should be made for storage configuration as iscsi connections can be established at the guest OS level or it may be configured at the ESX host level. Similarly, network design should be planned out ahead so that enough network devices are available for both workload LAN and storage fabric to meet the performance requirements of the messaging solution. Deployment Best Practices In this section, we discuss a few of the best practices and proceedures that should be followed when implementing the network and storage infrastructure for the virtualized Exchange 2007 deployment. Page 4

7 Network Setup and best practices Although not required or necessary, it is better to isolate the iscsi traffic from the remaining virtual machine traffic. This allows more fine grained monitoring and control over the networks differing configuration, e.g. Jumbo Frames for iscsi traffic versus normal MTU settings for the remaining traffic. Further, each network should be divided with multiple physical paths for load balancing and failover capabilities. In our test bed, we used one PowerConnect 6248 for virtual machine network and management traffic connected to one pair of NICs on the ESX server, and another PowerConnect 6248 for iscsi traffic connected to another pair of NICs from a dual port PCIe card on the ESX server. In highly mission critical installations, these networks can be further divided to ensure no single point of failure. ESX provides highly flexible tools when configuring separate networks. See Appendix A, Figure 6 for details of how the network topology was setup using the ESX networking management tool. Storage Best Practices Throughout each set of tests, each mailbox server was provided 80% of its allocated space for the mailbox database and 20% of its allocated space for the mailbox logs. Each mailbox server s iscsi target was not shared with other virtual machines. None of these particular iscsi targets housed the virtual machines VMDK files. The ESX array used RAID 10 across 14 disks with two hot spares. Prior testing using Jetstress showed that RAID 10 offers the best performance in similar environments. Our testing covered three different storage access configurations: iscsi initiator on guests with MPIO iscsi initiator on ESX with VMFS file systems iscsi initiator on ESX with Raw Device Maps (RDM) If any external systems need access to the database and log files, using RDM is advantageous. Additionally, using RDM allows switching the file system to be accessed with a guest based iscsi initiator utilizing a multipath I/O driver without reformatting. On the other hand, VMFS offers added high availability features such as integration with EqualLogic ASM/VMware editions to create VSS aware snapshots and backups. Whereas using the ESX 3.5 iscsi initiator allows only one session per target, using the guest based initiator allowed load balancing through MPIO. However, our tests showed there was no performance gain in doing so due to the increased overhead and added layers between the guests and the iscsi targets. Performance Evaluation of Virtualized Mailbox Role This study evaluates the scalability and performance characteristics of running Exchange Server 2007 in a VMware ESX environment. The performance of running Exchange Server roles (Mailbox, Hub Transport and CAS) in a virtual machine is compared against running them in a physical server to measure the virtualization overhead associated with a messaging workload. Test Methodology Page 5

8 The sizing of the VMs (CPU cores and memory) is based on Dell and Microsoft recommended guidelines for a heavy user Mailbox profile and prior testing done at Dell for physical Exchange deployments. The test configuration is monitored for bottlenecks by measuring Server, Storage and Exchange related counters within the Virtual Machine (VM) and also at the host ESX level. Microsoft Exchange Load Generator (LoadGen) tool (version ) is used to simulate user workloads and evaluate performance of the Exchange Server 2007 server configurations. For sizing recommendations, performance is evaluated at multiple levels application, server, and storage. Performance is measured at the application level by using Exchange related metrics measured by Exchange LoadGen. Microsoft Reliability and Performance monitor is used to measure server and storage level performance metrics. Esxtop is used to collect CPU and memory usage from the ESX server. Table 1 lists events that were measured and thresholds that were used to determine if a configuration performed as expected. Table 1: Counters Used to Evaluate Performance Performance Counter Expected Value (MS Recommended) SERVER PERFORMANCE Exchange VM: Processor(_Total)\% Processor Time Average less than 75% ESX : Esxtop Physical Cpu(_Total)\% Processor Time Average less than 75% APPLICATION PERFORMANCE LoadGen 95 th Percentile Send Mail Latency MS ExchangeIS: RPC Averaged Latency MS ExchangeIS Mailbox: Messages Queued for Submission STORAGE PERFORMANCE Database Disks: Average Sec/Read Database Disks: Average Sec/Write Log Disks: Average Sec/Write Less than 1000 ms Average less than 25 ms Average less than 50 Less than 20 ms Less than 100 ms Less than 10 ms Scalability Testing LoadGen was used to simulate 1000 to 4000 users and tested against the solution building block to study the scalability of the proposed solution configuration. Table 2 describes the VM configuration in terms of CPUs and memory that was allocated to the VM. These configurations are based on the resources that would be recommended for a physical deployment to satisfy similar number of users and workload. Page 6

9 Table 2 Exchange Server 2007 Mailbox VM Configuration 1000 User VM 2000 User VM 3000 User VM 4000 User VM CPUs 2 vcpu 2 vcpu 4 vcpu 4 vcpu Memory 2GB(OS)+5GB 2GB(OS)+10GB 2GB(OS)+15GB 2GB(OS)+20GB HardDisk (OS) 20GB 20GB 20GB 20GB Figure 2 illustrates the performance and scalability of the proposed architecture. The primary metrics used to compare performance were the host server s processor utilization and the 95th percentile response time for the LoadGen SendMail task. The proposed architecture building block is observed to provide good quality of service as the 95th percentile s average latency is less than 500 milliseconds and still has ample CPU headroom to handle spikes in the workload. Figure 2 Scalability of Exchange 2007 Server in ESX Environment Figure 3 illustrates two of the RPC specific counters that are commonly monitored to evaluate the health and performance of the messaging environment. RPC averaged latency represents the time in milliseconds that it takes Exchange (store.exe) to process the last 1024 packets received. It represents the shortest amount of time that clients have waited for a response from the server. The RPC averaged latency is well below the threshold of 25 ms for 4000 users indicating that the server is able to process requests in a reasonable amount of time. The Messages Queued for Submission performance counter indicates the number of messages in the mailbox store that are ready to be sent and are waiting to be submitted to a transport server. The queued messages are observed to be well within the recommended threshold of 50 indicating a healthy configuration. Page 7

10 Figure 3 Application Level Performance Figure 3 illustrates the observed disk subsystem performance observed for the proposed solution building block using EqualLogic iscsi Storage Arrays. Disk subsystem bottlenecks that usually manifest as high disk latencies can cause performance bottlenecks that lead to poorly performing Exchange deployments. Figure 4 Storage Level Performance The disks read and write latencies are within the 20 millisecond threshold that is recommended for optimal performance of the Exchange 2007 deployment. The mailbox database disk latencies are observed to increment as the number of users are increased from 1000 to 4000, but still remain well within the 20 ms threshold. The storage subsystem was designed to support 4000 heavy users and took in to account both the capacity requirements and the performance requirements for this workload. A RAID 10 configuration was used for both database and log disks to meet the performance requirements of the LoadGen heavy workload profile. Finally, in Figure 5 we compare the performance of the virtualized Exchange 2007 solution to a physical deployment to quantify the impact of virtualization. SendMail latency, CPU loads and RPC latency is Page 8

11 compared for virtual and physical deployments of users. SendMail latency is plotted on the primary axis and CPU Load and RPC latency is plotted on the secondary axis. The send mail latency is observed to increase from 64 ms for 1000 users to 94 ms for 3000 users for both physical and virtual Exchange deployments. There is practically no difference in user perceived latencies for either deployment. There is an increase in CPU load for the virtualized Exchange deployment and this is an artifact of both virtualization overhead and running additional Exchange roles on the server. The increase however is minimal, does not affect end user performance as observed from the SendMail latencies and leaves adequate CPU headroom to handle any spikes in the workload. Figure 5 Comparing Performance of Virtualized Building Block to Physical Exchange Deployment Conclusion This study provides an overview of best practices and using a building block approach to deploy Microsoft Exchange Server 2007 on Dell Servers running VMware ESX. Using the Exchange 2007 solution building block proposed in this paper, IT administrators can design, configure and deploy Exchange Server 2007 using Dell servers, EqualLogic storage and VMware ESX virtualization software. References [1] Planning Processor Configurations, Microsoft TechNet us/library/aa aspx [2] Exchange 2007 Mailbox Server Role Storage Requirements Calculator Page 9

12 [3] Dell Exchange Server 2007 Solutions Appendix A Figure 6 Network topology for Virtualized Building Block Page 10

13 Figure 7 Storage Configuration Setup and Details Page 11