1
Contents Introduction... 3 Microsoft SQL Server performance in a virtual infrastructure... 5 High Availability for SQL servers in a virtual infrastructure... 9 Monitoring System Performance... 10 Summary... 11 2
Copyright 1985-2010 All rights reserved. 5555 Oakbrook Parkway, Building 200, Norcross, GA 30093 This publication contains proprietary information that is protected by copyright. No part of this publication can be reproduced, transcribed, stored in a retrieval system, translated into any language or computer language, or transmitted in any form whatsoever without the prior written consent of the publisher, All trademarks and trade names used in this document refer to either the entities claiming the marks and names or their products. disclaims any proprietary interest in trademarks and trade names other than its own. 3
Introduction The drive towards the right-sizing of the datacenter has been a key driver behind the wide acceptance of the deployment of IT infrastructure in a virtual environment. Until now, however, only relatively lessimportant applications have moved to the virtual infrastructure, leaving the more business-critical frontline applications to run on dedicated physical servers. SQL servers, which are crucial to Business Intelligence (BI) and day-to-day business operations, are still mostly deployed in the traditional way on dedicated servers. However, recent studies have shown that with proper planning, even Tier 1 SQL servers can easily be moved into the VMware virtual environment, to take advantage of the well-known advantages of consolidation without affecting business requirements. To make the potential of the move to the virtual environment even more attractive, Microsoft has endorsed the deployment of its various products in a virtual infrastructure and has expressed its commitment to supporting its products under VMware. Deploying StorTrends as the shared storage server in such a virtual infrastructure running Microsoft SQL Server has many advantages. The rewards are many, but as the IT environment gets more intricate with the use of server and storage virtualization, it is imperative that the IT administrator clearly understand the details of these multi-faceted subsystems. The rules of configuring SQL Servers in dedicated physical servers utilizing shared storage apply mostly to the virtual infrastructure. However, server consolidation brings in certain new facets to the IT infrastructure that require a careful planning and consideration. This document will focus on some of these vital aspects and make recommendations of certain best practices to be followed while deploying Microsoft SQL Server using StorTrends in a VMware environment. Note: It is recommended that you read the document titled: Best Practices for Employing Microsoft SQL Server on StorTrends before reading this document, as many of the guidelines outlined in that document apply here also. Please also review the recommendations suggested by Microsoft for deploying SQL servers and by VMware for deploying Microsoft SQL Server with ESX. 4
Microsoft SQL Server Performance in a Virtual Infrastructure When you deploy Microsoft SQL Server in a virtual infrastructure, a number of factors have to be carefully considered to ensure the performance requirements for your business needs are met. This includes the role of the CPU, memory, storage subsystem, networking infrastructure and optimization and tuning of the virtual machines running SQL. In some cases, it may be useful to consider running Microsoft SQL Server on multiple virtual machines (VMs) instead of running multiple databases on a SQL server under a single VM. Here are six important recommendations to balance performance and reliability when migrating SQL Server to the virtual environment: 1. Use a 64-bit Platform and Code with SMP In a recent study of SQL server performance under VMware ESX published by VMware, it has been observed that using a 64-bit VM in a symmetric multiprocessing (SMP) environment delivers appreciably better performance for SQL transactions. So, the first step towards improving SQL Server performance is ensuring that your virtual environment uses 64-bit VMs, deploys 64-bit application server code and uses SMP with more virtual CPUs (VCPUs) per VM. Figure 1: OLTP TPS in 32-bit and 64-bit VMs (Courtesy VMware ) 2. Use a Late-Model CPU and Cut Back on Virtual Server Sprawling In any virtual environment, there are normally more virtual CPUs than the actual number of physical CPU cores, which can sometimes lead to trouble. VMware ESX permits over-commitment of CPU resources, and a careless implementation may lead to severe performance degradation sometimes referred to as Virtual Server Sprawl. Fortunately, VMware provides tools like its Capacity Planner and others to ensure that the overall system performance is balanced. In order to reach a desired level of performance, it is recommended to use newer hardware architecture with late-model processors. Newer CPUs not only feature with multiple cores, higher clock rates, and 64 bit architecture, but also provide phenomenal hardware assist technology for virtualization. Moreover, recent processors have larger CPU cache and a shorter execution pipeline. Finally, current-generation 5
processors have a vastly improved latency for VMEXIT instruction, which greatly improves context switching times and may have a telling effect on overall performance in a busy virtualization setup. 3. Use Sufficient Memory and Correctly Implement Virtual Memory Management The amount of memory needed for a Microsoft SQL Server virtual machine depends on the database workload intended to be assigned to that virtual machine. Allotting additional virtual memory to the VM hosting a Microsoft SQL Server is very important from a performance standpoint, because it allows for service of more I/O requests from the server cache, resulting in reduced disk activities. A thorough Capacity Planner exercise can easily help determine memory needed by a Microsoft SQL Server virtual machine. VMware ESX provides memory optimizations such as sharing and ballooning to reduce the amount of physical memory in the underlying host, and recommends that both memory page sharing and memory ballooning are enabled for this reason. As with CPU resources, ensure that the cumulative physical memory available on a server is adequate to meet the needs of the virtual machines by testing target workloads in the virtual environment, as it is possible to over-commit memory on ESX. For performance-critical SQL Server virtual machines (that is, production systems), ensure that sufficient memory resources are available and that memory overcommitment is not causing any performance degradation. If memory is over-committed on the host, set reserves for performance-critical SQL Server virtual machines to guarantee that the memory for those virtual machines is not ballooned or swapped out. If you set the SQL Server lock pages in memory parameter, make sure to set the virtual machine s reserve to match the amount of memory you set in the virtual machine configuration, as this setting can interfere with the ESX balloon driver. Setting reserves stops the balloon driver from inflating into the virtual machine's memory space. VMware ESX 3.5 supports 2MB memory pages (large pages), and gives the guest operating system the full advantage of using these large pages. If your SQL Server deployment on a native system benefits from large memory pages, you can potentially achieve a similar performance improvement in a SQL Server virtual machine backed with 2MB machine memory pages. Using large pages reduces memory management overhead and can therefore increase hypervisor performance. 4. Configure Shared Storage Server for Optimal Performance and Capacity A key concern for datacenters implementing SQL servers on shared storage servers is to ensure that the storage servers are properly configured. The shared storage server plays a vital role in any virtual infrastructure, and the demands on the shared storage are increased significantly when SQL servers are running under a virtual infrastructure. This means that the planning of storage capacity and performance requirements for shared storage servers must be handled with due care and attention. The number of disks and RAID level employed are critical factors contributing to good storage server performance. According to VMware, the top three performance concerns are: 1) Virtual Machine Disk Format (VMDK) files placed on VMware Virtual Machine File System (VMFS) volumes without enough spindles. 2) Many VMDK files placed on a single VMFS volume that needs additional spindles. 3) Database and log files placed on the same LUN. When working with VMware, the first decision to be made is whether to use VMware Virtual Machine File System (VMFS) or raw device mapping (RDM) disks. It should be pointed out that RDM enjoys a slight edge in performance over VDMS, and on StorTrends storage appliances, enables the use of StorTrends snapshots. In fact, RDM may be desirable in a small number of cases, such as when using MSCS-based failover clustering for SQL Server. However, VMware recommends the use of VMFS as it 6
offers certain capabilities, like vmotion. Additionally, the use of templates and self-provisioning becomes more complicated when RDM is used, along with storage administration with vcenter. I/O latency statistics disk I/O queuing can be monitored with esxtop (or resxtop) to see reports of device latency, time spent in the kernel, and latency seen by the virtual machine. I/O queuing can also be monitored in the VMkernel by checking the number of queued commands. 5. Storage Segment Alignments When running Microsoft SQL Server under a virtual infrastructure like VMware ESX Server, one has to particular attention to the creation of the SAN LUNs and database files. The top two criteria in this regard are partition alignment and selection of file-system block and cluster sizes. When virtual SAN Storage is employed in such an environment, multiple layers for serving storage capacity come into the picture. Each layer has its own groupings or boundaries which have to be aligned and matched in order to ensure optimum performance. For example, SAN servers utilize RAID, which is applied to storage entities In RAID stripes; typically, the stripe size is 64KB. On top of this, the Virtual LUNs may employ particular mapping granularity. For StorTrends this is called a chunk, which is also 64KB; the VMware VMFS file system has a similar allocation unit that is called a block. On top of this resides the Windows file system (NTFS), which works with entities called clusters. Another item that needs to be considered is the SQL I/O page size, since Microsoft SQL servers use 8KB I/O units for reading and writing the database files. In order to extract the best performance possible, it is therefore crucial to align these various allocation entities. Failing this, extra disk I/Os will be generated, causing a perceptible loss in performance. The following figures pictorially depict the benefits of having aligned storage boundaries across the various storage layers: SQL Page VMDK File (NTFS) Cluster Windows Partition VMFS Volume SANLUN MBR Block RAID File system Figure 2: Effect of misaligned file system and small NTFS cluster 7
SQL Page VMDK File (NTFS) Windows Partition MBR Cluster File system VMFS Volume SANLUN RAID Stripe Block Figure 3: Merits of aligned file system and large NTFS cluster Both Microsoft and VMware are very particular about this issue and have independently published their recommendations for ensuring such alignments. In a study published by VMware, it has been claimed that a properly aligned storage subsystem may offer up to a 62% performance advantage. StorTrends assures 64KB storage granularity and as such fits very well into a storage hierarchy serving SQL databases, offering excellent performance along the way. 6. Select an Enterprise Class ESX Server, Preferably ESX 4 There are many compelling reasons for enterprises to move towards a virtual infrastructure. As pointed out earlier, many organizations are now considering the migration of even their Tier 1 applications to a virtual infrastructure. If all other considerations remain constant, just moving applications from dedicated physical systems to a virtual environment would present a noticeable performance dip - one driving factor that would force a serious look at the viability of such migrations. However, ESX 4 from VMware has a number of performance enhancements that make this migration much more feasible. Here are some performance enhancements instituted in ESX 4 that help realize high performance for a virtual SQL server: Scalability Enhancements Increased number of Virtual CPUs per host (eight) 64 Logical CPUs and 512 Virtual CPUs Per Host 64-bit VMkernel VMDirectPath for Virtual Machines: This greatly enhances performance in an I/O intensive environment by allowing the VMs to directly access the underlying hardware. Enhancements in CPU Scheduling Hardware-assisted Memory Virtualization Hardware support is available in current processors from Intel and AMD (called EPT and RVI respectively) to improve memory translation performance in a virtual 8
environment. This feature alone is seen to offer a 10% efficiency improvement for SQL servers running under VM. Storage Enhancements ESX 4 offers a new storage interface called Paravirtual SCSI (PVSCSI). It has multiple I/O scheduling and interrupts coalescing enhancements that dramatically improve I/O performance in a busy system by up to nearly 20%. VMware recommends that a primary adapter is created for the boot disk and a separate PVSCSI adapter for the disk that will store user data (database). iscsi Support Improvements vsphere 4 includes significant enhancements for both software and hardware iscsi drivers. According to VMware, the CPU percentage savings can be as high as 26% for reads and 52% for writes. Running an OLTP benchmark for TPC-E type of workload on a SQL Server with four virtual CPUs on vsphere 4 showed 90% efficiency with respect to the native implementation. Figure 4: vsphere 4 SQL Server throughput vs. native configuration (Courtesy VMware ) Summary: High Availability for SQL Servers in a Virtual Infrastructure Ensuring high availability in a datacenter utilizing virtualization is of paramount importance. The reasons are obvious - component failures, software crashes or similar datacenter issues have the potential to take down a pool of virtual servers, resulting in irreparable losses to the business. To protect against potential losses, ensure that physical systems have redundant components like power supplies, HBAs, NICs and switches. Additionally, application servers must have redundant paths to the shared StorTrends storage appliance. VMware recommends a minimum of four paths to the shared storage. StorTrends should also be configured with a high availability (HA) configuration to ensure the maximum protection against failures in the storage subsystem. 9
Microsoft clustered servers (MSCS) may be used for VMs, but note that this is not a very commonly used configuration because of the cost and complexity. Alternatively, VMware HA and Distributed Resource Scheduler (DRS) may be employed to raise the availability to 99.99% for SQL servers. Keep in mind that a virtual infrastructure may be running other applications servers, and the administrator may prefer to have a homogeneous mechanism to ensure high availability. For SQL servers employing failover clusters, it is necessary to use the Eagerzeroedthick format while creating disks for the VMs (the default format is zeroedthick). Microsoft SQL Servers in the VMware infrastructure may use any of the SQL high availability features which are traditionally used in physical environments, including log shipping, database mirroring, and failover clustering. It is important that the secondary SQL Server (and the Witness VM) is configured on different ESX servers. If VMware DRS is in use, it is recommended to add the DRS anti-affinity rule to address this. Monitoring System Performance In order to ensure proper functioning of the virtual infrastructure, subsystem performance must be carefully monitored. Use VMware tools and especially the VMware Capacity Planner to assess the functionality and performance of one or more SQL Servers inside a VM. Note that monitoring the performance counters with the Windows Perfmon tool is not recommended, as the data shown may be inaccurate in a VM. Instead, it is recommended that you either the VMware esxtop tool or vcenter is used to monitor certain system performance parameters. Subsystem esxtop Counter vcenter Counter CPU %Ready %Used Ready Usage Memory %ACTV SWW/s SWR/s Active Swapin Swapout Network MbRX/s MbTX/s packetsrx packetstx Storage ACTV DAVG/cmd KAVG/cmd Commands devicewritelatency and devicereadlatency kernelwrite Latency and kernelreadlatency The %Ready counter indicates system load, while the %Used counter may reveal overloaded CPU resources. A higher swap rate may indicate shortage of memory resources or improper virtual memory configuration. Higher device latencies are indicative of inadequate disks and/or improper array configuration. A virtualization assessment with VMware Capacity Planner collects vital statistics to analyze and provide a feel for what is going on in the virtual infrastructure during run time. Certain counters that of special interest for SQL Virtualization deployment are noted below: SQL Server Access Method Counters Full scans/sec Index searches/sec Page splits/sec Table lock escalations/sec 10
SQL Server Buffer Manager Counters Buffer cache hit ratio Free pages Lazy writes/sec Page life expectancy Procedure cache pages Read-ahead pages/sec Stolen pages Total pages SQL Server Database Counters Data file(s) size(kb) Log file(s) size(kb) Transactions/sec SQL Server Statistics User connections Batch requests/sec SQL compilations/sec SQL recompilations/sec SQL Server Memory Manager and Locks Connection memory(kb) Granted workspace memory(kb) Lock memory(kb) SQL cache memory(kb) Target Server memory(kb) Total Server memory(kb) Average wait time(ms) Number of deadlocks/sec In addition to the above counters, there are a number of counters and SRM charts featured in the ManageTrends GUI in every StorTrends Storage Appliance that can greatly assist in monitoring the performance of SQL virtualization. Summary Application Server and Storage Server Virtualization have made significant inroads into the IT infrastructure. Relatively speaking, the late entrants into trend towards increasing virtualization are performance-critical first tier business applications. In response, considerable research and a number of case studies have been conducted to show that migration of Tier 1 applications to virtual infrastructure is not only feasible, but also in fact recommended practice. Without a doubt, such an IT infrastructure can quickly become exceedingly intricate, and following and maintaining certain guidelines can ensuring the proper functionality of the datacenter. The use of VMware may make operation more complex due to the interaction of the various VMs sharing valuable system resources. For this reason, it is best to use the latest VMware ESX 4 code running on latermodel server hardware. Because of the potential for conflict between multiple VMs sharing the same physical resources, it is recommended to monitor and analyze the various parameters using VMware 11
tools like Capacity Planner. Similarly, StorTrends offers state-of-the-art features and Storage Resource Management (SRM) tools to help the IT administrator in properly configuring, analyzing and managing the shared storage pool. Finally, guaranteeing service availability is of utmost importance - especially in a virtual infrastructure. The reason is obvious: a simple component failure could potentially throw a pool of virtual servers out of gear. One must therefore plan and deploy the DR strategies with careful consideration. There are a large number of options and alternatives like using Microsoft MSCS clusters, VMware high-availability configurations like HA, DRS, FT (for vsphere 4), so one must wisely choose options suited to the particular environment and needs. No matter what the final selection is for the application server high availability solution, StorTrends high availability and other data protection features like remote replication will complement the selection and work seamlessly with the virtualized environment to ensure end-to-end high availability for the datacenter. Note: VMware, ESX, Capacity Planner, vcenter, and other marks are the property of VMware in the United States and worldwide. Microsoft, Windows, SQL Server and other marks are the property of Microsoft in the United States and worldwide. 12