EMC CLARiiON Database Storage Solutions: Microsoft SQL Server 2008 in Virtualized Environments

Transcription

1 EMC CLARiiON Database Storage Solutions: Microsoft SQL Server 2008 in Best Practices Planning Abstract This technical white paper explains best practices associated with Microsoft SQL Server 2008 when deployed on EMC CLARiiON family storage platforms in virtualized environments. Storage considerations for database deployments with VMware and Hyper-V virtualization technologies are discussed. August 2009

2 Copyright 2009 EMC Corporation. All rights reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com All other trademarks used herein are the property of their respective owners. Part Number h6523 Best Practices Planning 2

3 Table of Contents Executive summary...4 Introduction...4 Audience... 5 Terminology... 5 Microsoft SQL Server 2008 considerations...6 Best practices... 6 Microsoft Windows Server 2008 considerations...10 Best practices Virtual infrastructure considerations...10 General virtualization best practices Best practices for VMware Best practices for Microsoft Hyper-V Network considerations...14 Best practices Storage system considerations...15 Best practices CLARiiON considerations...18 Best practices Storage-array caching considerations Disk type considerations RAID type considerations Conclusion...22 References...22 Related documents Best Practices Planning 3

4 Executive summary With the release of VMware vsphere 4 and Microsoft Hyper-V, virtual hardware support was extended to include the use of storage devices. This support enables vsphere and Hyper-V environments to take full advantage of EMC CLARiiON storage using the Fibre Channel and iscsi protocols. This provides a method to tie virtualized computing to virtualized storage, offering a dynamic set of capabilities within the data center and resulting in improved performance and system reliability. EMC is aggressively expanding product sets from high-end to midtier markets by extending support for virtualized environments to include VMware vsphere 4 and Microsoft Hyper-V with EMC CLARiiON. CLARiiON offers Fibre Channel and iscsi LUNs as storage options for creating virtual machines and virtual disks on VMware vsphere and Microsoft Hyper-V servers. CLARiiON also provides advanced scalability and reliability by combining the benefits of Fibre Channel with the flexibility and ease of use of Enterprise Flash Drive, FC, and iscsi storage. Microsoft SQL Server underpins many of the latest financial, manufacturing, e-commerce, and collaborative applications from leading third-party vendors including Microsoft. Many businesses have also developed in-house applications using SQL Server databases. As companies adopt SQL Server-based solutions for increasingly important business applications, it is critical that these solutions protect the integrity of both the data as well as the operational environment. EMC CLARiiON storage arrays, with their advanced information availability and replication features, are ideally suited for implementing SQL Server solutions. CLARiiON supports five to 960 disk drives, supports Fibre Channel or iscsi host connections, and accommodates SAN, NAS, and DAS environments. CLARiiON storage arrays are ideally suited for SQL Server solutions because they deliver high availability through a no-single-point-of-failure design; advanced information replication with multiple choices - local or remote, and synchronous or asynchronous; powerful storage management with EMC Navisphere ; and exceptional affordability by mixing ATA and Fibre Channel drive technology within the same system. The true differentiation of CLARiiON, however, can be found in its bulletproof data integrity handling. Maximizing the effectiveness of CLARiiON storage arrays in a SQL Server environment takes careful planning and understanding of database storage considerations. Understanding the database environment and required service levels all factor into proper database storage-system design. Introduction This white paper documents recommendations and guidelines associated with the operations of EMC CLARiiON storage arrays and Microsoft SQL Server It is important to plan a SQL solution that can grow while maintaining optimum performance, high availability, and disaster recovery. This paper is meant to be a resource for providing guidance in configuring for virtualized environments for SQL Server deployments and for optimizing the performance for SQL Server 2008 storage configuration on EMC CLARiiON via FC and iscsi. This paper is divided into six sections: Microsoft SQL Server 2008 considerations Microsoft Windows Server 2008 considerations Virtual infrastructure considerations Network considerations Storage system considerations CLARiiON considerations Best Practices Planning 4

5 Audience The intended audience includes IT administrators, database administrators, data architects, and system engineers who have an interest in implementing Microsoft SQL Server 2008 on EMC CLARiiON systems. Terminology Active Directory: An advanced directory service introduced with Windows 2000 Server. It stores information about objects on a network and makes this information available to users and network administrators through a protocol such as LDAP. Bandwidth: The amount of data a storage system can process over time, which is measured in megabytes per second. CLARiiON consistency: The logical concept of designating (and enforcing) a group of CLARiiON storage LUNs to be replicated by the replication software as a coherent set. The replication action is performed automatically for all the members in the set. Either all members will be replicated, or none will be replicated. Enterprise Flash Drives (EFD): Also known as solid state drives (SSD), EFDs contain no moving parts, which removes much of the storage latency associated with traditional magnetic disk drives. Fibre Channel (FC): A technology for transmitting data between storage devices and computer devices at data rates of up to 4 Gb/s on a dedicated network (optical fiber, or for short runs, coaxial cable or twisted pair). Guest operating system: An operating system that runs within a virtual machine or child partition. Hyper-V Manager: A Microsoft Management Console (MMC)-based management interface that allows administrators to perform basic operations such as provisioning or managing virtual machines. It is installed when a Hyper-V Server role is selected on Windows Server 2008 full installation. The Windows Server Core version does not come with the GUI interface. Hypervisor: A virtualization platform that allows multiple virtual machines to run on a physical host at the same time. Also known as a virtual machine monitor. Logical unit number (LUN): A unique identifier used to identify logical storage objects in a storage system. RAID: Also known as Redundant Array of Independent Disks, it is a way of storing the same data in several places on multiple hard disks. I/O operations can overlap in a balanced way, thereby improving performance. Since multiple disks increase the mean time between failures, storing data redundantly also increases fault tolerance. A RAID storage system appears to the operating system to be a single logical hard disk device. RAID 1: RAID method that provides data integrity by mirroring (copying) data onto another disk. This RAID type provides the greatest assurance of data integrity at the greatest cost in disk space. RAID 5: Data is striped across disks in large stripes. Parity information is stored so data can be reconstructed if needed. One disk can fail without data loss. Performance is good for reads, but slower for writes. RAID 6: RAID 6 is essentially an extension of a RAID 5 with a second independent distributed parity scheme. Data and parity are striped on a block level across multiple array members, just like in RAID 5, and a second set of parity is calculated and written across all the drives. This provides data protection in situations where two disks fail. Response time: The interval of time between submitting an I/O request and receiving a response. SP: Storage processor on a CLARiiON storage system. On a CLARiiON storage system, a circuitboard with memory modules and control logic that manages the storage-system I/O between the host s Fibre Channel adapter and the disk modules. SP A: Storage processor A. A generic term for the first storage processor in CLARiiON storage system. SP B: Storage processor B. A generic term for the second storage processor in CLARiiON storage system. Best Practices Planning 5

6 System database: A database that is installed as part of the installation of Microsoft SQL Server. The system databases include master, model, msdb, tempdb, and others. Throughput: The number of individual I/Os the storage system can process over time, which is measured in I/Os per second. User database: A non-system database that is put on the server after the installation of Microsoft SQL Server. Examples include an OLTP application database or data warehouse. Virtual local area network (VLAN): A group of devices that reside on different network segments but communicate as if they were on the same network segment. VLANs are configured by management software based on logical (instead of physical) connections to increase administrative flexibility. Virtual LUN technology: A technique whereby data stored on a LUN is bitwise migrated to another LUN within the same storage system, without restricting access to the data. Virtual machine: A virtualized x86 or x64 PC environment on which a guest operating system and associated application software can run. Multiple virtual machines can operate on the same physical machine concurrently. Volume Shadow Copy Service (VSS): A Windows Service that provides an infrastructure that enables third-party storage management programs, business programs, and hardware providers to create and manage consistent point-in-time copies of data, called shadow copies. Microsoft SQL Server 2008 considerations Like any other database, SQL Server manages data using database files and transaction log files. The database files are frequently accessed with random reads and writes. Transaction log files, on the other hand, consist mostly of sequential write operations, with occasional reads of recently written transaction records. The SQL Server tempdb database is read/write-intensive and is used for storing temporary tables, temporary stored procedures, and sub-queries, and for sorting aggregate operations. Best practices Plan for storage performance, not for capacity The most common error made while planning the storage for Microsoft SQL Server is designing for storage capacity and not for performance or I/Os per second (IOPS). With advances in disk technology, the increase in storage capacity of a disk drive has outpaced the increase in IOPS by almost 1,000:1. With this effect it is rare to find a system that, when planned for performance, does not meet the storage capacity requirements for the workload. Hence, the IOPS capacity is the standard to be used while planning SQL Server storage configurations. Only after considering the IOPS capacity of a configuration should the storage capacity (GB) be considered. Spread database files on RAID group consisting of more disk drives It is more I/O efficient to use a RAID group containing several small disks than a RAID group with few large disks. Place SQL Server transaction log and database files on physically separate RAID groups Placing SQL Server database and transaction log files on physically separate RAID groups avoids disk contention. It also decreases the probability of disk failures that cannot be handled by RAID, which will damage both your database and transaction log files. This by extension improves the available recovery options. Place transaction log files on RAID 1/0 volumes RAID 1/0 volumes provide excellent write performance as well as fault tolerance and is an ideal choice for the placement of sequentially write-intensive SQL Server transaction log files. RAID 1/0 also rebuilds faster than RAID 5 in case of a disk failure. Place database files on a volume with an appropriate RAID type Choose RAID 1/0 volumes for OLTP-type applications with highly active database files. Best Practices Planning 6

7 Choose RAID 5 for random I/O only if writes are less than 30 percent of the workload. RAID 5 can be used for data warehouse type applications with mostly large data reads. Choose RAID 6 if high availability is important and the application is willing to tolerate reduced disk performance. Consider placing database files on metaluns for improved performance, ease of expandability, and fast rebuild. Use SQLIOSim.exe to validate storage configuration When implementing any structural change to the storage subsystem used for SQL Server, we recommend that you validate that the system can support SQL Server loads. This includes the disk array, storage network, and any software that interacts with the I/O path. Microsoft publishes the SQLIOSim utility for this purpose. Prior to SQL Server installation, or database deployment, download and install SQLIOSim.exe from Microsoft as documented in KB The SQLIOSim utility simulates the I/O patterns, and the problem identification methods used by SQL Server in order to expose potential data integrity issues. It should not be considered as a performance benchmarking tool. Use MSCS or VMHA for high availability A cluster is a collection of servers known as nodes that together provide a single, highly available system for hosting applications such as Microsoft SQL Server Microsoft clusters provide a highly available environment that can protect against SQL Server 2008 server failures of hardware, operating systems, device drivers, or applications. If one of the nodes in a cluster is unavailable as a result of a failure, the service fails over to another node, which begins providing service in place of the failed node. It is important to note that not all cluster configurations are supported in a virtual machine. Both 64-bit and 32-bit Microsoft clusters are supported by VMware from update 1. Note: For more information, refer to: SQL Server 2008 failover clusters are supported with Hyper-V for Windows Server 2008 in a x64- based system. Note: For more information, refer to An alternative is to use VMware s built-in HA capabilities, which can achieve a similar level of high availability to Microsoft clustering. Make SQL Server part of an Active Directory domain The primary recommended method for security and account management in SQL Server is through Active Directory domain user accounts, using integrated security. This allows for greater security, at multiple levels, and makes user management easier. SQL Server should not be a domain controller, except in certain unusual circumstances. The added overhead of being a domain controller is likely to have a negative impact on SQL Server performance. Enable SQL Server to keep pages in memory Microsoft SQL Server dynamically allocates and deallocates memory based on the current state of the server, in an attempt to prevent memory pressure and swapping. However, if a process suddenly attempts to grab a substantial amount of memory, SQL Server may not be able to react quickly enough and the OS may swap some of SQL Server s memory to disk. Unfortunately, there is a good probability that the memory that was swapped to disk contains part of what SQL Server will soon be deallocating to decrease its memory use in response to the newly created memory pressure. It is recommended that SQL Server be enabled to prevent its memory from being swapped. This is known as Locking pages in ram. To do this, the account that the Microsoft SQL Server service is running under must be given the Lock pages in memory user right. Note: For more information, refer to Enable Windows fast file initialization Best Practices Planning 7

8 When Microsoft SQL Server creates or expands a file, the file must be initialized. Previous versions of SQL Server had only one option and that was to initialize the space by writing all zeros to the space, which would cause a substantial performance impact if a file growth occurred. SQL Server 2008 supports fast file initialization, which sets a file end pointer, and the process is then complete. This operation is nearly instantaneous and minimizes the impact of file growth on production systems. Fast file initialization is enabled at the OS level, by granting the user right Perform volume maintenance tasks to the account under which SQL Server service is running. By default, this right is granted to administrators. Set your database file sizes and autogrow increments appropriately Microsoft SQL Server 2008 supports the ability to automatically grow both data and log files as they fill. However, this should not be misconstrued as a method of database sizing. It is a best practice to set the file sizes appropriately and grow them manually at times of minimal system use, on a planned basis. Autogrowth should be used only as a safety net to prevent the files from becoming full and making the database read-only, at times when unpredicted substantial growth occurs. When database files are expanded there is an impact to performance. This impact is minimized but not eliminated through fast file initialization. Additionally, the file autogrowth increments should be set such that the time it takes for the growth to occur is short enough to minimize its impact to performance, but large enough to prevent many small allocations that invite file fragmentation. An adequate increase in file size that prevents fragmentation usually impacts the performance of the database. Hence, it is recommended that the file sizes be changed during periods of lowered activity on the database. Log files have an additional issue, as there are virtual log files within a physical log file, and a virtual log file cannot span file growth increments. Thus, if the log file were set to grow at 1 MB increments, then the virtual log file would not be able to exceed 1 MB either. This will have a performance impact as discussed previously due to the impact of file expansion. This limit may also make certain transactions impossible to complete. For all files, because of the impact to performance, it is recommended that an absolute growth increment (in MB or GB) be used instead of a percentage growth. It is also recommended that autoshrink should never be enabled. A file should not be shrunk, unless absolutely necessary, and only through a controlled manual action. Plan your database filegroups based on your workload SQL Server provides lots of options about how and where to lay out database tables and other structures on disk. The primary structure to control this behavior is a filegroup. Database structures are assigned to filegroups, which contain files on disk where that data can be stored. The placement of these data files is critical to the I/O performance of your database, and the recommendations for the best ways to set up filegroups vary based on the database workload. Consider table and index partitioning Partitioning breaks up database objects by allowing subsets of the data to reside on separate filegroups. There are numerous ways that this ability can be helpful in a SQL environment. Improved manageability: Partitioning makes large tables or indexes more manageable. Maintenance operations can be performed on subsets of data and target only the data that is required instead of the whole table. This can be used for numerous improvements including shortened maintenance windows, improved backup coverage, and reduced backup storage requirements. Reduced costs: In some types of environments it is common to have very large tables that contain historical data. This data may be less valuable than current data, but still required for the application. Using partitioning allows sections of the table to be moved to lower-cost storage without impacting the whole table. For example, you could have the active part of a table using Enterprise Flash Drives or Fibre Channel (FC) disks, while the historical part of a table uses Serial ATA (SATA). Improved availability: Partitioning enables database partial availability, which can be used in some environments to reduce downtime due to planned, and unplanned, events. It also enables partial Best Practices Planning 8

9 restores so that if a section of a table is damaged it can be restored from backup without impacting the other areas of the database. Table and index partitioning can allow improved manageability, reduced costs, and improved availability, but they must be planned and implemented based on the needs of each individual environment. If they are implemented improperly, or without consideration for the environment they may actually have the opposite effect. Plan the location, layout, and size of your tempdb By default the tempdb database is rather small and gets its characteristics from the model database. Each time the Microsoft SQL Server service is started; tempdb is dropped and re-created with its initial parameters. Thus, if tempdb is initially 128 MB and during operations it autogrows to 4 GB, on restart it will be 128 MB again. Then it will have to go through the autogrow again, which will impact the performance of your database. To minimize this impact it is recommended that tempdb be sized appropriately for the environment. The easiest way to size the tempdb database is the following: 1. Start with a reasonable size tempdb for the size of databases that are in the same SQL Server instance. For example, a 1 GB tempdb database is a reasonable starting place for a sum total of instance databases between 10 GB and 100 GB, but not for 1 TB. A good starting place is to sum the total size of the databases in the instance and size tempdb between 1 percent and 10 percent of that size. 2. Set a valid autogrow increment that will allow tempdb to grow without heavy fragmentation. The best way to do this is to set autogrow to 10 percent to 20 percent of the tempdb initial size. Do not use a percentage for the growth parameter; calculate the MB growth that corresponds to the percentage and set that as the autogrowth size. You should also make sure that fast file initialization is enabled. 3. Periodically verify the size and utilization of the tempdb database to see if it has grown significantly. 4. Reset the size of the tempdb database to something close to its size, before a shutdown. If our tempdb database from the previous example had grown from 1 GB to 5 GB, then resetting it to start at 5 GB would be advantageous, unless the new size is obviously excessive. For example, if the sum total of user databases was 10 GB and tempdb was 15 GB, this would seem excessive. It is possible that an odd set of scenarios came together to cause uncharacteristic tempdb growth. If you suspect that this may be the case, then the starting size should be set to something smaller than the current size. If the tempdb repeatedly grows to larger than is initially considered reasonable, then it is possible that this is simply the size of tempdb that is needed for your workload. From here, a DBA could diagnose what is causing the excessive growth and then determine if it is valid, or if anything needs tuning. It is recommended that tempdb be placed on its own spindles, where tempdb and user database activity cannot cause physical disk contention with each other. The number of spindles will be determined on a case-by-case basis using the same principles that are applied to designing storage for user databases. Use defaults for processors and memory When Microsoft SQL Server is first installed, most of its tunable parameters are set to automatic and it is recommended that, on a server dedicated to SQL Server s use, these parameters should be left at their automatic defaults. The only time they should really be changed is if there are other workloads running on the same server, or if issues arise from the use of the defaults. By default SQL Server will run at a standard priority and make all processors in the system available for use. Also by default, SQL Server will use as much memory as it needs until it notices that memory pressure is starting to build. If other processes start consuming memory, SQL Server will begin decreasing its memory footprint appropriately to decrease the possibility of swapping occurring. Use failover-aware applications When a Microsoft SQL Server failover occurs, using MSCS clustering, Database Mirroring, or other technologies, all database connections are lost and any in-flight transactions are rolled back. To minimize data loss, it is recommended that all applications be failover-aware and have reconnect/retry Best Practices Planning 9

10 logic. Thus, in case of a failover, the application will attempt to reconnect, and once it successfully reconnects, it will retry the transaction that was previously rolled back. Disable hyperthreading on Microsoft SQL Server 2008 servers Intel hyperthreading technology allows multithreaded operating systems to view a single physical processor as if it were two logical processors. A processor that incorporates this technology shares CPU resources among multiple threads. In theory, this enables faster enterprise-server response times and provides additional CPU processing power to handle larger workloads. As a result, server performance will improve. However, testing has shown that hyperthreading can have a negative impact on many Microsoft SQL Server 2008-based processor loads. Unless it can be proven that hyperthreading helps the performance of a particular SQL Server 2008 workload, it is recommended that hyperthreading be disabled. Hyperthreading must be disabled at the hardware (BIOS setting) level, not through the application of processor affinity or other software means. Note: For more information, refer to Microsoft Windows Server 2008 considerations This section details recommendations for the configuration of Windows Server 2008 for use with a Microsoft SQL Server instance. Best practices Only use hardware that is approved by Microsoft Using hardware that is on the Windows Hardware Compatibility List (WHCL) decreases the possibility of compatibility problems and increases the level of support that Microsoft will provide, should a problem occur. Note: In the case of using Windows on VMware ESX, use only hardware that is approved by VMware. Use the latest verified NIC driver For best performance and stability it is recommended to install the latest vendor NIC driver that has been validated for use with Windows Server Note: In the case of using Windows on VMware ESX, use the latest driver that is approved by VMware. When using MSCS, reboot the passive node occasionally Many times configuration changes, especially disk configuration changes, may not be detected by the passive node until a reboot is completed. If the passive node has not detected these changes then a cluster failover may not succeed. Use a dedicated VLAN for cluster heartbeat connectivity If MSCS is used, it is recommended that the cluster heartbeat network be physically isolated from other networks. For example, in a two-way cluster it is common for a crossover cable to be used between the two machines as the heartbeat network. Virtual infrastructure considerations Virtualization has been a familiar term used in technical conversations for many years now. We have grown accustomed to almost all things virtual, from virtual memory, to virtual networks, to virtual storage VMware vsphere 4 contains numerous performance-related enhancements that make it easy to virtualize a resource-heavy database with minimal impact to performance. The improved resource management capabilities in vsphere facilitate more effective consolidation of multiple SQL Server virtual machines (VMs) on a single host without compromising performance or scalability. Greater consolidation can significantly reduce the cost of physical infrastructure and of licensing SQL Server, even in smaller-scale environments. Best Practices Planning 10

11 In 2008, Microsoft released Hyper-V, its first bare-metal hypervisor-based technology, built into Windows Server The goal of Hyper-V is to provide a framework for companies to better partition and utilize their hardware resources, and to compete in the virtual server space with products like VMware ESX Server and Citrix XenServer. With the Hyper-V architecture, hardware-assisted, x64-based systems can run independent virtual environments, with different operating systems and resource requirements, within the same physical server. Several Microsoft products, such as System Center Virtual Machine Manager (SCVMM), Data Protection Manager (DPM), and Performance Resource Optimization (PRO), can virtualize hardware resources. However, the virtualization guidance provided in this white paper focuses on the use of VMware vsphere 4 and Hyper-V Server to virtualize SQL Server 2008 environments. Both options are supported by EMC for use with CLARiiON so it is the user s responsibility to determine which virtualization technology best meets their specific deployment requirements. General virtualization best practices This section provides guidance regarding a number of general performance considerations. Spread I/O loads over the available paths to the storage (across multiple HBAs) and storage processors (SPs). Applications or systems that write large amounts of data to storage, such as data acquisition or transaction logging systems, should not share Ethernet links to a storage device. These types of applications perform best with multiple connections to storage devices. For iscsi, make sure that your network topology does not contain Ethernet bottlenecks, where multiple links are routed through fewer links, potentially resulting in oversubscription and dropped network packets. Any time a number of links transmitting at near capacity are switched to a smaller number of links, such oversubscription is a possibility. Recovering from dropped network packets will result in a large performance degradation. In addition to time spent determining that data was dropped, the retransmission uses network bandwidth that could otherwise be used for new transactions. When performing cloning of virtual machines, make sure a new SID is generated. This can be accomplished by running sysprep or executing the newsid utility. Note: For more information visit Make sure that file system partitions within the guest are aligned. Performance design for a storage network must take into account the physical constraints of the network, not logical allocations. Using VLANs or VPNs does not provide a suitable solution to the problem of link oversubscription in shared configurations. VLANs and other virtual partitioning of a network provide a way of logically configuring a network, but don't change the physical capabilities of links and trunks between switches. Multiple physical network adapters between a single virtual switch (vswitch) and the physical network constitute a NIC team. NIC teams can provide passive failover in the event of hardware failure or network outage and, in some configurations, can increase performance by distributing the traffic across those physical network adapters. Unused or unnecessary virtual hardware devices can impact performance and should be disabled. Carefully select the amount of memory you allocate to your virtual machines. You should allocate enough memory to hold the working set of applications you will run in the virtual machine, thus minimizing swapping, but avoid overallocating memory. Allocating more memory than needed unnecessarily increases the virtual machine memory overhead, thus using up memory that could be used to support more virtual machines. To establish a network connection between two virtual machines that reside on the same ESX system, connect both virtual machines to the same virtual switch. If the virtual machines are connected to different virtual switches, traffic will go through a wire and incur unnecessary CPU and network overhead. Best Practices Planning 11

12 Disable screensavers and Window animations in virtual machines. It is best to create a separate gigabit interface on a different subnet for each network usage to isolate traffic (for security and performance reasons). Use one or more gigabit network interfaces to access IP storage targets. Do not share physical network interfaces that are used by external VM networks with the management console or IP storage to avoid conflicts between them. Best practices for VMware This section details recommendations when using VMware vsphere 4 to virtualize the SQL Server environment. Use VMware vcenter Server The management of the virtual data center is made much easier and more featureful, through the use of VirtualCenter Server. VC Server is important for the proper functioning of several VM features such as VMHA, DRS, and VMotion. Also, VC Server adds ease of use to the following features and many others: VM Template creation and deployment throughout the data center Historical and realtime performance tracking of ESX hosts and VMs Tracking of VM-to-host mapping Initialization of new storage, including alignment of VMFS volumes Make vcenter Server highly available The most critical piece of any VMware ESX implementation is a combination of the VirtualCenter server and the license server (together referred to as the vcenter Server). Without these two components, which are usually installed on the same host, it is possible to lose much of ESX s functionality, including the ability to do things like DRS, VMHA, VMotion, and/or simply starting a VM. Therefore, it is important that the vcenter Server be highly available. The easiest and most reliable way to accomplish this is to run the vcenter Server on physical machines in an MSCS cluster. It is also critical that the Microsoft SQL Server database that the vcenter Server uses be implemented for high availability. Be aware of virtual machine time considerations Due to the way the hypervisor-based virtualization solutions virtualize, the clock tick coming from the physical processor and some measurements of time can be skewed in the virtual machine. There are numerous ways that this can impact a system, and there are many ways to address the situation. As a best practice, refer to all the relevant documentation from your hypervisor provider to understand how it can impact your environment. For example, VMware has a guide on timekeeping called Timekeeping in VMware Virtual Machines at: Configure maximum queue depth for HBA cards The driver queue depth can be set for some VMkernel drivers. For example, while the default queue depth of the QLogic driver is 32, specifying a larger queue depth may yield higher performance. You can also adjust the maximum number of outstanding disk requests per virtual machine in the VMkernel through the vsphere Client. Setting this parameter can help equalize disk bandwidth across virtual machines. Allocate only as much virtual hardware as required for each virtual machine Provisioning a virtual machine with more resources than it requires can, in some cases, reduce the performance of that virtual machine as well as other virtual machines sharing the same host. Disconnect or disable unused or unnecessary physical hardware devices, such as: COM ports LPT ports USB controllers Best Practices Planning 12

13 Floppy drives Optical drives (that is, CD or DVD drives) Network interfaces Storage controllers Disabling hardware devices (typically done in BIOS) can free interrupt resources. Additionally, some devices, such as USB controllers, operate on a polling scheme that consumes extra CPU resources. Lastly, some PCI devices reserve blocks of memory, making that memory unavailable to ESX. Use separate virtual switches for the ESX service console, VMkernel, and virtual machines Use separate virtual switches each connected to its own physical network adapter to avoid contention between the ESX service console, the VMkernel, and virtual machines, especially virtual machines running heavy networking workloads. Use guest operating systems that are supported by ESX See the VMware Guest Operating System Installation Guide for a list. Note: VMware Tools might not be available for unsupported guest operating systems. Use vsphere Client to create VMFS partitions The alignment of your file system partitions can impact performance. VMware makes the following recommendations for VMFS partitions: Like other disk-based file systems, VMFS suffers a penalty when the partition is unaligned. Using the vsphere Client to create VMFS partitions avoids this problem since it automatically aligns the partitions along the 64KB boundary. To manually align your VMFS partitions, check your storage vendor s recommendations for the partition starting block. If your storage vendor makes no specific recommendation, use a starting block that is a multiple of 8KB. Multiple heavily used virtual machines concurrently accessing the same VMFS volume, or multiple VMFS volumes backed by the same LUNs, can result in decreased storage performance. Appropriately configured storage architectures can avoid this issue By default, Active/Passive storage arrays use the Most Recently Used path policy. Do not use the Fixed path policy for Active/Passive storage arrays to avoid LUN thrashing. Make sure the system has enough memory to avoid ESX host swapping, otherwise performance in the virtual machines is significantly reduced. Pay attention to the balloon driver s inflation, which induces swapping in the guest. Information is available at Install the latest version of VMware Tools in the guest operating system. Make sure to update VMware Tools after each ESX upgrade. Installing VMware Tools in Windows guests updates the BusLogic SCSI driver included with the guest operating system to the VMware-supplied driver. The VMware driver has optimizations that guest-supplied Windows drivers do not. VMware Tools also includes the balloon driver used for memory reclamation in ESX. If you choose to use the BusLogic virtual SCSI adapter, and are using a Windows guest operating system, you should use the custom BusLogic driver included in the VMware Tools package. The default storage adapter in ESX 4.0 is either BusLogic or LSILogic, depending on the guest operating system. However, ESX 4.0 also includes a new virtual storage adapter, paravirtualized SCSI (PVSCSI, also called VMware Paravirtual). PVSCSI adapters offer a significant reduction in CPU utilization as well as potentially increased throughput compared to the BusLogic or LSILogic virtual storage adapters, and are thus the best choice for environments with very I/O-intensive guest applications. For additional details regarding VMware deployments using EMC CLARiiON storage, reference EMC s VMware ESX Server Using EMC CLARiiON Storage Systems TechBook. Best Practices Planning 13

14 Best practices for Microsoft Hyper-V This section details recommendations when using Microsoft Hyper-V to virtualize the SQL Server environment. In an I/O-intensive environment such as an OLTP/DSS database workload, we strongly recommended that you utilize fixed size VHDs or pass-through disk to achieve best performance. Do not use dynamically expanding disks for a production workload, as a performance penalty is imposed as a result of data blocks being allocated on demand. It may be difficult to distinguish one interface from another if there are multiple interfaces, especially when new interfaces are added during virtual network creation. Be sure to rename and label each interface in Network Connections to make it easier to identify their designated purposes. Segment network traffic for IP storage through a private LAN using either a VLAN or an independent IP SAN switch. For best performance, configure two iscsi connections per session via two separate subnets, and choose the Round Robin load-balance policy. Use guest operating systems that are supported by Hyper-V. Note: For more information, refer to For additional details regarding Hyper-V deployments using EMC CLARiiON storage, reference Using EMC CLARiiON with Microsoft Hyper-V Server Applied Technology and the Microsoft white paper Running SQL Server 2008 in a Hyper-V Environment. Hyper-V guest virtual machines are limited to a maximum of four CPU cores; therefore, you should run SQL Server within Hyper-V guest virtual machines only if your workload performance can be satisfied by no more than four CPUs. Network considerations This section details recommendations for the configuration of your IP networks for use with Microsoft SQL Server Best practices Use Gigabit Ethernet switches with VLAN capabilities For best performance, use GbE switches that are capable of setting up virtual LANs (VLAN) to segment different kinds of traffic. Use CAT6 cables for GbE connectivity Use CAT6 cables for best performance and reliability. CAT6 cables show superior results when compared to CAT5E cables when used for 1000 Mb connectivity. Set network speed and duplexing Once the setup is completed and it has been verified that the infrastructure supports GbE properly, then the switch ports and NIC ports should be configured to 1 Gb/s and FULL duplex. During setup it may be necessary to use AUTO settings to ensure that everything works properly in a new environment, however, the proper speed and duplex settings should be set explicitly in production systems. Plan for network high availability One common oversight is to provide for high availability at the server level using clusters and the storage level using RAID, but then forget about the network connecting them. To ensure uninterrupted communication between systems in your environment, plan your networks for high availability. This includes redundant switches and paths as well as redundant NIC ports and cards. Best Practices Planning 14

15 Storage system considerations Storage performance issues are most often the result of configuration issues with underlying storage devices and are not specific to ESX. Storage performance is a vast topic that depends on workload, hardware, vendor, RAID level, cache size, stripe size, and so on. Consult the appropriate documentation from VMware as well as the storage vendor. Many workloads are very sensitive to the latency of I/O operations. It is therefore important to have storage devices configured correctly. The remainder of this section lists practices and configurations recommended by VMware for optimal storage performance. This section details recommendations for the configuration of your EMC storage system for use with Microsoft SQL Server Best practices Plan storage layouts for performance, not for capacity The most common error made while planning storage for Microsoft SQL Server is designing for capacity and not for performance or I/Os per second (IOPS). To properly plan the disk layout there must be an estimate as to the number of IOPS that need to be supported on a sustained basis, the peak IOPS, and the duration of the peak. Many customers gather data while the application is running, then use a 90th percentile to determine the level that should be planned for. There are three primary variables used for determining the number of spindles for database storage: IOPS (or sometimes MB/s, if a serial workload) RAID level When planning for performance, striped RAID 1 (RAID 10) will require fewer spindles than RAID for almost all read/write workloads. They are approximately equal in a readonly workload. Latency goals Table 1 shows the Microsoft guidelines for good performance with SQL Server 2008: Table 1. Latency goals Read Write Average latency 20 ms 10 ms Max latency 50 ms 50 ms With advances in disk technology, the increase in storage capacity of a disk drive has outpaced the increase in IOPS by almost 1,000:1. As a result, it is rare to find a system that does not meet the storage capacity requirements for the workload. Therefore, IOPS capacity is the standard to be used while planning SQL Server storage configurations. Only after considering the IOPS capacity of a configuration should the storage capacity (GB) be considered. Use diskpart to align your LUNs for best performance It is recommended to align your disk partition using diskpart. When a Windows partition is created, it is created starting at the 64th sector. This misaligns the partition with the physical disk, which can cause the I/O operation to straddle stripe element boundaries and result in a significant reduction in performance. Performance is improved as much as 40 percent when diskpart is used to align. Note: In-depth discussion on this subject can be found in the Using diskpar and diskpart to Align Partitions on Windows Basic and Dynamic Disks white paper. After the LUN creation process is complete on the production system, the active MSCS node should be able to see the LUN as a raw volume. Partition the LUN using the Microsoft command line utility DISKPART, ensuring that the partition is created using the ALIGN=64 switch. The following example uses diskpart against drive 4: Best Practices Planning 15

16 C:\>Diskpart Microsoft DiskPart version Copyright (C) Microsoft Corporation. On computer: JC27Q91X32 DISKPART> list disk Disk ### Status Size Free Dyn Gpt Disk 1 Online 136 GB 112 GB Disk 2 Online 267 GB 0 B Disk 3 Online 267 GB 0 B Disk 4 Online 600 GB 600 GB DISKPART> select disk 4 Disk 4 is now the selected disk. DISKPART> create partition primary align=64 DISKPART succeeded in creating the specified partition. Using the Microsoft Disk Manager select the drive letter or mount point to be associated with the corresponding LUN. After selecting this information, format the drive NTFS at 64k Allocation Unit Size for database data and log files. Note: This is no longer required with Windows Align new storage volumes on virtualized servers When using a virtualized server, there are additional layers of abstraction that must be considered for alignment. All of the layers between the host application and the storage must be aligned, including NTFS, and VMware s VMFS. It is still necessary to align NTFS volumes by following the instructions. If any layer in the stack is misaligned, performance will decrease. Since VMFS is automatically aligned when the volume is created by the VMware VI client, that is the recommended method for creating a new VMware VMFS volume. Note: See the VMware white paper Recommendations for Aligning VMFS Partitions for more information. Set the NTFS allocation unit to 64 KB When formatting a new drive using Disk Administrator, the performance will be affected by the allocation or block size that is chosen. For Microsoft SQL Server 2008, Microsoft recommends using a 64k block size. Note: For more information refer to Do not exceed 80% utilization of LUNs For the best performance, the utilized drive (NTFS formatted) capacity must not exceed 80 percent. There will be performance bottlenecks if this threshold is exceeded. This is because NTFS Best Practices Planning 16

17 needs additional space to work efficiently. If the space is not available, NTFS cannot function to its full potential and performance can degrade. This situation may spur additional performance degradation by creating excessive disk fragmentation. If an FC/iSCSI LUN that is servicing Microsoft SQL Server 2008 reaches 80 percent utilized drive capacity, do one or more of the following: Remove unnecessary data from the disk. Move some of the data to disks with more space. Add more disk space. Following this recommendation will also provide you with some protection against application failure if there is unexpected growth in your database. Use clone replicas for testing and development When testing changes to an application where access to production data is required, but modification to that data is not, it is common to create a replica of the database so that changes can be evaluated with no impact on the production system. In such a case, it is possible to create a logical point-in-time replica, which is called a snapshot, or a physical replica, which is called a clone. Snapshot replicas track changed blocks of data but refer to the production data for blocks that are not changed. Clones, on the other hand, create a complete copy of the data. In a traditional testing and development scenario, the replica will be subjected to at least a sample of the production load in order to measure the impact of any changes. In a snapshot replica, this load will interact with, and could potentially negatively impact, the production workload. For this reason, it is recommended that testing and development operations where the replica will need to respond to a load should use clone replicas. Plan storage operations for minimal disruption Some operations on the storage array can consume resources and may cause an impact on the production system if executed during a heavy production load. For this reason, storage-based operations that may consume resources should be scheduled for non-peak hours to minimize the potential for such occurrences. Be aware of the capacity and performance limitations of vault drives Vault and array firmware drives are the first five drives of the first disk tray. These drives have slightly reduced capacity, and slightly lower available IOPs while performing array management or configuration tasks. It is preferable to store less performance critical data in these drives. Bind at least one hot spare for every 30 drives A fully configured rack should have two to three hot spares, depending on database criticality and break/fix response times. Connect sufficient Fibre Channel ports into the storage network Usually, this is as many front-end ports as are available; however, all eight ports may not be required for a given workload, and the extra expense may not be justified. MirrorView, if configured, supports both switch connection and sharing of host and mirror traffic across the same port. Use multiple paths for high availability and improved performance Configure multiple paths between host servers and storage arrays using multiple HBAs and switches. Use at least two NICs or HBAs in each host. Install PowerPath software on all hosts, which provides failover and load balancing across available active paths. Load balancing I/O loads across multiple paths to the storage system ports avoids storage bottlenecks. PowerPath monitors data flow through all active paths. In case of a path failure, it removes the failed path and reroutes all data access through alternate live paths until the failed path is repaired and restored. Be aware of the impact of reserved LUN pool on database performance Allocation of the reserved LUN pool used for snapshot cache and asynchronous mirroring can have significant ramifications on overall array and database performance. Understand the anticipated I/O load required for these functions and locate and size the reserve LUN pool accordingly. For more information about reserved LUN pools allocation considerations for CLARiiON storage arrays, refer Best Practices Planning 17

18 to EMC CLARiiON Reserved LUN Pool Configuration Considerations Best Practices Planning at EMC.com and Powerlink. CLARiiON considerations The importance of understanding and planning the storage system cannot be underestimated since the storage system affects SQL Server performance more than any other factor. Performance improvement depends heavily on hardware characteristics such as disk layout, I/O characteristics, disk allocation units, and storage array caching practices. Following general guidelines should suffice for most midrange database designs. Best practices Increase CLARiiON prefetch settings to improve restore performance The default prefetch settings for a LUN is variable prefetch using a 4/4 multiplier. When a LUN is used as a point to store backups and restore from, increasing the multiplier to 32/8 yielded more than a 3x increase in speed. It should be noted that increasing the prefetch may hurt performance when nonserial operations are performed. However, a backup LUN should only be used for serial operations, so it can have a minimal negative effect and a substantial positive effect whenever data is read from the LUN. Balance LUNs design A CLARiiON family storage arrays has two SPs (storage processors). It is recommended to balance the load across the SPs as this ensures that a single SP does not get overloaded while the other is near idle. Additionally, this maximizes cache usage. Name the LUNs Naming the LUNs allows for their quick identification as shown in the following example, which provides a quick summary of the name, size, and usage. C:\Program Files\EMC\Navisphere CLI>navicli h getlun 7 Findstr LUN Capacity(Blocks) Name: LUN 20-TPCC_Data_lof2 LUN Capacity (Megabytes): LUN Capacity (Blocks): Use the latest verified HBA driver For best performance and stability, it is recommended to install the latest vendor HBA driver that has been validated for use with Windows Server Use EMC PowerPath/VE on physical servers EMC PowerPath/VE delivers PowerPath Multipathing features to optimize VMware vsphere and Microsoft Server 2008 Hyper-V virtual environments. With PowerPath/VE, you can standardize path management across heterogeneous physical and virtual environments. PowerPath/VE enables you to automate optimal server, storage, and path utilization in a dynamic virtual environment. This eliminates the need to manually load-balance hundreds or thousands of virtual machines and I/Ointensive applications in hyper-consolidated environments. The CLARiiON storage processors (SP) run in a mode that allows one to instantly take over the load of the other, in case of a failure. However, this failure will change the path. EMC PowerPath/VE software can detect this and correct it without I/O loss. Also, PowerPath/VE is used for dynamic path load balancing, which allows the load from a single host to be balanced across all available FC ports on the SP. Storage-array caching considerations Onboard cache on CLARiiON storage array can greatly enhance the performance of SQL Server I/O operation. The CLARiiON storage arrays offer a wealth of options for enabling and tuning the storage- Best Practices Planning 18

19 array cache. Read and write caching are independently configurable on each LUN in the array. Default array behavior is to enable read and write caching for all volumes. There is no default for array cache allocation. The following cache configuration best practices may deliver more efficient database performance from CLARiiON arrays. Best practices for EFD Due to the extremely high performance of EFDs, cache settings for EFDs do not follow traditional guidelines as explained next: EFDs are extremely fast, so when the read cache is enabled for the LUNs residing on them, the read cache lookup for each read request adds a significantly higher overhead as compared to FC drives, in an application profile that is not expected to get many read cache hits at any rate. Thus it can be faster to disable read cache to directly read the block from the EFD. In a real world scenario, where the CLARiiON CX4 is being shared by several applications and, especially, deployed with slower SATA drives, the write cache may become fully saturated, placing the EFDs in a fore flush situation, which adds latency. In these situations, it is better to write the block directly to EFDs than to write cache of the storage system. The cache setting for the storage systems should be configured as depicted in Table 2. Table 2. Recommended default cache settings SP Read Cache SP Write Cache LUN Read Cache LUN Write Cache FC Drive ON ON ON ON EFD ON ON OFF OFF Best practices for FC Set the storage array cache block size to the default 8 KB This setting matches the array cache block size to the standard SQL Server database page size of 8 KB. Allocate sufficient read and write cache on storage array For more information about cache allocation considerations for CLARiiON storage arrays, refer to the The RAID engine cache section in the EMC CLARiiON Best Practices for Fibre Channel Storage: CLARiiON Release 26 Firmware Update on Powerlink. Enable write caching for all volumes Disable write caching for read-only or non-database LUNs only if the write cache becomes saturated. Navisphere Analyzer can be used to monitor cache utilization and forced flush rates. Enable read and write caching for SQL Server transaction log volumes Minimizing response time for transaction log writes allows SQL Server more concurrency. Enable write caching for RAID 5 database volumes Write caching helps alleviate the effects of the RAID 5 lower write performance. Disable read caching for volumes with mostly random reads Consuming storage processor cache resources for these read operations will not significantly improve data response times, and will detract from the performance of other volumes that are less random in nature. The exception to this guideline applies to LUNs that will see sequential access for backup operations. In this case, the read cache should be enabled during the backup operation. Enable read caching for volumes with frequent sequential data reads Database table scans are sequential in nature and will benefit from read prefetching. Enabling read caching for volumes with small to medium indexes that are frequently accessed will also benefit from read prefetching. Best Practices Planning 19

20 Disk type considerations Select a disk type based on performance, cost, and reliability requirements of a SQL Server application environment. Best practices Use EFDs for high-performance and latency-sensitive applications EFDs are packaged in a standard 3.5-inch disk drive form factor used by existing CLARiiON disk drive array enclosures, making for simple integration with existing infrastructure. Due to their solid state design, these drives are especially well suited for latency-sensitive applications that require consistently low read/write response times. Since there are no moving parts, EFDs use far less energy when compared to drives with rotating media and weigh less as well. In addition to consuming less energy, they produce less heat, which also translates to energy savings in terms of cooling costs. The real energy savings becomes evident when comparing traditional disk drives to EFDs based on the comparable numbers of IOPS delivered per drive. EFDs can provide two very important benefits: A single EFD can replace many short-stroked drives by its ability to provide a very high transaction rate (IOPS). This reduces the total number of drives needed for the application, increases power saving by not having to keep many spinning disks, and eventually means reduced floor space in the data center as well. EFDs provide very low latency, so applications, where predictable low response time is critical and not all the data can be kept at the host or CLARiiON cache, will highly benefit from using such drives. EFD transfer rate is extremely high and data is served much faster than the best response time that can be achieved with a large number of short-stroked hard drives. Use FC drives for high-performance SQL Server applications FC drives provides high reliability and high random read/write performance. FC drives are available in either 10k or 15k rpm models. Use the faster 15k rpm FC drives for OLTP type workloads with a large number of small random I/O reads and writes. Use ATA drives for low cost and large capacity ATA drives cost less and provide large capacity. But ATA drives have a lower duty cycle and deliver less performance than FC drives. Because of lower random I/O performance, ATA drives are not suitable for OLTP type applications. ATA is ideal for storing aged data, replication, or backup copies of database files that are read or written to disk in large sequential I/Os. RAID type considerations Consideration should be given for the RAID type used in SQL Server database, transaction log and backup and restore volumes, as this decision will affect database performance, fault tolerance, and cost. RAID consists of multiple disk drives used primarily for improved I/O performance and large storage size. In CLARiiON, such a group of disk drives is referred as a RAID group. You can achieve optimal response time from the array subsystem by configuring the RAID levels to match your application I/O workload patterns. CLARiiON storage arrays offer multiple RAID level choices of RAID 0 striping, RAID 1 mirroring, RAID 1/0 striped mirrors, RAID 3 and RAID 5 striping with parity, and RAID 6 striping with dual parity. For more information about RAID configuration considerations for CLARiiON storage arrays, refer to the EMC CLARiiON Best Practices for Fibre Channel Storage white paper. Best practices Use CLARiiON hardware RAID instead of Windows server software RAID Software RAID uses host processor cycles that can impact the performance of SQL Server applications. Use more disk drives in a RAID group to improve performance Best Practices Planning 20