Storage Tiering for Microsoft SQL Server and EMC Symmetrix VMAX with Enginuity 5874

Storage Tiering for Microsoft SQL Server and EMC Symmetrix VMAX with Enginuity 5874 Applied Technology Abstract This white paper examines the application of managing Microsoft SQL Server in a storage environment utilizing multiple storage configurations to optimize performance cost-effectiveness of storage allocations. Practical examples are provided of automated data movement between Storage Types to meet performance requirements for SQL Server databases and cost-effectiveness of EMC Symmetrix VMAX resources. May 2010

Copyright 2009, 2010 EMC Corporation. All rights reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com All other trademarks used herein are the property of their respective owners. Part Number h6734.1 Applied Technology 2

Table of Contents Executive summary...4 Introduction...5 Audience... 5 Evolution of Storage Tiering...5 Product and features overview...6 Symmetrix VMAX... 6 Enterprise Flash Drives... 7 Enhanced Virtual LUN (VLUN) migration... 8 Symmetrix Fully Automated Storage Tiering (FAST)... 8 How FAST is configured... 9 Device movement... 9 Symmetrix Management Console... 10 Skewed LUN access... 10 Storage Tiering mechanics... 11 Working with Fully Automated Storage Tiering (FAST) and SQL Server databases...11 Test methodology and configuration... 11 Configuring FAST for the environment... 13 Monitoring workload performance... 16 Identification of database workloads... 16 Selection of migration targets... 18 Scheduling migrations... 20 Assessing the benefits of FAST... 21 Conclusion...23 Applied Technology 3

Executive summary The EMC Symmetrix VMAX is the newest addition to the Symmetrix product line. Built on the strategy of simple, intelligent, modular storage, it incorporates a new scalable Virtual Matrix interconnect design that allows the storage array to seamlessly grow from an entry-level configuration into the world s largest storage system. The Symmetrix VMAX provides improved performance and scalability for demanding enterprise storage environments while maintaining support for EMC s broad portfolio of platform software offerings. EMC Symmetrix VMAX delivers enhanced capability and flexibility for deploying Microsoft SQL Server databases throughout the entire range of business applications, from mission-critical applications to test and development. In order to support this wide range of performance and reliability at minimum cost, Symmetrix VMAX arrays support multiple drive technologies including Enterprise Flash Drives (EFD), Fibre Channel drives (FC), both 10k rpm and 15k rpm, and 7200 rpm SATA drives. In addition, various RAID protection mechanisms are allowed that affect the performance, availability, and economic impact of a given Microsoft SQL Server system on a Symmetrix VMAX storage array. Customers are under increasing pressure to improve performance as well as the return on investment (ROI) for storage technologies, and ensuring that data is placed on the most appropriate Storage Type 1 is a key requirement. It is common to find applications utilizing the services of the SQL Server environment, having differing access patterns to different files, directing a significant workload to a relatively small subset of the data stored within the database. Other data files, and the data they contain, are less frequently accessed. This phenomenon is referred to as data access skewing. Optimizing the placement of these various components of the database to the Storage Type that matches their needs best, will help increase performance, reduce the overall number of drives, and improve the total cost of ownership (TCO) and ROI for storage deployments. As companies increase deployment of multiple drive types and protection types in their high-end storage arrays, it provides the storage and database administrators with the challenge of selecting the correct storage configuration for each application. Often, a single Storage Type is selected for all the data in a given database, effectively placing both active and idle data portions on fast FC drives. This approach can be expensive and inefficient, because infrequently accessed data will reside unnecessarily on highperformance drives. Alternatively, making use of high-density low-cost SATA drives for the less active data, FC drives for the medium active, and Enterprise Flash Drives (EFD) for the very active, enables efficient use of the storage resources, and reduces the overall cost and the number of drives necessary. This, in turn, also helps to reduce energy requirements and floor space, allowing the business to grow more rapidly. To achieve this tiered approach with Microsoft SQL Server databases, administrators can use Symmetrix Enhanced Virtual LUN Technology to move logical devices between drive types and RAID protections seamlessly inside the storage array. Symmetrix Virtual LUN technology does not require application downtime. It preserves the device IDs, which means there is no need to change file system mount points, volume manager settings, database file locations, or scripts. It also maintains any TimeFinder or SRDF business continuity operations even as the data migration takes place. This manual and often time-consuming approach to Storage Tiering can be automated using Fully Automated Storage Tiering or FAST. FAST uses policies to manage sets of logical devices and available Storage Types. Based on the policy guidance and the actual workload profile over time, The FAST Controller will recommend and even execute automatically the movement of the managed devices between the Storage Types. This white paper describes a tiered storage architecture approach for SQL Server databases, and the way in which devices can be moved nondisruptively, using either Enhanced Virtual LUN or FAST technologies in order to put the right data on the right storage at the right time. 1 Storage Type refers to a combination of drive type and RAID protection, as explained in the Evolution of Storage Tiering section later. Applied Technology 4

Introduction This white paper demonstrates how to use a Storage Tiering strategy to move volumes that belong to a SQL Server system by placing the volumes with high activity onto a high-performing Storage Type and volumes with low activity onto a more cost-effective Storage Type that provides the necessary performance. The performance of the SQL Server environment is measured first and then recorded. Volumes are selected for relocation and then relocated to different Storage Types. Then the performance of the SQL Server environment is measured again. Finally, the Fully Automated Storage Tiering (FAST) functionality is applied to the SQL Server environment. Audience This white paper is intended for Microsoft SQL Server database administrators, storage administrators and architects, customers, and EMC field personnel who want to understand the implementation of Storage Tiering in a Symmetrix VMAX. Evolution of Storage Tiering From a business perspective, Storage Tiering generally means that policies coupled with storage resources having distinct performance, availability, and other characteristics are used to meet the service level objective (SLO) for a given application. (By SLO we mean a targeted I/O service goal, that is, performance for an application.) This remains the case with Fully Automated Storage Tiering (FAST). For administrators, the definition of Storage Tiering is evolving. Initially, different storage platforms met different SLOs. For example: Gold Tier Symmetrix ; Silver Tier CLARiiON ; Bronze Tier Tape More recently, Storage Tiering meant that multiple SLOs are achievable in the same array: Gold Tier 15k, FC RAID 1; Silver Tier 10k, FC RAID 5; Bronze Tier 7.2k, SATA, RAID 6 FAST changes the categories further. Because multiple Storage Types can support the same application, tier is not used to describe a category of storage in the context of FAST. Rather, EMC is using new terms: Storage Group logical grouping of volumes (often by application) for common management Storage Class combination of Storage Types and FAST Policies to meet service level objectives for Storage Groups FAST Policies - policies that manage data placement and movement across Storage Types to achieve service levels for one or more Storage Groups Storage Type a shared storage resource with common technologies, namely drive type and RAID scheme For example, users might establish a Gold Storage Class as follows: Service level objective Storage Class FAST Policy Storage Type Read/write response time objective Gold 10% 40% 50% 15k rpm RAID 1 10k rpm RAID 5 7.2k SATA RAID 6 Figure 1 provides a quick summary of the relationship between Storage Types, FAST Policies, and Storage Groups. Applied Technology 5

Figure 1. FAST relationships In the rest of this paper, Storage Type at times will be referred to as Symmetrix Tier, in order to map clearly to specific commands in tools such as SMC. "FAST Policies" and "Storage Groups" will remain the same throughout this paper. Product and features overview The following is a brief summary of EMC Symmetrix technologies that must be understood in order to be able to deploy a Storage Tiering strategy with the Microsoft SQL Server database environment. Symmetrix VMAX The EMC Symmetrix VMAX is the newest addition to the Symmetrix product family. Built on the strategy of simple, intelligent, modular storage, it incorporates a new scalable Virtual Matrix interconnect that connects all shared resources across all VMAX Engines, allowing the storage array to grow seamlessly and cost-effectively from an entry-level configuration into the world s largest storage system. The Symmetrix VMAX provides improved performance and scalability for demanding enterprise storage environments while maintaining support for EMC s broad portfolio of platform software offerings The Enginuity operating environment for Symmetrix version 5874 is the latest Enginuity release supporting Symmetrix VMAX arrays. With the release of Enginuity 5874, Symmetrix VMAX systems now deliver new software capabilities that improve capacity utilization, ease of use, business continuity, and security. These features provide significant advantage to customer deployments in a virtualized environment, where ease of management, and protection of virtual machine assets and data assets are required. Symmetrix VMAX arrays extend the scalability of previous generations of Symmetrix DMX technology, by providing a superior level of scalability, and support for a broad new range of drive technologies as detailed in Figure 2. Additionally, Symmetrix VMAX offers the ultimate in flexibility, including the ability to incrementally increase back-end performance by adding VMAX Engines and storage bays. Each high-availability VMAX Engine controls eight redundant Fibre Channel loops that support up to either 240 or 360 drives depending upon configuration. Subsequently, each high-availability VMAX Engine provides front-end as well as back-end connectivity, providing enhanced scalability. Applied Technology 6

1 to 8 redundant VMAX Engines Up to 2.1 PB usable capacity Up to 128 FC FE ports Up to 64 FICON FE ports Up to 64 GigE / iscsi FE ports UP to 1 TB global memory (512 GB usable) 48 to 2,400 drives Enterprise Flash Drives 200/400 GB FC drives 146/300/450 GB 15k rpm FC drives 300/450/600 GB 10k rpm SATA drives 1 TB 7.2k rpm Figure 2. Symmetrix VMAX hardware scalability The Symmetrix VMAX systems also maintain customer expectations for high-end storage in terms of availability. High-end availability is more than just redundancy; it means nondisruptive operations and upgrades, and being always online. Beyond previous Symmetrix generations, Symmetrix VMAX arrays provide: Nondisruptive expansion of capacity and performance at a lower price point Sophisticated migration for multiple storage tiers within the array The power to maintain service levels and functionality as consolidation grows Simplified control for provisioning in complex environments Enterprise Flash Drives With the Symmetrix VMAX support of Flash drives, EMC has created a new ultra-high performance storage tier that removes previous performance limitations imposed by magnetic disk drives. For years, the most demanding enterprise applications have been limited by the performance of magnetic disk media. Performance in storage arrays has been constrained by the physical limitations of hard disk drives. Enterprise Flash Drives deliver unprecedented performance and response times, which are benefits well suited for demanding SQL Server configurations. Enterprise Flash Drives dramatically increase performance for latency-sensitive databases like SQL Server. Enterprise Flash Drives, also known as solid state drives (SSD), contain no moving parts, which removes much of the storage latency delay associated with traditional magnetic disk drives. A Symmetrix VMAX with Enterprise Flash Drives can deliver single-millisecond application response times and up to 30 times more I/O operations per second (IOPS) than traditional Fibre Channel hard disk drives (HDD). Additionally, because there are no mechanical components, Flash drives consume significantly less energy than hard disk drives. When replacing a larger number of HDDs with a lesser number of Enterprise Flash Drives, energy consumption can be reduced by up to 98 percent for a given IOPS workload. The high-performance characteristics of Enterprise Flash Drives eliminate the need for organizations to purchase large numbers of traditional hard disk drives, while only utilizing a small portion of their capacity to satisfy the IOPS requirements of SQL Server deployments. The practice of underutilizing a hard disk drive for increased performance is commonly referred to as short-stroking. Enterprise Flash Drives can Applied Technology 7

increase SQL Server database performance and eliminate the need to short-stroke drives, thus keeping storage footprint and power consumption to a minimum and reducing total cost of ownership (TCO). Enhanced Virtual LUN (VLUN) migration The Enhanced Virtual LUN Technology migration feature introduced with Symmetrix VMAX with Enginuity 5874 offers SQL Server DBAs the ability to transparently migrate LUNs used by SQL Server databases between different Storage Types. The Storage Types are defined based on a number of attributes that may include a particular storage device technology such as high-performance Enterprise Flash Drives (EFDs), Fibre Channel drives, or high-capacity low-cost SATA drives. Migrations also allow for a change in RAID protection, which may also be an attribute of a Storage Type. Enhanced Virtual LUN is fully integrated with Symmetrix replication technology and maintains consistency and source/target device relationships in replications such as SRDF, TimeFinder/Clone, TimeFinder/Snap, or Open Replicator. The migration does not require swap or Dynamic Relocation Volumes (DRV) space, and is nondisruptive to the attached SQL Server environment and other internal Symmetrix applications such as TimeFinder and SRDF. All migration combinations of drive types and protection types are valid except for unprotected volumes. The SQL Server LUNs can be migrated to either unallocated space (also referred to as unconfigured space) or to configured space, which is defined as existing Symmetrix volumes that are not currently assigned to a server. The data on the original source volumes is cleared using instant VTOC once the migration has been completed. The device migration is completely transparent to both Microsoft SQL Server and the underlying Microsoft Windows Server operating system since the operation is executed against the Symmetrix device; thus all attributes of the LUNs are not changed and SQL Server operations are uninterrupted. Furthermore, in SRDF environments, the migration does not require customers to re-establish their disaster recovery protection after the migration. Enhanced Virtual LUN migration helps customers to implement an Information Life Management (ILM) strategy for their databases, such as the movement of the entire database, specific files, or filegroups between Storage Types. It also allows adjustments in service levels and performance requirements to application data. For example customers often provision storage for a particular application before clear performance requirements are known. LUN migration may be utilized at a later time, once the requirements are better understood to implement any adjustment to increase user experience and ROI using the correct Storage Class. Symmetrix Fully Automated Storage Tiering (FAST) Introduced in the Enginuity 5874 Q4 service release, EMC Symmetrix VMAX Fully Automated Storage Tiering (FAST) is Symmetrix software that utilizes intelligent algorithms to continuously analyze device I/O activity and generate plans for moving and swapping logical devices for the purposes of allocating or re-allocating application data across different performance Storage Types within a Symmetrix array. FAST proactively monitors workloads at the Symmetrix device (LUN) level in order to identify busy devices that would benefit from being moved to higher-performing drives such as EFD. FAST will also identify less busy devices that could be relocated to higher-capacity, more cost-effective storage such as SATA drives without altering performance. Time windows can be defined to specify when FAST should collect performance statistics (upon which the analysis to determine the appropriate Storage Type for a device is based), and when FAST should perform the configuration changes necessary to move devices between Storage Types. Movement is based on userdefined Storage Types and FAST Policies. The primary benefits of FAST include: Automating the process of identifying volumes that can benefit from EFD and/or that can be kept on higher-capacity, less-expensive drives without impacting performance Applied Technology 8

Improving application performance at the same cost, or providing the same application performance at lower cost. Cost is defined as space, energy, acquisition, management and operational expense Optimizing and prioritizing business applications, allowing customers to dynamically allocate resources within a single array Delivering greater flexibility in meeting different price/performance ratios throughout the lifecycle of the information stored The management and operation of FAST can be conducted using either the Symmetrix Management Console (SMC) or the Solutions Enabler Command Line Interface (SYMCLI). Additionally, detailed performance trending, forecasting, alerts, and resource utilization are provided through Symmetrix Performance Analyzer (SPA). And if so desired, EMC Ionix ControlCenter provides the capability for advanced reporting and analysis that can be used for chargeback and capacity planning. How FAST is configured FAST is configured by defining three distinct objects: Storage Groups. A Storage Group is a logical grouping of up to 4,096 Symmetrix devices. Storage Groups are shared between FAST and Auto-provisioning Groups; however, a Symmetrix device may only belong to one Storage Group that is under FAST control. Storage Types. Storage Types are a combination of a drive technology (for example, EFD, FC 15k rpm, or SATA) and a RAID protection type (for example, RAID 1, RAID 5 3+1, or RAID 6 6+2). There are two types of Storage Types static and dynamic. A static type contains explicitly specified Symmetrix disk groups, while a dynamic type will automatically contain all Symmetrix disk groups of the same drive technology. A Storage Type will contain at least one physical disk group from the Symmetrix but can contain more than one. If more than one disk group is contained in a Storage Type, the disk groups must be of a single drive technology type. FAST Policies. FAST Policies associate a set of Storage Groups with up to three Storage Tiers, and include the maximum percentage Storage Groups volumes can occupy in each of the Storage Tiers. The percentage of storage specified for each tier in the policy when aggregated must total at least 100 percent. It may, however, total more than 100 percent. For example, if the Storage Groups associated with the policy are allowed 100 percent in any of the tiers, FAST can recommend for all the storage devices to be together on any one tier (capacity limit on the tiers is not forced). In another example, to force the Storage Group to one of the tiers simply set the policy to 100 percent on that tier and 0 percent on all other tiers. At the time of association, a Storage Group may also be given a priority (between 1 and 3) with a policy. If a conflict arises between multiple active FAST Policies, the Fast Policy priority will help determine which policy gets precedence. FAST can be configured to operate in a set and forget mode (Automatic) in which the system will continually gather statistics, analyze, and recommend and execute moves and swaps to maintain optimal configuration based on policy, or in a user approval mode (User Approved) in which all configuration change plans made by FAST must be approved for a FAST suggested plan to be executed. Device movement There are two methods by which a device will be relocated to another type: move or swap. A move occurs when unconfigured (free) space exists in the target tier. Only one device is involved in a move, and a DRV (special Symmetrix device used for device swapping) is not required. Moves are performed by creating new devices in unconfigured space on the appropriate Storage Type, moving the data to the new devices, and deleting the old device. A swap occurs when there is no unconfigured space in the target type, and results in a corresponding device being moved out of the target Storage Type. In order to preserve data on both devices involved in the swap, a single DRV is used (DRV should therefore be sized to fit the largest FAST controlled devices). Applied Technology 9

Moves and swaps are completely transparent to the host and applications and can be performed nondisruptively, without affecting business continuity and data availability. Symmetrix metadevices are moved as a complete entity; therefore, metadevice members cannot exist in different physical disk groups. FAST optimizes application performance in Symmetrix VMAX arrays that contain drives of different technologies. It is expected that customers will have their arrays configured with Flash, Fibre Channel, and/or SATA drives, resulting in storage tiers with different service level objectives. Rather than leave applications and data statically configured to reside on the same Storage Type, FAST will allow customers to establish the definitions and parameters necessary for automating data movement from one type to another according to current data usage. The first version of FAST as is released with Enginuity 5874 will move data at the full device granularity. Symmetrix Management Console Many large enterprise data centers have stringent change control processes that ensure reliable execution of any modification to their IT infrastructure. Often changes are implemented using scripts that are fully documented, have been thoroughly reviewed, and can be consistently executed by all storage administrators. An alternative to using Solutions Enabler scripting is to use the Symmetrix Management Console (SMC) Symmetrix Management Console contains a number of wizards to simplify the configuration of multi-step tasks. In subsequent sections, individual steps are executed to display SMC functionality; however SMC provides a wizard to initially configure Symmetrix FAST environments as shown in Figure 3, and is the recommended method to appropriately configure initial deployments. Figure 3. SMC FAST Configuration Wizard Skewed LUN access Skewed LUN access is a phenomenon that is present in most databases. Skewing is when a small number of SQL Server LUNs receive a large percentage of the I/O from the applications. This skewed type of activity can be transient, that is to say lasting only a short period of time, or persistent and lasting much longer periods of time. Skewed, persistent access to volumes makes those volumes good candidates to place on a higher-performing Storage Type, if they are not already on the highest-performing type. Skewing by its nature means that there are SQL Server LUNs that are receiving lower activity than others. If these volumes are persistently receiving fewer requests than others they may be good candidates to move a lower-performing Storage Type. Microsoft SQL Server database environments are created by defining one or more filegroups (the PRIMARY filegroup will always exist). Each filegroup is subsequently defined to exist on one or more Applied Technology 10

data files located on NTFS volumes. Due to the proportional fill mechanism used by SQL Server, data files within a given filegroup will generate almost identical I/O patterns. This highly correlated workload also leads to contention, and is the primary motivation for the best practice recommendation to distribute these files across volumes and LUNs. It is a commonplace occurrence, however, to have storage allocations for these multiple LUNs be provided from within the same storage pool. Thus, while the workload is distributed across different LUNs and NTFS volumes at the host level, the workload is generated to the same physical spindles defined within the storage pool. Storage Tiering mechanics The goal of an effective Storage Tiering approach in a multi-tiered storage configuration is to place the right data on the right storage at the right time. A given SQL Server LUN may be highly active and highly accessed when data is created on it in the first instance. But over time, its usage may drop to where it could be deployed on a Storage Type that has lower-cost and lower-performance characteristics. Alternatively, co-located storage devices within a given Storage Type may be contending for I/O resources. One workload may represent a much more demanding I/O profile. Identification of this demand, and mapping this to available Storage Types within the array, may not only improve the performance of the higher workload, but in so doing, may provide benefits to the remaining portions of the database as the available resources are able to satisfy the reduced I/O demand. A typical Storage Tiering approach uses the following steps: 1. Monitor volume performance 2. Identify candidate volumes for migration 3. Find space to move volumes 4. Schedule the movement 5. Repeat the process at a later time Fully Automated Storage Tiering in this initial release is the automated processing for the five steps. Working with Fully Automated Storage Tiering (FAST) and SQL Server databases To validate the mechanisms and show the efficacy of the FAST solution for Microsoft SQL Server environments a test environment was defined to simulate a user environment. A moderate to high user workload was generated against a SQL Server 2008 environment. Storage allocations were initially provided in a manner that is consistent with current provisioning practices deployed within the SQL Server user community. Storage Classes were defined within the environment, and a FAST Policy was defined for the SQL Server database. FAST monitoring was implemented, and a user approval mode was utilized to identify the actions the FAST Controller determined. These suggested migrations were subsequently approved, and the system monitored for the resulting performance changes. Test methodology and configuration For all testing, Microsoft Windows Server 2008 R2 and Microsoft SQL Server 2008 SP1 were utilized. The primary user database was implemented to be comprised of several filegroups where each filegroup mapped to one or more files located over eight LUNs, as shown in Figure 4. In addition to this user database environment, the TEMPDB database was modified to implement two data files, each located on two additional LUNs, and the transaction log defined on a separate LUN. Applied Technology 11

Using the information presented in Figure 4, it is possible to map the locations of the SQL Server logical structures, such as tables and indexes, to physical locations. Given the proportional fill algorithm used by SQL Server, each logical entity, such as the Broker filegroup will have allocations from each of the constituent files. Subsequently, all I/O activity generates against tables and indexes defined within the Broker filegroup, will be serviced by the physical files. As a result this I/O will be directed at the LUNs which provide the storage for the files. Each LUN was defined as approximately 120 GB in size. Each LUN contained a single NTFS volume, and each NTFS volume was mounted at a mount point located under the directory C:\SQLDB. Most volumes only contained a single data file, except DATA1 which contained the Misc_1.ndf file as well as the Broker_1.ndf file. Figure 4. Overview of relationships of filegroups, files, and physical drives The simulated environment was that of a TPC-E like environment. As such, it represents the workload as anticipated of an online brokerage environment. Simulated users perform a range of processing tasks, such as stock transactions, and look-ups, as well as the generation of larger reports. The workload generated a significant read/write requirement against the various portions of the database. The Windows Perfmon instrumentation was set to a 15-second interval and the results before, during, and after the volume relocation were collected and analyzed. Additionally, the SQL Server Performance Warehouse was utilized to collect instrumentation from the environment. The SQL Server Performance Warehouse is a management feature available within the SQL Server 2008 environment. In addition to providing drive-based statistics in a similar manner to Windows Performance Manager, it also correlates internal SQL Server metrics that can assist in determine SQL specific attributes. The results are presented in the following sections Each metavolume presented to the host as a LUN was created as a striped configuration using four members, whereby each member hypervolume was 30 GB in size. Hypervolumes were of a RAID 5 (3+1) configuration. Thus each hypervolume was located on a set of four physical spindles. All hypervolumes, and subsequently all metavolumes, were created in a single disk group that contained 96 drives. These drives were 450 GB 15k rpm. Applied Technology 12

As each hypervolume was defined across four physical spindles, and each metavolume was configured from four hypervolumes, each metavolume was potentially located over 16 physical spindles. With nine such metavolumes (eight for data, and one for transaction log), this would have effectively required 144 spindles. However, the disk group was constrained to a set of 96 drives, and thus there was sharing of spindles. In addition to the single database environment, an additional constant workload was generated against the 96 physical spindles. IOMETER was used to generate a moderate (22,000 IOPS at or above a 70 percent read hit), but constant, read workload from the physical spindles. The read workload from IOMETER resulted in an average of 68 read requests per spindle. The IOMETER workload was executed throughout the testing and is thus a constant in the configuration, and represents other user workloads that are expected in a customer environment. The tests outlined in this white paper were performed on the following configuration: Configuration Aspect Description Storage controller Symmetrix VMAX 1667 Microcode 5874 Q4 2009 service release Microsoft Windows Microsoft Windows Server 2008 R2 SQL Server SQL Server 2008 SP1 Volumes 12 x 4-member striped metavolume Symmetrix Tier 4 x 400 GB Flash drives Symmetrix Tier 96 x 450 GB 15k rpm Fibre Channel Symmetrix Tier SATA 1 TB Configuring FAST for the environment When initially configured, the storage allocations for all LUNs were made from a single pool of physical spindles with a single RAID protection scheme. This is synonymous with a Storage Type, as previously defined. The Symmetrix VMAX array contained a number of different technologies including Enterprise Flash Drives (EFD), 450 GB 15k rpm drives, and 1 TB SATA drives. Using these differing technologies and available RAID levels, it is possible to construct the various Storage Types that may be applied. In the tested environment, multiple Storage Types tiers were defined, as shown in Figure 5. A type of storage may differ on technology implemented (the physical drive characteristics) or by the RAID protection scheme implemented. The tiers created in this configuration varied both in terms of the underlying technology, where the tier FLASH was defined as RAID 5 3+1 protection on EFD, FC_R53 was defined as being a RAID 5 3+1 protection scheme on Fibre Channel 450 GB 15k rpm drives, and SATA_5 was defined as being a RAID 5 7+1 protection scheme on 1 TB 5,400 rpm drives. Other entries also exist, but these three tiers were subsequently used to define a policy for the SQL Server environment. Applied Technology 13

Figure 5. Defining Storage Types within Symmetrix Management Console The Storage Types were applied to create a policy named SQL_PROD. This policy defined the applicable Storage Types, and applied capacity percentages to the various types, referred to as tiers in SMC. The percentages define how much of the storage space used by the Storage Group defined in the policy can be utilized from each of the tiers. Thus, the values as shown in Figure 6 indicate that when applied to a Storage Group, 90 percent of the storage allocation for the group may come from the FC_R53 Storage Type, 20 percent may come from the FLASH Storage Type, and 20 percent may be allocated from the SATA_R5 Storage Type. The total allocation in this instance is 130 percent, and will allow the FAST Policy to allocate storage from the most appropriate tier based on its statistical sampling. Figure 6. Tier definition within Symmetrix Management Console The allocation of a Storage Group to a defined tier binds the policy to the devices contained with the Storage Group. The Storage Group is defined when implementing Auto-provisioning Groups, and defines the storage devices that will be available to a host when bound to a view. For a tested workload the Storage Group name was SQL_FAST_PRD and defined all storage devices that were presented to the SQL Server host. This Storage Group naming is typically conducted when initially provisioning storage to the Applied Technology 14

SQL Server host during initial deployment. In Figure 7 the Storage Group is displayed with its component devices. Figure 7. Allocating a Storage Group to a policy in Symmetrix Management Console To complete the implementation of the FAST mechanisms, it is necessary to set appropriate time periods for the policy engine to collect statistics for workload analysis. Statistics collection is defined within the Symmetrix Optimizer environment, and may also be set through Symmetrix Management Console, as shown in Figure 8. Figure 8. Symmetrix Optimizer collection and swpa/move windows It is possible to configure Symmetrix Optimizer to execute swaps automatically or require user approval for swaps. If automatic mode is set for Symmetrix Optimizer, the FAST-suggested movements will be executed, as Optimizer is used for effecting movement. In the tested configuration, Optimizer was left in user approval mode, to show the planned device migrations. Applied Technology 15

Having defined the Storage Group association to the Storage Types through the policy, and scheduling the application time periods within Symmetrix Optimizer, the environment was configured appropriately. To validate the selection of devices, and planned movement, the operating system and SQL Server monitoring were also configured. Monitoring workload performance Microsoft SQL Server database administrators and Windows Server administrators will typically utilize Windows performance counters to monitor overall system performance. The instrumentation provided by the Windows performance counters represents a valid performance profile of the behavior of the storage devices presented to the host. Additionally, Microsoft SQL Server implements a Performance Warehouse implementation that can be used to specifically monitor those portions of the environment utilized by a SQL Server database. The SQL Server Performance Warehouse can be a valuable tool for database administrators to identify specific performance and transaction counters. Both Windows performance counters and the SQL Server Performance warehouse will be used in the following sections to monitor and qualify migrations. Identification of database workloads The user workload was executed for an extended period of time to ensure that the workload reached a constant state. Both SQL Server Performance Warehouse and Windows Performance Monitor counters were being collected over the time interval. In Figure 9 the overview screen of the SQL Server Performance Monitor is displayed, covering a 4-hour time period when the initial workload was executed. Figure 9. SQL Server Performance Warehouse overview Of note in Figure 9 is the SQL Server Waits graph, which helps immediately identify how much wait activity occurs in various areas of the system. The larger portion of the graph can be seen to attribute a significant wait occurrence on Buffer I/O. This wait state is typically incurred as a result of long read times for storage components. It is possible to drill into the specific waits for SQL Server by selecting the graph, and expanding the view to look at specific I/O activity for the database environment. The resulting display for the test environment can be seen in Figure 10, where the logical file devices are listed in workload and I/O latency order. For the tested environment, the Broker4 and Broker5 logical files can be seen to have the greatest latency of 8 ms and 6 ms, respectively, at throughput rates around 60 MB/s. Correlating the logical files with physical storage devices, shown in Figure 4, it is possible to identify that Broker4 and Broker5 are located on NTFS volumes DATA4 and DATA5, which in turn are provisioned from the metavolumes 119 and 11D. Applied Technology 16

Figure 10. Performance Warehouse Virtual File statistics It is also possible to derive from the report in Figure 10 that since all LUNs are effectively the same configuration and coming from the same pool of physical drives, that the resulting latency differences may be attributed to contention at the drive level. The I/O wait and latency numbers displayed by SQL Server Performance Warehouse can easily be cross-referenced with Windows performance counters. Figure 11 shows the read per second workload for the data volumes (there is little read activity on the transaction log in comparison to the data files). Figure 11. Read I/O rates for data volumes Read per second numbers are rarely sufficient to determine that there is any issue with the given configuration. Rather, it is necessary to put the read and write activity into context with the latency for the given workload style as well as size of the I/O itself. In the case of the read workload to the data files, the average I/O size was 8 KB (the SQL Server page size). The latency numbers for the read workload are shown in Figure 12. These latency statistics match those shown in the SQL Server Performance Warehouse, and the larger latencies are generated by the LUNs containing the Broker files. Applied Technology 17

Figure 12. Read latency for data volumes It is clear therefore that there are differing performance characteristics for the various data files. The Broker files have a significant workload generated against them as compared to the other files and resulting LUNs. These files contribute to the increased SQL Server waits shown in Figure 9. Selection of migration targets Selection of the migration targets is based on the relevant policy being applied. In the tested configuration, all devices were initially created within a single Storage Type. All devices were from the one pool of 88 drives. All hypervolumes were configured with RAID 5 3+1 protection, and each LUN was configured as a striped metavolume of four member hypervolumes. This style of design is consistent with user implementations for SQL Server database environments. Certainly latencies for the environment were within general best practice guidance, although as was shown, the configuration was suffering from high wait times within SQL Server. Through the use of both Windows Performance Monitor and SQL Server Performance Warehouse statistics the most underperforming storage locations could be identified. Specifically those metavolumes providing storage locations for Broker4 (DATA4) and Broker5 (DATA5) suffered from the greatest latency. In a manually managed environment, such devices would require administrator intervention. In a policy-based environment such as that provided by FAST, identification of underperforming devices, and actionable movements within the policy, can improve the overall performance. As the monitoring time interval was set to analyze all devices within the system during the course of the workload run, and the policy engine was utilizing these statistics. A plan for movement was automatically generated by the FAST Controller. The plan can either be viewed through Symmetrix Management Console through the Swap/Move list, or via the symfast CLI as shown in Figure 13. The suggested plan is uniquely identified by the Plan Identifier, and details the reason for the plan (Performance) and the suggested movement. Again, as the setting on the system was defined to be user approved, the plan shows a Plan State of NotApproved. Applied Technology 18

Figure 13. FAST-generated performance movement plan The targeted devices include the storage location for Broker4 (DATA4), which is the metavolume defined by device 119 (the metavolume head) and its three members (devices 11A, 11B and 11C). Also included is the storage location for Broker5 (DATA5), which is the metavolume defined by device 11D (the metavolume head) and its three members (devices 11E, 11F, and 120). The plan suggests that the devices be relocated to the FLASH Storage Type, and displays the associated disk group number and name. Additionally, the protection type is displayed, in this case R5 (3+1), as this was the protection type defined for this Storage Type. Also applicable to a migration are other styles of policy compliance. In Figure 15, a further migration is suggested and in this instance, the Group status indicates that the move is based on compliance needs. Currently the storage allocation is out of compliance with the FAST Policy, which as defined, indicated that 20 percent of storage allocation could be satisfied from EFD storage, and 90 percent could be satisfied from the RAID 5 Storage Type. While two devices were moved to the EFD Storage Type, this continued to leave the policy out of compliance, which is shown in Figure 14, where the allocation for the Storage Type needs to be decreased to maintain compliance. Figure 14. FAST Policy compliance In such cases, the FAST Controller will determine those storage devices that can be moved to ensure the compliance to the policy. In the case where a movement is to a lower-performing Storage Type, devices that exhibit the lowest workload patterns will be identified to be moved. Such movements are scheduled in the same manner as performance moves, and a special designation for the plan is shown, as seen in Figure 15. Applied Technology 19

Figure 15. FAST-generated compliance movement plan In the tested configuration, the device selected was the storage device allocated to one of TEMPDB data file locations. As TEMPDB is not utilized in this configuration, any of the three devices (two TEMPDB data files and one TEMPDB Log) locations were equal candidates. Scheduling migrations Migrations that are either based on a user approval mode, or automatically approved, are subject to a further scheduling policy defined within the Optimizer environment. The migration time period, especially for up-tiering events, which are generally trying to address underperforming configurations, should be scheduled during hours when heavy production utilization is not typical. Quality of service (QoS) mechanisms within the Symmetrix VMAX environment will ensure that user workloads are not significantly affected when they occur, but scheduling movements such that they complete outside of production periods is recommended. User approval of a given FAST plan may be either approved through Symmetrix Management Console in the Symmetrix Optimizer section, or by utilizing the symfast CLI, as shown in Figure 16. Note that the plan identifier needs to be supplied, and is the same plan identifier that was identified in Figure 13. Figure 16. User approval of a suggested FAST plan Approved plans will wait for the defined swap window to arrive before execution. Once executing, a migration cannot be terminated, and will run until concluded. Depending on volume sizes, and the number of migrations in process, the time for migration completion will vary. Symmetrix QoS mechanisms will prevent the migration from adversely affecting production workloads should the migration continue into normal work hours. It is also possible to apply manual priority settings to further limit the copy process by utilizing the symqos CLI and lowering the Mirror Pace setting. While the migration is executing, it is possible to query the progress of the migration by again using Symmetrix management Console or the symfast CLI. In Figure 17 additional information is displayed for a plan that is executing a migration, including the time that the actual migration began. Applied Technology 20

Figure 17. FAST migration in process Once all devices have fully migrated, the migration will automatically terminate, and the targeted devices will exist on the selected Storage Type. The performance attributes of the Storage Type will automatically apply to the devices. Assessing the benefits of FAST The migrations implemented by the FAST Policy engine improved overall performance of the more heavily accessed volumes by migrating them to a higher-performing Storage Type. In the example configuration, two LUNs were migrated to the EFD Storage Type, resulting in improved workload throughput and reduction in I/O latency time for these storage devices. The migration of these heavily accessed volumes also resulted in a reduction of contention within the original storage pool, thereby improving response times for the remaining devices. Post-migration response times were significantly better than prior to the move, as shown in Figure 18. Average latencies dropped for all volumes well below a 4 ms latency, as compared to before the move, where average latencies for a number of volumes were 6 ms and higher. Figure 18. Read latencies for data volumes - Post-migration In addition to improved latency for read requests, overall throughput for the LUNs increased. Figure 19 shows that read requests per second increased from the highest of around 7,000 reads/s to over 9,000 read/s after migration. Applied Technology 21

Figure 19. Read I/O rates for data volumes Post-migration The performance improvements were correlated by the data collected by the SQL Server Performance Warehouse and shown in Figure 20. Figure 20. Performance Warehouse virtual file statistics after movement As a direct result of higher throughput and lower latencies, the operation of the SQL Server environment improved across all major attributes. Figure 21 details the relative improvement in performance for the environment. Batch requests/s can provide a general view of SQL Server throughput, and increased by a factor of 62 percent. Figure 21. Comparison of improvement for major metrics Applied Technology 22

Full scalability of the system will have been limited due to the inability to drive additional workload. As a result of improving the performance, CPU utilization increased to 100 percent after the FAST volume migrations completed. This behavior will have artificially capped the full potential improvement in system performance as any latencies caused by the storage configuration were relieved, allowing SQL Server to execute much more effectively. Figure 22 shows total CPU utilization for the comparative workloads, and shows that alleviating the I/O latencies allowed the SQL Server database environment to consume all available CPU resources. Figure 22. CPU utilization of pre- and post-migration workloads Conclusion Microsoft SQL Server database environments often exhibit a skewed workload across files comprising differing filegroups. In many implementations, the LUNs used as storage devices are allocated from the same storage pools, which will often lead to contention for physical resources. In enterprise-class storage arrays, tiered storage configurations are implemented to provide differing performance and cost-effective storage solutions. Migrations of data devices that represent the more highly accessed volumes from a lower-performing Storage Type to a higher-performing Storage Type help performance not only for the volumes being migrated, but for the other volumes within the previous Storage Type that suffer from a lower level of contention. Moving lightly accessed volumes from a higher-performing Storage Class to a more appropriate Storage Class helps further improve utilization and drives down cost. Applied Technology 23