BEST PRACTICES GUIDE: Nimble Storage Best Practices for Scale-Out

BEST PRACTICES GUIDE: Nimble Storage Best Practices for Scale-Out

Contents Introduction... 3 Terminology... 3 Planning Scale-Out Clusters and Pools... 3 Cluster Arrays Based on Management Boundaries... 3 Adding, Removing and Merging Scale-out Cluster Arrays... 4 Pooling Arrays within a Scale-out Cluster... 4 Pool Arrays for Consolidation and Fault Isolation... 5 Pool Arrays with the Same Storage Capacity... 5 Pool Rebalancing... 6 Non-Disruptive Pool Rebalancing Use Cases... 6 Networking Best Practices... 7 SAN Bandwidth Availability... 8 Cluster Arrays on the same Subnets... 8 Choosing iscsi Switches... 8 Use Nimble CIM and PSP Provider for VMware Hosts... 9 Use Nimble Connection Manager on Windows Hosts... 9 Non-Windows/VMware Host Scale-out Connectivity...10 Getting Started...10 B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R S C A L E - O U T 2

Introduction Nimble Storage scale-out cluster functionality allows scaling of performance and capacity beyond the limits of a single array, while maintaining the management simplicity of a single array. Adding multiple arrays into the cluster linearly increases both performance and capacity, while leveraging the resources of all arrays in the cluster. Scale-out storage is ideal for IT environments looking to eliminate individual silos of storage across their infrastructure. Multiple types and generations of arrays can be combined in a single Nimble Storage cluster, protecting your investment. This document walks through the planning, hosting, and networking best practices associated with Nimble Storage s scale-out cluster. Terminology Scale-Out Cluster: Multiple Nimble Storage arrays that are combined together and managed as a single logical storage entity. Storage Pool: A pool is a logical collection of one or more arrays in a cluster containing volumes and their respective snapshots and clones. Volumes within a pool are striped across all the pool member arrays. Storage Array: Arrays are a highly available purpose-built storage appliance that consists of redundant storage controllers, disks and solid state flash storage. Planning Scale-Out Clusters and Pools Nimble Storage provides a rich set of features associated with scale-out clustering around data rebalancing, migration and host integration to get the most out of your investment. The following section features best practices and suggestions to follow when planning your deployment. Cluster Arrays Based on Management Boundaries A Nimble Storage scale-out cluster or group contains one or more individual arrays that are managed as a single logical storage array (up to 4 in the 2.x releases). This means that there is a single management domain and single pane of glass management view. This provides the ability to scale beyond a single array, managing all of the pools of storage within the cluster. B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R S C A L E - O U T 3

Cluster boundaries can also make sense for organizational boundaries to keep data assets separate. Creating separate scale-out clusters of arrays might also make sense for sensitive data such as finance and human resources to meet security audit requirements. If you require more granular array grouping then you should use separate Storage Pools. Adding, Removing and Merging Scale-Out Cluster Arrays Changing cluster membership is a quick process that makes storage performance and capacity available immediately, or removes it entirely. Moreover, it is transparent to hosts i.e. there is no downtime for hosts or applications. However, it is important to understand that changing pool membership can take time while the existing data volume, snapshots and clones are rebalanced on the resulting pool arrays. For example, if you would like to remove an array from a cluster, then you should first evacuate it from its current storage pool to migrate data to the remaining pool members. Merging scale-out clusters is a quick process because it doesn t require any immediate data rebalancing; however this process may be followed by migrating volumes between pools or merging pools which does result in data rebalancing across the member arrays. For further information about the rebalancing process, please see the following section titled Pool Rebalancing to fully understand the time associated with cluster and pool management. Pooling Arrays within a Scale-Out Cluster Nimble Storage scale-out architecture provides pooling of arrays for horizontal scalability of both performance and capacity to existing storage arrays. Scale-out architecture grows linearly as your SAN needs grow which allows you to continue to leverage older hardware, and eliminates the penalty associated with incorrectly forecasting the future performance and capacity needs of a SAN. Pooling arrays within a cluster allows you to distribute I/O across multiple arrays which increases performance and allows capacity to scale across all arrays in the pool. A Nimble Storage pool consists of whole arrays and arrays are members of a single storage pool at a time. B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R S C A L E - O U T 4

Pool Arrays for Consolidation and Fault Isolation Nimble Storage scale-out clusters allow you to pool whole arrays to consolidate storage based on logical boundaries. Pooling storage can be helpful for departmental chargebacks, fault isolation between production and non-production environments, or by application. Nimble s scaling and migration technology eliminates planned downtime when storage pools are changed. Pool Arrays with the Same Storage Capacity Nimble Volumes are distributed between each array in a storage pool relative to the storage capacity of the array. This data distribution storage architecture also has the effect of distributing I/O operations based on the data distribution. Pooling arrays of the same capacity will evenly distribute I/O operations across all arrays in the storage pool and permit maximum performance B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R S C A L E - O U T 5

across all arrays. Placing arrays with significantly differing capacity within the same pool may not fully utilize the performance resources available in the pool. By creating pools that span arrays of similar capacity, performance is balanced equally across all arrays in the pool which maximizes utilization of all performance resources. Pool Rebalancing Nimble Storage pools are rebalanced when storage objects such as arrays, pools and volumes are added, removed or merged. Rebalancing is a non-disruptive low-impact process that allows application I/O to continue uninterrupted even to the data sets being migration. Pool rebalancing gives highest priority to active data I/O and performs the rebalancing process with a lower priority. The relative speed of rebalancing is dependent upon the use case being performed and how quickly a typical user would expect the procedure to complete. Non-Disruptive Pool Rebalancing Use Cases Volume Migration - Volumes can be selectively migrated between pools including their associated snapshots. Volumes support read/write during the migration process. Adding Arrays to a Pool - Add performance and storage capacity to an existing pool. Evacuating Pools - Remove an array from a pool when it s no longer needed or when it is being replaced by another array. There must be enough capacity B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R S C A L E - O U T 6

within the remaining pool members to accommodate the volumes and snapshots migrated from the evacuating pool member. Merging Pools - Useful when combining storage resources that were previously in separate pools thus increasing performance scaling across multiple arrays. Networking Best Practices Nimble Storage Scale-out architecture allows you to manage a Storage Cluster that spreads volumes and their I/O requests between multiple arrays. A host cannot assume that a volume can be accessed through paths to one specific array or another. Instead of advertising all of the iscsi interfaces on the array, Nimble Storage scale-out clusters advertise one IP address (iscsi Discovery). Volume I/O requests are redirected to the appropriate array by leveraging deep integration with the Microsoft and VMware host operating system platforms or using iscsi redirection. Storage network architecture is of paramount importance to ensure high-speed lowlatency communication from hosts to arrays and from array to array. The following network best practices will help you to create the most robust iscsi storage network for your Nimble scale-out environment. B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R S C A L E - O U T 7

SAN Bandwidth Availability Nimble Storage scale-out clusters perform communication between arrays to coordinate production I/O in-addition to performing rebalancing of data between arrays. The intra-san communication and data migration is designed to be a low-impact process that won t interfere with production I/O. However, the amount of available SAN bandwidth will impact the speed with which volume migration and pool rebalancing can take place. Cluster Arrays on the same Subnets All of the arrays within the same Group must be able to see the same sub-networks. This ensures that host connectivity follows the proper network pathing as well as to ensure that volume and pool migration works properly and with maximum performance. Choosing iscsi Switches Network switches provide a critical part of an iscsi storage area network. There are many different classifications of switches and it is important to understand the characteristics that make a switch good for supporting iscsi storage traffic. Use the following table when evaluating network switches: Non-blocking Backplane Flow Control (802.3x) Buffer Space per Switch Port A switch used for iscsi data communication should have a backplane that provides enough bandwidth to support full duplex connectivity for all ports at the same time. For example, a 24 port Gigabit switch backplane should provide at least 48 Gigabits per second of bandwidth or (1 Gbps * 2 for Full Duplex * 24 Ports). Flow control provides a mechanism for temporarily pausing the transmission of data on Ethernet networks when a sending node transmits data faster than the receiving node can accept it. You should enable flow control on all hosts, switch, and array ports to ensure graceful communication between network nodes. Nimble Storage array NICs have flow control enabled by default. Switches are used to provide communication between hosts and arrays and for Nimble Storage scaling communication between arrays. Each switch port should have at least 512 Kilobytes of buffer memory per port to ensure B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R S C A L E - O U T 8

full performance between connected nodes. Support for Jumbo Frames Can Disable Unicast Storm Control Ethernet frames that transport data are typically 1,500 Bytes in size. While this does a good job of balancing application network traffic between network clients and servers, while host to storage communication tends to be measured in Kilobytes. Jumbo frames were created to better handle the flow of iscsi SAN traffic and consist of 9,000 Byte frames. Enable Jumbo frames to improve storage throughput and reduce latency. Storage traffic can appear bursty to switches which can be mistaken by some switches as a packet storm and blocked. Disabling Unicast Storm Control ensures that the storage traffic is transmitted unfettered. Use Nimble CIM and PSP Provider for VMware Hosts Nimble Storage provides scale-out network integration for VMware hosts using a CIM and PSP provider installed on each ESX host server. During installation, you specify the discovery IP of the scale-out cluster and the CIM connects to it for discovery of all possible network connections between each host and the arrays in the storage cluster. The CIM is then responsible for maintaining an optimal number of network sessions between the VMware ESX host and the storage array. The Nimble PSP provider communicates with the scale-out cluster for each volume attached to its host to maintain a bitmap of the logical volume layout. The PSP uses the volume bitmap to direct I/O requests to the appropriate array using the paths maintained by the CIM provider. Nimble s CIM and PSP providers will periodically update their network path and logical volume bitmaps with the arrays to maintain an optimal muti-path I/O (MPIO) strategy that maximizes performance. Use Nimble Connection Manager on Windows Hosts Connecting Microsoft Windows host servers to iscsi volumes is a simple procedure for a single connection. However, creating connections to support Multi-Path I/O (MPIO) on many host servers can be a cumbersome process with many repetitive steps. Nimble Storage provides a solution called Connection Manager that integrates with Microsoft Windows hosts to provide MPIO connectivity in a more automated fashion. Using Nimble Connection Manager (NCM), simply point it to the scale-out cluster s Discovery IP address and Nimble Connection Manager will configure B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R S C A L E - O U T 9

the appropriate network paths. NCM will create iscsi connections for each host network interface card and each array in the group. Nimble Connection Manager leverages a logical volume map that permits quick direction of I/O requests based on the scale-out volume s physical distribution between arrays. Nimble Connection Manager will periodically refresh the network pathing and logical volume bitmap to maximize storage performance. Non-Windows/VMware Host Scale-out Connectivity Host operating systems that do not have integration permitting third-party multi-path I/O direction can still use Nimble Storage scale-out features. Connections will need to be configured manually between host NIC ports and array NIC ports. If an I/O request reaches an array that cannot service it, then an iscsi redirection is triggered to send the requestor to the appropriate array. Getting Started Get started on planning by performing a quick audit of your current applications and storage to understand how much storage capacity and performance is needed. Document how these applications are expected to grow over the next year, three years and five years. Also, document any specific requirements around management, high-availability and isolation for specific workloads along with any additional requirements around data protection. These are good factors to incorporate in the initial design and deployment. Nimble Storage provides the flexibility to scale-to-fit applications and workloads storage needs and a little planning upfront can help ensure a smooth deployment of a scale-out storage cluster for any and all data center needs today and in the future. Nimble Storage, Inc. 2740 Zanker Road, San Jose, CA 95134 Tel: 408-432-9600; 877-364-6253) www.nimblestorage.com info@nimblestorage.com B E S T 2012 P R ANimble C T I C EStorage, S G U I DInc. E : All Nrights I M B Lreserved. E S T O RCASL A G E is Ba trademark E S T P R Aof CNimble T I C E S Storage F O R Inc. S C ABPG-SO-0812 L E - O U T 10