BEST PRACTICES GUIDE: Nimble Storage Best Practices for Networking
Contents Network Connectivity... 3 Management Network... 4 Data Network... 5 Choosing iscsi Switches... 7 B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 2
Network Connectivity This section will help you to properly connect your Nimble arrays to a redundant Ethernet network to ensure optimal performance and availability. Nimble Storage arrays are designed with redundant controllers that provide high availability access to your storage in the event that the active controller fails. In each of the associated diagrams you will see both solid lines and dashed lines. The solid lines represent an active connection while the passive lines represent passive connections that will become active in the event of a Nimble controller fail-over. It is also important to wire each sibling interface on each controller to the same switch. For example, Controller A, Eth1 connects to Switch 1 and Controller B, Eth1 also connects to Switch 1. Best Practice: To make wiring easier, match Odd Numbered Ports with Switch 1 and Even Numbered Ports with Switch 2. E.g. Odd-to-Odd and Even-to-Even. Nimble Storage arrays are typically configured to connect to a management network and to data networks. B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 3
Management Network Management Network Diagram Resiliency of the management network is important to permit access by administrators to the Nimble Storage arrays for management purposes. The management network is typically wired to the eth1 and eth2 ports that are located as labeled in the Management Network Diagram. The Management IP Address can float between network ports that are designated as management ports. Management IP Screen B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 4
Data Network In-general, you should configure the two stacked ports for management only which leaves the remaining (4) 1 Gigabit ports or (2) 10 Gigabit ports available for the data network. While the Nimble Storage management features permit mixing management and data networks, this configuration is rarely needed and requires special care to configure properly. If you are unsure of your networking needs, then you can contact Nimble Storage technical support for further assistance. Best Practice: If operating system attaching to Nimble Storage arrays permits the choosing of load balancing algorithm for multi-path I/O, you should choose Least Queue Depth (LQD). The Least Queue Depth algorithm is superior to Round Robin algorithms since it takes into consideration pending I/O operations to avoid overloading a particular connection. 1 Gigabit Network Wiring Best Practice: Enable Jumbo frames on data ports to maximize throughput. You must enable Jumbo frames on each network connection including the Nimble array, the switch and B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 5
servers. Failure to enable Jumbo frames on one or more connections will not achieve the benefit of the larger Ethernet frame size. 10 Gigabit Networking Wiring B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 6
Choosing iscsi Switches Network switches provide a critical part of an iscsi storage area network. There are many different classifications of switches and it is important to understand the characteristics that make a switch good for supporting iscsi storage traffic. Use the following table when evaluating network switches: Non-blocking Backplane Flow Control (802.3x) Buffer Space per Switch Port Support for Jumbo Frames A switch used for iscsi data communication should have a backplane that provides enough bandwidth to support full duplex connectivity for all ports at the same time. For example, a 24 port Gigabit switch backplane should provide at least 48 Gigabits per second of bandwidth or (1 Gbps * 2 for Full Duplex * 24 Ports). Flow control provides a mechanism for temporarily pausing the transmission of data on Ethernet networks when a sending node transmits data faster than the receiving node can accept it. You should enable flow control on all hosts, switch, and array ports to ensure graceful communication between network nodes. Nimble Storage array NICs have flow control enabled by default. Switches are used to provide communication between hosts and arrays and for Nimble Storage scaling communication between arrays. Each switch port should have at least 512 Kilobytes of buffer memory per port to ensure full performance between connected nodes. Ethernet frames that transport data are typically 1,500 Bytes in size. While this does a good job of balancing application network traffic between network clients and servers, while host to storage communication tends to be measured in Kilobytes. Jumbo frames were created to better handle the flow of iscsi SAN traffic and consist of 9,000 Byte frames. Enable Jumbo frames to improve storage throughput and reduce latency. B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 7
Can Disable Unicast Storm Control Storage traffic can appear bursty to switches which can be mistaken by some switches as a packet storm and blocked. Disabling Unicast Storm Control ensures that the storage traffic is transmitted unfettered. Nimble Storage, Inc. 2740 Zanker Road, San Jose, CA 95134 Tel: 408-432-9600; 877-364-6253) www.nimblestorage.com info@nimblestorage.com B E S T 2012 P R ANimble C T I C EStorage, S G U I DInc. E : All Nrights I M B Lreserved. E S T O RCASL A G E is a Btrademark E S T P R Aof C Nimble T I C E S Storage F O R NInc. E TBPG-NET-0313 W O R K I N G 8