Scale Computing VMware Best Practices Guide. Scale Computing 5225 Exploration Drive Indianapolis, IN, 46241

Transcription

1 Scale Computing VMware Best Practices Guide Scale Computing 5225 Exploration Drive Indianapolis, IN, 46241

2

3 CHAPTER 1 Introduction Revision History CHAPTER 2 Best Practices for Storage Virtualization Configuring and Sizing Storage Resources Gathering Application I/O Requirements Choosing Storage Based on I/O Workload Determining Benefits of Pooling Storage Resources Choosing the Best Protocol Revision History CHAPTER 3 Using VMware ESX/ESXi Hardware Configuration Working with iscsi and VMware Multipathing Configuration Options (iscsi Only) Best Practices for Connecting to an iscsi Volume Using VMware ESX/ ESXi Provisioning Storage and Connecting to the iscsi Initiator Note IP Address for VMware Host VMkernel Network Interface 12 Create Targets 12 Creating an iscsi Target LUN Enable the iscsi Initiator 16 Discovery 18 Partition and Format a Datastore 20 Working with NFS Shares and VMware NFS Shares, ESX/ESXi, and VirtualCenter Create a New Share Connecting to an NFS Share Using VMware ESX/ESXi Revision History CHAPTER 4 Provide Feedback or Contact Support Revision History Scale Computing i

4 Scale Computing ii

5 List of Figures Two NIC Configuration Four NIC Configuration Six NIC Configuration Create an iscsi Target Dialog Box Create iscsi LUN Dialog Box Selecting Storage Adapters Properties Option Dynamic Discovery Tab Rescan Screen 20 Select Storage Add Storage Wizard Select a LUN to Partition and Format Enter Datastore Name Select Block Size Creating a New Share Select Storage Choose Storage Type Locate Network File System Screen Network File System Screen Scale Computing iii

6 Scale Computing iv

7 Revision History CHAPTER 1 Introduction Welcome to the Scale Computing VMware Best Practices Guide! This guide provides information regarding best practices for integrating Scale Computing clusters with VMware products. Server virtualization and clustered storage have the same main purpose - to provide a way for you to more efficiently use resources. By using virtualization, you can create multiple virtual servers for your different applications and run them all on one physical server. This cuts down the number of physical servers you need while simultaneously increasing the use of each server. When clustered storage and virtualization are paired you will see an improvement in I/O and network throughput. Since multiple virtual servers can run on one physical server, I/O and network throughput are reduced. Scale Computing clusters add to this by striping information across the entire cluster to increase I/O speed of the aggregate solution. When you add additional nodes to your cluster, you also increase I/O and throughput as each node contains its own processor, memory, and disks. This guide provides information that will help you get the most out of pairing your Scale Computing cluster with VMware s virtualization products. Revision History This section contains information describing how this chapter has been revised. Release 2.4.1: No change. Scale Computing 1

8 Revision History Release 2.4: Minor edits. Release 1.0: New chapter. Scale Computing 2

9 Configuring and Sizing Storage Resources CHAPTER 2 Best Practices for Storage Virtualization The Scale Computing cluster is optimized for virtualization, having achieved the VMware Ready certification (among others such as Citrix Ready). VMware Ready is VMware s highest level of endorsement for products created by its established partners. VMware products can use Scale Computing clusters to provide storage for virtual machines. To ensure good results, there are five key tasks to address: Configuring and Sizing Storage Resources Gathering Application I/O Requirements Choosing Storage Based on I/O Workload Determining Benefits of Pooling Storage Resources Choosing the Best Protocol Configuring and Sizing Storage Resources When arranging your storage resources, configure and size them for optimal I/O performance first and storage capacity second. Taking the time to determine the volume of traffic and plan accordingly increases efficiency of retrieval from storage. For best performance, create LUNs of 500 GB or less. Note that Scale Computing s recommendations regarding maximum LUN size coincide with VMware s informal rule-of-thumb suggestions regarding LUN size. If you want to create larger LUNs and use the replication feature, please contact Scale Computing Technical Support for guidance. Scale Computing 3

10 Gathering Application I/O Requirements Gathering Application I/O Requirements Determine I/Os per second and the capacity of your storage resources before placing multiple workloads on them. You never want to exceed the total throughput capacity of your storage resource. You can most easily determine I/Os per second and capacity by examining the physical environment prior to virtualization and then using that information to calculate the throughput each workload generates in the virtual environment. Some tools you can use to take measurements for these calculations are Windows Perfmon and sar in Unix. Choosing Storage Based on I/O Workload Avoid estimating I/O workload whenever possible. Ideally, you should measure your applications I/Os per second and capacity for a period of several days in a physical environment before moving them to a virtualized environment. Use the information gained from your several days of observation to choose what protocol, redundancy protection, and array features to use. If you have any issues, contact Scale Computing Technical Support. Determining Benefits of Pooling Storage Resources While pooling storage resources can increase use and simplify management, there are also trade-offs to be considered. Heavy workloads have an impact on performance. Most of the time it is a good idea to use a shared Virtual Machine File System (VMFS) volume or NFS for virtual disks. However, heavy I/O virtual disks do best on their own VMFS volume, Raw Device Mapping (RDM), or Scale Computing cluster. By taking the time to determine throughput issues, you can choose the best structure for your system. Scale Computing 4

11 Choosing the Best Protocol Choosing the Best Protocol VMware products such as vsphere use iscsi or NFS to provide storage for virtual machines. iscsi can also provide a hard drive for virtual machines. When to use iscsi - if you are setting up a database (MySQL or Microsoft SQL etc.) you should use the iscsi protocol. When to use NFS - if you are setting up a shared pool of virtual machines on Scale Computing storage for ESX or other types of IP based storage. Revision History This section contains information describing how this chapter has been revised. Release Minor edits to Configuring and Sizing Storage Resources section. Release 2.4: Minor edits. Release 1.0: New guide. Scale Computing 5

12 Revision History Scale Computing 6

13 Hardware Configuration CHAPTER 3 Using VMware ESX/ ESXi This chapter covers configuration requirements and options for working with VMware ESX/ ESXi. The following sections provide information regarding best practices and configuration methods for VMware ESX/ESX: Hardware Configuration Working with iscsi and VMware Working with NFS Shares and VMware Hardware Configuration The VMkernel interface performs the software initiator function within ESX/ESXi in addition to VMotion and fault tolerance (FT) logging. The number of network interface cards (NICs) you choose to use determine the placement and configuration of the VMkernel interface on your server. The more NICs you use, the more opportunity you have for good performance and redundancy. Ideally, isolate different types of traffic onto different pairs of NICs. The following figures display recommended vswitch configurations for two, four, and six physical NICs: Figure 3-1, Two NIC Configuration Figure 3-2, Four NIC Configuration Figure 3-3, Six NIC Configuration Two NICs- In a configuration with two NICs, everything resides on a single NIC pair with one vswitch. Use the following assignment: vswitch0: iscsi, production traffic, VMotion Scale Computing 7

14 Hardware Configuration While this configuration works, it is less than ideal. For best performance separate iscsi traffic from your production network. Settings for a two NIC configuration are shown in Figure 3-1, Two NIC Configuration. FIGURE 3-1. Two NIC Configuration Four NICs - In a configuration with four NICs, you can separate traffic types onto two different NIC pairs using vswitch0 and vswitch1. Use the following assignments: vswitch0: production and management traffic vswitch1: iscsi, VMotion, FT logging If your four ports consist of a mix of interfaces, configure your vswitches to use interfaces from different sources to protect against card or bus failure. Settings for a four NIC configuration are shown in Figure 3-2, Four NIC Configuration. Scale Computing 8

15 Hardware Configuration FIGURE 3-2. Four NIC Configuration Six NICs - In a configuration with six NICs, you can separate traffic types onto three different NIC pairs using vswitch0, vswitch1, and vswitch2. Use the following assignments: vswitch0: iscsi traffic vswitch1: production traffic vswitch2: VMotion and FT As with a four NIC configuration, if you are using a mix of interfaces, configure your vswitches to use interfaces from different sources to protect against card or bus failure. Settings for a six NIC configuration are shown in Figure 3-3, Six NIC Configuration. Scale Computing 9

16 Working with iscsi and VMware FIGURE 3-3. Six NIC Configuration Working with iscsi and VMware This section provides details on multipathing and setting up iscsi connections with one-toone volumes and shared volumes. Shared volumes are required if you want to use vmotion or other VMware high availability options. Topics covered include: Multipathing Configuration Options (iscsi Only) Best Practices for Connecting to an iscsi Volume Using VMware ESX/ESXi Provisioning Storage and Connecting to the iscsi Initiator Scale Computing 10

17 Working with iscsi and VMware Multipathing Configuration Options (iscsi Only) The Scale Computing cluster supports ESX multipathing using the Fixed (default) or Most Recently Used (MRU) policy. For best performance and fault tolerance, two NICs and two VMkernels are required on the vswitch enabled for iscsi. While not recommended, you can enable multipathing on a vswitch enabled for iscsi by using a single VMkernel and one or more NICs. However, this configuration does not provide fault tolerance on the host side of the I/O path. NOTE: Do not use SPC-3 PR Compliance enabled targets with VMware. During target configuration, leave the SPC-3 PR Compliance box turned off. If you are using a multiple VMkernel configuration, for each VMkernel, ensure that: The configuration uses one physical NIC as active and the other(s) as unused. You make a note of the VMkernel ports (to the left of the IP address). You access the shell prompt on your ESX host with the console. You make a note of the host bus adapter (HBA) number assigned to the iscsi adapter. You bind the initiator to each of the VMkernel interfaces defined earlier, substituting each VMkernel port and initiator HBA number. For example, if the HBA number is vmhba33, the command looks like this: vmkiscsi-tool -V -a vmk0 vmhba33 vmkiscsi-tool -V -a vmk1 vmhba33 Best Practices for Connecting to an iscsi Volume Using VMware ESX/ESXi Before you start, there are a few important items to keep in mind: You must note the IP addresses for each of your iscsi VMkernel ports to use when you define the target access control list (ACL). If you use an open ACL on iscsi targets, be careful. ESX automatically connects to any volume returned by the discovery process which can result in data corruption. Scale Computing 11

18 Working with iscsi and VMware VMware s file system (VMFS) supports a maximum LUN size of 2 terabytes, but when interfacing with a Scale Computing cluster, the ideal maximum LUN size is 500GB. You would ideally have four Scale Computing LUNs to match up with VMware s 2TB LUN, or some other similar arrangement that keeps the Scale Computing LUNs under 500GB apiece. Provisioning Storage and Connecting to the iscsi Initiator This section covers how to create storage and connect to the iscsi initiator. Topics covered include: Note IP Address for VMware Host VMkernel Network Interface Create Targets Enable the iscsi Initiator Discovery Partition and Format a Datastore Note IP Address for VMware Host VMkernel Network Interface On the Scale Computing cluster, you must provide each target with the IP addresses of the hosts that you want to be able to see the target. To collect the IP address for each host (via the host s VMkernel network interface), take the following steps: 1 Open the vsphere Client. 2 Click on the name of the host you want the IP address for. 3 In the panel that appears to the right of the menu of hosts, click the Configuration tab. 4 Make a note of the IP address provided in the VMkernel Port section. If you plan to create a target that is shared by multiple VMware hosts, make a note of the IP address for each host. Create Targets Scale Computing 12

19 Working with iscsi and VMware Depending on whether you want to create a one-to-one connection between the Scale Computing cluster and a VMware host, or a one-to-many connection between the Scale Computing cluster and VMware hosts, the steps for target creation differ slightly. To create a target take the following steps: 1 Navigate to the iscsi Management screen. 2 To create a new iscsi target, click + below the Targets panel. This brings up the Create iscsi Target dialog box as shown in Figure 3-4, Create an iscsi Target Dialog Box FIGURE 3-4. Create an iscsi Target Dialog Box Scale Computing 13

20 Working with iscsi and VMware 3 Assign the target a name in the Target Name field. NOTE: You cannot begin a target name with a number. 4 Ensure that the Strict SCSI Compliance checkbox is turned on. If this checkbox is not on, your target may not work properly. 5 Ensure that SPC-3 PR Compliance is not enabled. SPC-3 PR Compliance enabled targets are not compatible with VMware. 6 Add or remove IP addresses from the Access List as needed. Access List entries must be full IP addresses in dotted notation: If you are planning to connect to one VMware host, then retrieve the IP address for that host and provide it here. If you are connecting to multiple hosts, you must retrieve the IP address for each host and provide it here. For more information about collecting the IP address for each host, refer to Note IP Address for VMware Host VMkernel Network Interface. 7 Click Create iscsi Target in the lower righthand corner of the dialog box to create your iscsi target with the settings you have selected. After creating your target, create some LUNs and assign them to the target. For more information about creating a LUN, refer to the Creating an iscsi Target LUN section. Creating an iscsi Target LUN The Scale Computing Cluster Manager allows you to create multiple LUNs for each target. To add a LUN to a target, take the following steps: 1 In the iscsi Management screen, in the Targets panel, click the target you want to add a LUN to. Then click the + below the LUNs panel. The Create iscsi LUN dialog box appears, as shown in Figure 3-5, Create iscsi LUN Dialog Box. Scale Computing 14

21 Working with iscsi and VMware FIGURE 3-5. Create iscsi LUN Dialog Box 2 In the LUN Name field, enter a name for your LUN. 3 In the Size field, use the up and down arrows to select the size for your new LUN. Alternatively, you can type a value in using the keyboard. Use the dropdown menu to represent whether the size is megabytes, gigabytes, terabytes, or petabytes. NOTE: Do not create LUNs larger than 500 GB. 4 In the Target field, you can specify the target to which you want this LUN assigned. The default is the currently selected target. 5 In the Replication Targets table, turn on the checkboxes for the targets you want to replicate your LUN to. Choose carefully, you will not be able to add replication targets to your LUN later. If you have not created any replication schedules yet, you will be able to add the LUN to a replication schedule when you first create it. 6 To finish creating your LUN, click Create iscsi LUN to commit the new LUN to the cluster. Scale Computing 15

22 Working with iscsi and VMware NOTE: Unlike CIFS and NFS shares, an iscsi LUN represents a physical device. You cannot change the size of a LUN after creating it. Be sure to plan accordingly when setting up your storage layout. Because LUNs are thin provisioned, you can create a LUN that anticipates future needs without consuming existing resources unnecessarily. Your new LUN appears in the LUNs panel. In addition to the name and size you provided, the Scale Computing Cluster Manager assigns a LUN ID and Serial Number that can be viewed in the LUNs panel. Enable the iscsi Initiator Before connecting to an iscsi volume, you must ensure that the iscsi initiator is enabled. Log in to the vsphere client using a Windows console. You will not have full capabilities in vsphere from other console arrangements (browser, Mac). Then enable the iscsi initiator by taking the following steps: 1 Open the vsphere client. 2 From the list of choices in the menu on the left side of the vsphere client, select the ESX host you want to work with. 3 Locate the tabs across the top of the vsphere client. Click the Configuration tab. 4 Under the Configuration tab, in the menu on the left under Hardware, select Storage Adapters. The Storage Adapter page appears, as shown in Figure 3-6, Selecting Storage Adapters. Scale Computing 16

23 Working with iscsi and VMware FIGURE 3-6. Selecting Storage Adapters 5 Under the Device table at the top of the page, choose the appropriate iscsi storage adapter by clicking on it. Using the example displayed in Figure 3-6, Selecting Storage Adapters, you would choose vmhba33. 6 From the right side of the screen, under the Details section, select Properties... as shown in Figure 3-7, Properties Option. Scale Computing 17

24 Working with iscsi and VMware FIGURE 3-7. Properties Option 7 When the iscsi Initiator (vmhbaxx) Properties window opens, click the Configure... button. The General Properties dialog box appears. 8 Turn on the Enabled checkbox. 9 Click OK. 10 In the iscsi Initiator (vmhba33) Properties window click Close. Wait for ESX to complete the task. The initiator is now enabled and ready for use. Discovery After you enable the iscsi initiator, you must discover the targets on the Scale Computing cluster and make them available to your VMware host. If you are going to create a shared volume, ensure that the target containing the LUN to be used as a shared volume has the IP address for each VMware host included in the access list. To discover targets, take the following steps: 1 After the task completes, re-open the Properties window. 2 Select the Dynamic Discovery tab. The Dynamic Discovery tab opens. iscsi server locations are displayed in the iscsi Server Location table. Scale Computing 18

25 Working with iscsi and VMware 3 At the bottom of the tab, click Add. The Add Send Target Server dialog box appears as shown in Figure 3-1, Dynamic Discovery Tab. FIGURE 3-8. Dynamic Discovery Tab 4 In the iscsi Server field, enter a LAN IP address for each of your targets. SPC-3 PR Compliance enabled targets should not be used with VMware ESX/ESXi. 5 Click OK after you add an address. You will have to add each IP address separately. 6 When you are done adding addresses, return to the Dynamic Discovery tab and click Close. 7 You are prompted to perform a scan after closing the Properties window. Scale Computing 19

26 Working with iscsi and VMware FIGURE 3-9. Rescan Screen 8 After the scan completes, you should see all targets/luns your host has access to. If you wanted the targets to be available to more than one host, make sure they are showing up on each host. If not, be sure you put the IP address for each host in the access list for the target. Partition and Format a Datastore Each datastore uses a single LUN to house virtual machine images. You can create a datastore by doing the following: 1 In the vsphere client under the Configuration tab in the Hardware menu on the left side of the screen, select Storage. The Storage screen appears as shown in Figure 3-7, Select Storage. Scale Computing 20

27 Working with iscsi and VMware FIGURE Select Storage 2 In the same row as the title of the first table, Datastores, are a set of options. Click Add Storage... The Add Storage Wizard appears as shown in Figure 3-8, Add Storage Wizard. 3 Select Disk/LUN and click Next. Scale Computing 21

28 Working with iscsi and VMware FIGURE Add Storage Wizard 4 You are presented with a list of all LUNs that do not contain a recognized VMFS datastore. Select the LUN you wish to partition and format as shown in Figure 3-9, Select a LUN to Partition and Format and click Next. Scale Computing 22

29 Working with iscsi and VMware FIGURE Select a LUN to Partition and Format 5 A screen presenting partition layout information appears. Click Next. 6 You are prompted for a name for the datastore. Enter a unique datastore name in the Enter a datastore name as shown in Figure 3-10, Enter Datastore Name. Scale Computing 23

30 Working with iscsi and VMware FIGURE Enter Datastore Name 7 Click Next. 8 You are prompted to select a block size. In the Maximum file size section of the page, choose an appropriate block size from the drop-down menu. Block size determines the largest file size that the VMFS volume can support. A block size of 1MB allows for a maximum single file size of 256 GB. Scale Computing 24

31 Working with iscsi and VMware FIGURE Select Block Size 9 Click Next. 10 You are presented with a summary of the partitioning and formatting actions that will be performed. If everything looks correct, click Finish. Your newly created datastore appears under Storage. You can select the datastore and click Properties to get additional information about it. If you wanted the datastore to be available to more than one host, it should appear for all hosts, so long as the target the datastore resides on has the IP address for each host in its Access List. Scale Computing 25

32 Working with NFS Shares and VMware Working with NFS Shares and VMware This section contains information about working with NFS shares, how to connect to them on a Scale Computing cluster and how to avoid some pitfalls during setup. The material is divided into the following topics: NFS Shares, ESX/ESXi, and VirtualCenter Create a New Share Connecting to an NFS Share Using VMware ESX/ESXi NFS Shares, ESX/ESXi, and VirtualCenter When you are working with NFS shares, ESX/ESXi, and VirtualCenter, you must use the virtual IP associated with the node containing the NFS share in your Scale Computing cluster. If you have multiple mount points, you must provide the proper virtual IP for each point to VirtualCenter. NOTE: Do NOT use LAN IPs for NFS shares when working with ESX/ESXi and VirtualCenter. If you use ESX/ESXi without VirtualCenter, connect to NFS shares on the Scale Computing cluster using the LAN IP for the node containing the share you want to reach. Create a New Share Take the following steps to create a new share: 1 Click the + underneath the Shares table. The Manage Share dialog box appears as shown in Figure 3-15, Creating a New Share. Scale Computing 26

33 Working with NFS Shares and VMware FIGURE Creating a New Share 2 In the Name field, enter a name for your new share. 3 In the Description field, describe what the share will be used for. 4 Turn on the Quota checkbox if you want to apply a global quota to your share. You can select what unit of measurement you want to use for your quota from the dropdown beside the field you enter the quota number in. This setting limits the amount of storage available to the share over NFS. Because storage allocated to shares is managed by the Scale Computing Cluster Manager, you can adjust the quota for a share after it is created. Like iscsi Scale Computing 27

34 Working with NFS Shares and VMware LUNs, shares are thin provisioned, so you can assign a quota larger than your current cluster capacity. 5 Turn on the checkbox for NFS. The dialog box changes accordingly to show you the NFS table. 6 In the NFS Host Access List table click the + at the bottom of the table to add a new host. You can type a complete IP address, or use the wildcard * to represent a range of addresses. 7 If you want your NFS share to be available to multiple hosts, enter the IP address for each host that will have access to the share. 8 For the new NFS host, click the entry in the table and fill in the hostname. 9 For the new NFS host, click the entry in the table and select the appropriate permissions for the host under the Permissions column. You do this by clicking inside the entry in the Permissions column and selecting your choices from the provided dropdowns. 10 For the new NFS host, click the entry in the table and select the appropriate write style under the Write Style column. NFS provides for both synchronous and asynchronous write styles. Synchronous write style means the NFS protocol waits until it has received an indication data has actually been written to disk before it confirms the write to the client. Asynchronous write style means that NFS immediately indicates a write has completed to the client, even though the actual data may still be in cache. Synchronous writing is much more robust, but significantly impacts performance, as the NFS subsystem must wait for data to be propagated across the cluster before it can confirm the write operation. A single share can support both synchronous and asynchronous client connections. 11 You can delete an NFS permission entry by clicking on the entry you want to delete, then clicking the - at the bottom of the table. 12 From the Replication Targets table, choose the targets you wish to replicate your share to by turning on the checkbox for each appropriate target. Choose wisely, if you do not select replication targets now, you will not be able to add them to the share later. If you have not created any replication schedules yet, you will be able to add the share to a replication schedule when you first create it. 13 When you complete set up of your NFS share, click Create Share. Your new share is now available. Scale Computing 28

35 Working with NFS Shares and VMware Connecting to an NFS Share Using VMware ESX/ESXi This section contains information about how to connect to an existing Network File System (NFS) share on the Scale Computing cluster. To connect to an NFS share, take the following steps: 1 In the vsphere client under the Configuration tab in the Hardware menu on the left side of the screen, select Storage. The Storage screen appears as shown in Figure 3-16, Select Storage. FIGURE Select Storage 2 In the same row as the title of the first table, Datastores, are a set of options. Click Add Storage... The Add Storage Wizard appears as shown in Figure 3-8, Add Storage Wizard. 3 Select Network File System and click Next as shown in Figure 3-17, Choose Storage Type. Scale Computing 29

36 Working with NFS Shares and VMware FIGURE Choose Storage Type 4 The Locate Network File System screen appears as shown in Figure 3-18, Locate Network File System Screen. Enter the address of your server in the Server field. 5 Enter the name of the folder to be used to store the datastore in the Folder field. 6 If you want your NFS to mount as read-only, turn on Mount NFS read only. 7 Enter a name for your new datastore in the Datastore Name field. 8 Click Next. Scale Computing 30

37 Working with NFS Shares and VMware FIGURE Locate Network File System Screen 9 The Network File System Screen appears as shown in Figure 3-19, Network File System Screen. The screen displays your new NFS. If everything looks ok, click Finish. Scale Computing 31

38 Revision History FIGURE Network File System Screen If you found anything you want to change, instead of clicking Finish, click Back until you arrive at the screen containing the information you wish to modify. Revision History This section contains information describing how this chapter has been revised. Release 2.4.1: Scale Computing 32

39 Revision History Revamp of entire chapter, addition of information showing how to create a shared volume in iscsi. Added Create a New Share section. Release 2.4: Minor edits to include information about SPC-3 PR Compliance enabled targets. Release 1.0: New chapter. Scale Computing 33

40 Revision History Scale Computing 34

41 Revision History CHAPTER 4 Provide Feedback or Contact Support If you need help, call SCALE-59 ( ), and someone from the Scale Computing technical support team will be happy to help you. You can also Scale Computing technical support at support@scalecomputing.com, or find us on the web at If you have comments or suggestions regarding documentation, you can send them to documentation@scalecomputing.com. Revision History This section contains information describing how this chapter has been revised. Release 2.4.1: New chapter. Scale Computing 35

42 Revision History Scale Computing 36