Best Practices when implementing VMware vsphere in a Dell EqualLogic PS Series SAN Environment

Transcription

1 Technical Report Best Practices when implementing VMware vsphere in a Dell EqualLogic PS Series SAN Environment Abstract This Technical Report covers Dell recommended best practices when configuring a VMware vsphere environment connected to Dell EqualLogic PS Series SANs. TR1091 V1.1

2

3 Copyright 2013 Dell Inc. All Rights Reserved. EqualLogic is a registered trademark of Dell Inc. Dell is a trademark of Dell Inc. All trademarks and registered trademarks mentioned herein are the property of their respective owners. Information in this document is subject to change without notice. Dell Inc. will not be held liable for technical or editorial errors or omissions contained herein. The information in this document is subject to change. Reproduction in any manner whatsoever without the written permission of Dell is strictly prohibited. Authored By: Don Williams, Will Urban [Aug 2013]

4 Preface PS Series arrays optimize resources by automating performance and network load balancing. Additionally, PS Series arrays offer all-inclusive array management software, host software, and free firmware updates. Audience The information in this guide is intended for VMware and Dell EqualLogic administrators and installers. Related Documentation For detailed information about PS Series arrays, groups, volumes, array software, and host software, log in to the Documentation page at the customer support site. Dell Online Services You can learn about Dell products and services using this procedure: 1. Visit or the URL specified in any Dell product information. 2. Use the locale menu or click on the link that specifies your country or region. Dell EqualLogic Storage Solutions To learn more about Dell EqualLogic products and new releases being planned, visit the Dell EqualLogicTechCenter site: Here you can also find articles, demos, online discussions, technical documentation, and more details about the benefits of our product family. For an updated Dell EqualLogic compatibility list please visit the following URL: iii

5 iv

6 Table of Contents Revision Information... vi Executive Summary... 1 Introduction... 1 Host Based Connectivity and High Availability... 1 Keep ESX updated to current revision... 1 iscsi Subnets and Routing Recommendations... 1 Jumbo Frames... 2 CHAP Authentication for Access Control... 2 EqualLogic Storage Heartbeat VMkernel port... 2 Conceptual Image of iscsi connectivity with Storage Heartbeat... 3 Selecting optimal MPIO mode... 4 Host Based Performance... 9 Disabling Delayed ACK... 9 Disabling Large receive offload (LRO) Storage I/O control Datastore Size and Queue depth Virtual Machine Considerations Data Drives VMDK on VMFS iscsi in the Guest VM Conclusion Technical Support and Customer Service v

7 Revision Information The following table describes the release history of this Technical Report. Report Date Document Revision 1.0 August 2013 Initial Release 1.1 August 2013 Added best practices around CHAP and ACL The following table shows the software and firmware used for the preparation of this Technical Report. Vendor Model Software Revision Dell Dell EqualLogic PS Series Firmware 5.x, 6.x VMware VMware vsphere/esx 4.1, 5.0, 5.1 The following table lists the documents referred to in this Technical Report. All PS Series Technical Reports are available on the Customer Support site at: support.dell.com Vendor Dell Dell Dell Dell Dell Dell Dell Dell VMware Document Title TR1067 EqualLogic Virtual Storage Manager: Installation Considerations and Datastore Manager TR1076 Virtual Machine Protection with Dell EqualLogic Virtual Storage Manager v3.5 TR1066 Dell EqualLogic PS Series Arrays: Advanced Storage Features in VMware vsphere TR1074 Configuring and Installing the EqualLogic Multipathing Extension Module for VMware vsphere and PS Series SANs TR1072 Dell EqualLogic PS Arrays Scalability and Growth in Virtual Environments TR1075 Configuring iscsi Connectivity with VMware vsphere 5 and Dell EqualLogic PS Series Storage TR1049 Configuring VMware vsphere Software iscsi With Dell EqualLogic PS Series Storage TR1074 Configuring and Installing the EqualLogic Multipathing Extension Module for VMware vsphere and PS Series SANs KB Routing iscsi Not Supported A multivendor post on using iscsi with VMware vsphere: vi

8 using-iscsi-with-vmware-vsphere.html vii

9

10 EXECUTIVE SUMMARY This technical report is aimed at VMware and Dell EqualLogic PS Series SAN administrators on recommended best practices when configuring their virtual environment. This document will cover connectivity and high availability as well as some common performance tuning best practices. In addition there is a discussion on various methods to implement for data drive connectivity for the virtual machines. INTRODUCTION As more organizations are virtualizing their datacenter using VMware vsphere and Dell EqualLogic PS Series SAN, the need for best practices and configurations will help keep these environments running smoothly. These best practices and recommendations are a combination of various technical report guidelines as well as knowledge provided by the technical support staff at Dell. Various documents exist that go into extensive detail on how to configure some of these settings. In these instances, instead of adding the same information to this document, they will be referenced for further reading. This document will break each section into recommendations for the various versions of ESX as each version might have a slightly different setting or configuration step. HOST BASED CONNECTIVITY AND HIGH AVAILABILITY There are several best practices when connecting your VMware ESX servers to a PS Series SAN that should be implemented not only for performance but for high availability. Keep ESX updated to current revision Customer issues have been resolved by updating to the current ESX version, also known as the build number of ESX. Selecting an individual ESX sever in the ESX client GUI will show the version and build number. iscsi Subnets and Routing Recommendations VMware vsphere 4.1, 5.0, and 5.1 do not support routing when using the iscsi port binding, which is a required for MPIO with EQL storage also it s a best practice for iscsi connectivity. Therefore, the VMkernel ports that will be used to communicate with the PS Series SAN must be on the same subnet and able to communicate directly with the SAN not via a gateway. 1

11 Jumbo Frames Jumbo frames are not required but are recommended for the end to end switching environment. In order to support Jumbo Frames the ESX VMkernel ports and vswitch must be configured to use an MTU of The physical switches must also be capable of using Jumbo frames and properly configured for them. When using iscsi Offload cards, verify the compatibility of the cards and their support for Jumbo Frames before enabling it in the environment. CHAP Authentication for Access Control Manipulating MPIO modules and increasing volume sizes are both great ways to manage volume connections but there are other best practices and recommendations that can also help control how connections are made to PS Series volumes. CHAP authentication for access control lists can be very beneficial. In fact, for larger cluster environments, CHAP is the preferred method of volume access authentication, from an ease of administration point of view. Each PS volume ACL (Access Control List) has a maximum of 16 entries (this is not related to iscsi connections) which can be CHAP accounts, IQN strings or IP addresses. In keeping with designing a scalable, flexible environment, CHAP allows rapid deployment and discovery of volumes with a single ACL entry. For example, Cluster A could have a single CHAP entry to talk to all of its volumes and Cluster B has a different single CHAP entry to talk to all of its volumes. Shared volumes such as templates or shared data would have both Cluster A CHAP and Cluster B CHAP assigned to it dramatically decreasing the administration of ACL lists. Note: In a VMware environment, this also allows rapid movement of datastores between clusters. By simply unregistering the VMs, changing the ACL and rescanning, entire volumes or datastores can be moved between clusters. Using CHAP can help with scalability as well as ease of management. When a new host server is introduced to a cluster, the administrator need only set the CHAP configuration on the iscsi initiator to enable the new host access to all of the clusters storage resources. When a new volume is created, only one ACL entry and an initiator rescan, is all that is needed for all the cluster host servers to be able access the additional storage capacity. EqualLogic Storage Heartbeat VMkernel port In the VMware virtual networking model, certain types of VMkernel network traffic are sent out on a default VMkernel port for each subnet. Typically this is the lowest numbered VMkernel port on that subnet. The iscsi multipathing network configuration requires that each iscsi VMkernel ports are bound to a 2

12 single physical NIC as an uplink. As a result, if the physical NIC that is being used as the uplink for the default VMkernel port goes down, network traffic that is using the default VMkernel port will fail. This includes vmotion traffic, SSH access, and ICMP ping replies. Although existing iscsi aren t directly affected by this condition, a side effect of the suppressed ping replies is that the EqualLogic PS Series group will not be able to accurately determine connectivity during the login process, the initiator will attempt to re-establish the lost sessions. In some scenarios, depending upon array, server and network load, logins may not be completed in a timely manner. This will result in not all the paths to the volume being properly established or even an all paths down (APD) event to occur. To prevent this from occurring, Dell recommends that a highly available VMkernel port, or Storage Heartbeat VMkernel port, be created on the iscsi subnet serving as the default VMkernel port for such outgoing traffic. Because this Storage Heartbeat has access to all of the available physical network cards, the ICMP ping replies will continue to work even if one of the physical network cards has lost connectivity. Conceptual Image of iscsi connectivity with Storage Heartbeat ESX 5.1 The Storage Heartbeat VMkernel port is no longer required for ESX servers running version 5.1 and greater. ESX 5.0 The Storage Heartbeat is recommended for any ESX hosts running version 5.0. For more information on how to configure connectivity including the Storage Heartbeat please refer to technical report TR1075 Configuring iscsi Connectivity with VMware vsphere 5 and Dell EqualLogic PS Series Storage. ESX 4.x The Storage Heartbeat is recommended for all ESX hosts running any version of 4.x software. For more information on how to configure connectivity including the Storage Heartbeat please refer to technical report TR1049 Configuring VMware vsphere Software iscsi With Dell EqualLogic PS Series Storage. 3

13 Note: If you are using Jumbo Frames, the Storage Heartbeat VMkernel port must also be configured for Jumbo Frames, otherwise proper negotiation of Jumbo Frames will likely not occur. Selecting optimal MPIO mode Dell Multipath Extension Module (MEM) The Dell EqualLogic Multipath Extension Module is a software package that can be installed in VMware environments to provide intelligent load balancing, multipathing, and failover. If you have a VMware Enterprise or Enterprise+ license, VMware allows you to install vendor specific MPIO software. These Path Selection Policy (PSP) modules, allow storage vendors to tune MPIO optimally for their hardware. In environments with the appropriate licensing, this is the preferred option for best performance and high availability. The MEM will optimize both datastore IO and Raw Device Mapped (RDM) volumes as well. In addition to providing intelligent MPIO, the MEM also has setup scripts that can automate the creation of the iscsi configuration and connection to the PS Series SAN. While the steps may differ between versions, the MEM is supported for VMware ESX versions 4.1, 5.0, and 5.1 For more information on the Multipath Extension Module and how to install and configure it, refer to technical report TR1074 Configuring and Installing the EqualLogic Multipathing Extension Module for VMware vsphere and PS Series SANs. VMware Round Robin If MEM is not an option, then it is recommended to set the Path Selection Policy to VMware s Round Robin. This will enable all of the physical NICs to participate in load balancing to the datastore. By default VMware will not load balance I/O across all of the physical NICs but will instead use the additional paths only in the event of a failure. This is called FIXED. This could potentially prevent performance scaling past one NIC. In order for Round Robin to be configured as an option, the iscsi connectivity must be properly configured to use more than one physical adapter. In addition each iscsi VMkernel port will be bound to only one of these physical adapters as discussed in the connectivity technical reports. It is possible to change the path selection policy from FIXED to Round Robin in the VMware vcenter GUI by managing the path for each datastore. This can become unwieldy as each datastore has to be manually changed and any new datastores must also be changed. In addition to changing the path selection policy to VMware s Round Robin, there is an additional setting that can be modified to make Round Robin load 4

14 balance better. By default, Round Robin will send 1000 IOs down one path before switching to the next. This may not fully utilize the entire available bandwidth to the SAN when multiple paths are available. It is a recommendation to change this from 1000 IOs per path down to 3 IOs per path to better load balance across all available adapters. A downside to VMware s Round Robin is the IOs per path parameter default can t be changed. This means that new volumes will not inherit the 3 IOs per path setting. As you add datastores and RDMs you must go to each ESX host, and reset the IOs per path parameter. Once configured, the settings will persistent across reboots/startups. To better automate this process there are some scripts available to use. These can be copied onto the ESX server. As you add new volumes, including RDM volumes, you must re-run the script to insure that all the EQL volumes are set. ESX 5.x This script will set all EqualLogic volumes to Round Robin and set the IOPs value to 3. This script can be copied to the console or SSH session to make it easier. This script must be run on all ESX nodes esxcli storage nmp satp set --default-psp=vmw_psp_rr -- satp=vmw_satp_eql ; for i in `esxcli storage nmp device list grep EQLOGIC awk '{print $7}' sed 's/(//g' sed 's/)//g'` ; do esxcli storage nmp device set -d $i --psp=vmw_psp_rr ; esxcli storage nmp psp roundrobin deviceconfig set -d $i -I 3 -t iops ; done This next command will set the default so that new EQL volumes will inherit the proper setting. To be effective this new rule requires a reboot of the ESX host. Until then if you add more EQL volumes, you will need to re-run the script above on all ESX nodes. esxcli storage nmp satp rule add -s "VMW_SATP_EQL" -V "EQLOGIC" -M "100E-00" -P "VMW_PSP_RR" -O "iops=3" You can run this command to verify that the correct settings are in effect: esxcli storage nmp device list The output should look similar to the following: naa.6090a098703e5059e3e2e483c401f002 Device Display Name: EQLOGIC iscsi Disk (naa.6090a098703e5059e3e2e483c401f002) Storage Array Type: VMW_SATP_EQL Storage Array Type Device Config: SATP VMW_SATP_EQL does not support device configuration. Path Selection Policy: VMW_PSP_RR Path Selection Policy Device Config: 5

15 {policy=iops,iops=3,bytes= ,useano=0;lastpathindex=3: NumIOsPending=0,numBytesPending=0} Note the Storage Array Type: VMW_SATP_EQL to indicate the EqualLogic volume and how the Path Selection Policy is VMW_PSP_RR for Round Robin. In addition the policy=iops,iops=3 verifies that the number of IOs per path has been set to 3. ESX 4.x In VMware ESX 4 the following commands can be run to set the default policy for all Dell EqualLogic volumes to Round Robin. This will also automatically set new EqualLogic volumes to Round Robin. The IOs per path parameter must still be set on these new datastores when they are added. To configure the default policy for all EqualLogic volumes to Round Robin type in the following three commands hitting <ENTER> after each one: esxcli nmp satp setdefaultpsp -satp VMW_SATP_EQL -psp VMW_PSP_RR esxcli corestorage claimrule load esxcli corestorage claimrule run In order to change existing datastore volumes to Round Robin the following script can be run: for i in `esxcli nmp device list grep -i -B1 "ay Name: EQLOGIC" grep -i "naa." grep -i -v "ay Name"` ; do esxcli nmp device setpolicy --device $i --psp VMW_PSP_RR; done 6

16 Once existing datastores and all new datastores have been configured for Round Robin, the following script must be executed to change the IOs per path to 3: for i in `esxcli nmp device list grep -i -B1 "ay Name: EQLOGIC" grep -i "naa." grep -i -v "ay Name"` ; do esxcli nmp roundrobin setconfig --device $i --iops 3 --type iops; done Any new datastores will automatically inherit the Round Robin setting but will still need the IOs per path script run to change it back down to 3. To verify the settings type in the following command: esxcli nmp device list The output should look similar to the following: naa.6090a098604ebfe262d Device Display Name: EQLOGIC iscsi Disk (naa.6090a098604ebfe262d ) Storage Array Type: VMW_SATP_EQL Storage Array Type Device Config: SATP VMW_SATP_EQL does not support device configuration. Path Selection Policy: VMW_PSP_RR Path Selection Policy Device Config: {policy=iops,iops=3,bytes= ,useano=0;lastpathindex=0: NumIOsPending=0,numBytesPending=0} Working Paths: vmhba41:c1:t3:l0, vmhba41:c0:t3:l0 Note the Storage Array Type: VMW_SATP_EQL to indicate the EqualLogic volume and how the Path Selection Policy is VMW_PSP_RR for Round Robin. In addition the policy=iops,iops=3 verifies that the number of IOs per path has been set to 3. Default iscsi Timeout Values In addition to configuring Round Robin or MEM for multipathing, there is an additional recommendation for the host iscsi timeout settings. This is the maximum time that the host will wait to receive iscsi traffic from the PS Series SAN. In some versions of ESX this timeout might be too low in large environments which could cause paths to not re-connect or potentially an All Paths Down (APD) event. By default, the MEM configuration script will make an attempt to set each of these timeout values to 60 seconds which is the recommendation. 7

17 ESX 5.0 and 5.1 The default timeout in ESX 5.1 is 5 seconds. This should be changed to 60 seconds on each host to better allow time for iscsi traffic to reach the host. This field is not changeable until the ESX 5.0 server has been updated to patch ESXi or higher. Once this patch is installed the field for LoginTimeout will be editable. 1. Log into the vcenter GUI and on the Home screen click on Hosts and Clusters and select a host. 2. Click on the configuration tab and then Storage Adapters under Hardware and select the iscsi Software Adapter and click Properties. 3. Click the Advanced button and scroll down to the section LoginTimout. Change this value to 60 and hit OK. This can also be done via command line by typing in the following command: esxcli iscsi adapter param set --adapter=vmhba -- key=logintimeout --value=60 ESX 4.1 The default timeout in ESX 4.1 is 15 seconds. This option is not editable until applying update 3. Once the field can be modified the LoginTimeout can be changed to 60 seconds in the Advanced button on the iscsi Software Adapter exactly the same way as ESX 5.0 and

18 HOST BASED PERFORMANCE Disabling Delayed ACK Delayed ACK is a TCP/IP method of allowing segment acknowledgements to piggyback on each other or other data passed over a connection with the goal of reducing IO overhead. Example of Delayed ACK from Multivendor Post on using iscsi with VMware One side effect of delayed ACK is that if the pipeline isn t filled, acknowledgement of data will be delayed. In SANHQ this can be seen as higher latency during lower I/O periods. Latency is measured from the time the data is sent to when the acknowledgement is received. Since we are talking about disk I/O any increase in latency can result in poorer performance. As more customers are moving to 10Gbe and using jumbo frames they may see this show up more often when monitoring their environment with SANHQ. This VMware KB has information on this as well. This includes information on how to disable it on ESX 3.5 through 5.x. displaykc&externalid=

19 ESX 5.1, 5.0 and 4.1 Log into the vcenter GUI and on the Home screen click on Hosts and Clusters and select a host. Click on the configuration tab and then Storage Adapters under Hardware and select the iscsi Software Adapter and click Properties. The next set of steps are similar but depend on if you are setting the property globally for the iscsi initiator, setting it for an individual discovery address, or setting it for a statically assigned target. Globally 1. Click the Advanced button and scroll down until you see Delayed ACK 2. Uncheck the checkbox and hit OK 3. Reboot the host Individual discovery address 1. Click the Dynamic Discovery tab and select the discovery address to modify and click Settings 2. Click the Advanced button and scroll down until you see Delayed ACK 3. Uncheck Inherit from parent 4. Uncheck the checkbox for DelayedAck and hit OK 5. Do this for each discover address you wish to modify 6. Reboot the host Statically assigned target 1. Click the Static Discovery tab and select the static address to modify and click Settings 2. Click the Advanced button and scroll down until you see Delayed ACK 3. Uncheck Inherit from parent 4. Uncheck the checkbox for DelayedAck and hit OK 5. Do this for each static address you wish to modify 6. Reboot the host 10

20 Disabling Large receive offload (LRO) LRO works in a similar way to Delayed ACK, by aggregating packets into a buffer before the received data is sent to the TCP stack. With iscsi storage this inserts latency into the process, potentially reducing performance. Within VMware, the following command will query the current LRO value. # esxcfg-advcfg -g /Net/TcpipDefLROEnabled To set the LRO value to zero (disabled): # esxcfg-advcfg -s 0 /Net/TcpipDefLROEnabled NOTE: a server reboot is required. Info on changing LRO in the Guest network. There are numerous resources online that discuss disabling LRO. VMware has a kb article on disabling LRO to resolve poor performing Linux Guest OS s: displaykc&externalid=

21 Storage I/O control Storage I/O Control (SIOC) is a means of ensuring that the excessive storage I/O demands of a particular virtual machine disk (VMDK) do not negatively impact the storage I/O needs of other VMDKs residing on the same datastore. Because of the various load balancers in the Dell EqualLogic PS Series SAN, the Storage I/O Control (SIOC) feature should also be disabled on EQL based data stores. VMware vcenter might get false information and attempt to load balance at the same time causing unnecessary data movement. For more information about SIOC please refer to technical report TR1066 Dell EqualLogic PS Series Arrays: Advanced Storage Features in VMware vsphere. 12

22 DATASTORE SIZE AND QUEUE DEPTH Continually administrators are trying to optimize their data layout and performance and a frequent question that comes up is how big a datastore should be. While there is no hard set number because every environment is different, you will find recommendations of VMs per datastore. There are a number of factors to include in this including speed of the disks, RAID type, and intensity of the VMs. It is usually recommended to stay in the middle of various recommendations. Having one VM per volume can pose an administrative headache, while putting every single VM on a single datastore could cause a performance bottleneck. Monitor your environment with SANHQ to find volumes that may be underperforming. In addition, you can monitor the queue depth to see if there are outstanding I/Os to a volume which might indicate too many VMs are residing on that datastore. ESX 5.1 and 5.0 With ESX 5.0 and 5.1, the maximum size supported datastore is 64TB. As of the time of this writing, the PS Series SAN can create a maximum size volume of 15TB. ESX 4.1 With ESX 4.1 the maximum volume size is 2TB-512b. If a volume of 2048GB is created the ESX servers will be unable to find and format it. If you create a 2TB volume via the EQL GUI, the LUN size will be 2048GB, use 2047GB instead. If you resize a volume past 2047GB when the capacity is re-read you will no longer be able to access that Datastore or RDM, until the volume size is returned to less than 2048GB. Should this happen contact Dell Support, they can assist you with shrinking the volume while retaining your data. VIRTUAL MACHINE CONSIDERATIONS There are a few best practices that can be performed on the virtual machine to allow them to perform better. One of these is the setting of the disk timeout values to 60 seconds. For Windows Guests the VMware tools will typically do this by default. For more information on the disk timeout settings recommendations, refer to the OS Initiator Guide in the PS Series Firmware download section of the support site. DATA DRIVES There are three primary ways to connect Data Drives to your VMs in your VMware and EqualLogic environment. This is based off the assumption that your OS is running its C:\ or /root off of a VMFS datastore volume. 13

23 VMDK on VMFS iscsi in the Guest (Sometimes called Direct Connect, or Storage Direct) Raw Device Mapping (RDM) VMDK on VMFS A VMDK disk on a VMFS volume is probably the most commonly used practice for data drives. This is when you carve out some additional space on a VMFS datastore and assign it to a virtual machine. Advantages: Easy to configure - Easily add more storage to a VM from free space in existing datastores or provision another datastore. Viewable in Virtual Center - This keeps the administration overhead light as you always know where the data drive resides and makes it easier to integrate vcenter commands such as cloning, templates, and storage vmotion. Doesn t require Storage Team interaction - As long as the datastore has free space you can add data drives to a VM without any storage interaction. Allows for tiering of data drives to different VMFS volumes/pools based on workload - This allows you to have a datastore on a 10k R50 volume and a database drive residing on a 15k R10 volume Uses vsphere MPIO to maintain the same performance benefits. Able to leverage the Dell EqualLogic Virtual Storage Manager (VSM) to take hypervisor consistent snapshots of the VM and its associated data. Disadvantages: No host integration with Auto Snapshot Manager/Microsoft or Linux Edition - Dell provides a suite of tools including a VSS application consistent snapshot tool for NTFS volumes, SQL, Sharepoint, and Exchange. However this tool cannot be leveraged on drives that are VMDK on VMFS volumes. VMDK on VMFS Best Practices When configuring a VMDK on VMFS, there are a few best practices to maintain performance. 1. Click on the virtual machine you want to add the data drive too 2. Click Edit Virtual Machine Settings. 3. Under hardware choose Add -> Hard Disk and click Next. 4. Choose to create a new virtual disk and click Next. 5. Choose the size that you want the VM to see, the disk provisioning options you want and here is where you can choose to keep it on the same VMFS datastore as the VM or browse for another one. 14

24 6. On the next screen this is an important step that is often overlooked. We want to change the virtual SCSI adapter to be a different one than what the base OS drive is residing on. Dell s support staff has seen tremendous performance improvements by just making these few changes. Change the Virtual Device Node to 1:0 or 2,3 for each additional device past the OS drive sitting on 0:0. This will also add a new SCSI Controller to the virtual machine. For VMs that are supported you should change it from LSI to Paravirtual SCSI Adapter for better performance. Note: Virtualized RDMs and VMDKs are still limited to 2TB-512b in size in all versions of ESX. iscsi in the Guest VM Because iscsi traffic is just standard network traffic, you can take advantage of iscsi in the guest VM by using the guest VM s iscsi software initiator. This also allows vmotion and all of the other tools to work because to the VM, it s just network traffic. 15

25 Advantages: Overcome 2TB Limits of VMFS/RDM in vsphere If you are using large file systems and still on vsphere 4.1 this ability allows your VMs to talk to a volume much larger than a 2TB volume or avoid using extents. Utilize host integration software such as Auto Snapshot Manger/Microsoft Edition and off host backup - By using iscsi in the guest VM with iscsi software initiator attached volumes, you can take advantage of the data protection snapshot tools provided by Dell. This also gives you the benefit of taking advantage of transportable snapshot technologies with backup vendors to offload snapshots to a backup server to eliminate LAN traffic and backup window contention. Isolates the data drives for Data Protection/DR strategies - This technique also allows you to isolate your data drives to a volume that may have different replication and snapshot strategies than the volume that houses the parent VM. Can be mounted on physical or other virtual machines - If you have a VM crash or just want to dismount and remount the data you can easily do this through the iscsi initiator. Uses same best practices from physical environments - This technique is exactly the same as from the physical environment. If you P2V a server that has iscsi attached volumes, you can continue to run the VM with very little change to the server. Allows for tiering of data drives to different volumes/pools based on workload - Since the data drives reside on a volume, you can have faster performing or larger data size data residing on the SAN tier that they require which may be different than the VMs. Disadvantages: Isn t visible to Virtual Center - Because the volume is managed from the VM itself, vcenter will not see it in the storage tab and you won t see it connected to the VM when editing the properties. This can cause some additional management overhead. Requires Storage Team intervention - Since you are creating a brand new volume, you need to create the volume to be seen by the VM not the ESX environment. This means installing and configuring the iscsi software initiator, connecting to the SAN with proper pathing and configuring and formatting the volume to be seen by the VM. Needs to be considered for DR Plans separate from VMs - Because the volume isn t seen by vcenter or any of the VMware tools like Site Recovery Manager, you need to have different considerations for the protection of these volumes. They need to be put into the recovery and protection plans according to their SLAs. 16

26 iscsi in the Guest VM Best Practices Configuring iscsi in the guest is exactly the same as configuring it for a physical server. Install the iscsi initiator into the guest VM, connect to the SAN and format the presented volume. There are, however, some considerations to think about on the ESX servers. 1. Normally the iscsi network and Public LAN network are isolated so you first have to make sure that the VM can see the PS Series SAN. 2. Verify the VM is configured properly to take advantage of MPIO inside the guest VM if you need it. 3. Modify the vswitch that is delivering iscsi traffic to include two iscsi Guest Port Groups. 4. In the diagram below these are iscsi Guest 1 and iscsi Guest 2. These are installed on the vswitch that can communicate on the SAN network. 5. Once those are created, the next thing to do is to guarantee traffic across the physical NICs. If you monitor esxtop even if you give a VM two virtual NICs, ESX doesn t guarantee the traffic across all the physical NICs from a single VM so we have to force it to do that. But you don t want to explicitly set it so that you lose failover ability so we just take advantage of the vswitch NIC Teaming. 6. So inside the vswitch, select the first iscsi Guest and click properties. 7. Click the NIC Teaming tab and put a check box in Override vswitch Failover order. 8. Make one of the physical NICs active and the other standby (not unused). Do this for the other iscsi Guest and use the other adapter. 17

27 9. The last thing to do is then assign new Ethernet adapters to the VM and choose the iscsi Guest networks as one of the networks you just created. Raw Device Mapped LUNs Raw Device Mapped LUNs (RDMs) are used when you want to isolate the data onto volumes but still retain its view inside vcenter. Advantages: Easy to configure - Each RDM is a single volume on the SAN. Viewable in Virtual Center under the VMs it is attached to - The volume will show up inside vcenter under the VM and allows for vcenter integration with software such as SRM. Allows for tiering of data drives to different volumes or pools based on workload - You can isolate the data drive RDMs to another tier of storage for performance. Uses vsphere MPIO for high bandwidth Apps - Continues to leverage the MEM or Round Robin for multipathing. Isolates data from OS and enable the use of 3rd party applications which work with RDMs - There are some 3rd party applications that require some form of RDM for either backup or protection or tools. Disadvantages: No guest integration with Auto Snapshot Manager/Microsoft Edition - ASM/ME cannot recognize RDM volumes for smart copy operations. No VM integration with Dell EqualLogic Virtual Storage Manager Smartcopies - VSM doesn t recognize RDM volumes for smart copy operations for VM snapshots. Each ESX server needs connectivity so connection counts need to be evaluated - Since each ESX server needs to see the RDM volume you will 18

28 have multiple connections to the RDM even though only one server is using the volume at a time. When configuring RDMs, follow the same best practices as VMDK on VMFS with the new SCSI adapter and paravirtualized adapter if supported. VM Data Drive Formatting Options Dell has found improvements in performance by using a 64k partition alignment. This will be necessary in Windows Operating Systems before Windows 2008, Solaris, and Linux guest OS s. Additionally, SQL, Exchange, and other IO intensive applications will benefit from formatting the filesystem with a 64K cluster size. This better aligns IOs with the 64K stripe size of the EQL arrays. CONCLUSION As more environments become virtualized, it is necessary to follow best practices to maintain not only the performance of these environments, but also the high availability. Dell recommends these practices be followed even if there is no apparent performance issue as it allows the environment to be in a known healthy state. 19

29 Technical Support and Customer Service Dell support service is available to answer your questions about PS Series SAN arrays. Contacting Dell 1. If you have an Express Service Code, have it ready. The code helps the Dell automated support telephone system direct your call more efficiently. 2. If you are a customer in the United States or Canada in need of technical support, call If not, go to Step Visit support.dell.com/equallogic. 4. Log in, or click Create Account to request a new support account. 5. At the top right, click Contact Us, and call the phone number or select the link for the type of support you need. Warranty Information The MODEL array warranty is included in the shipping box. For information about registering a warranty, visit 20