VMware vsphere 5.0: 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP 3PAR Utility Storage

Transcription

1 Deployment Guide VMware vsphere 5.0: 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP 3PAR Utility Storage Create robust, highly available vsphere 5.0 Fibre Channel environments with best-of-breed HP 3PAR Utility Storage and HP ProLiant DL servers

2 Table of contents Emulex Solution Implementer s Series... 3 Executive summary... 3 Introduction... 4 About this guide... 5 Solution components... 6 ESXi HP ProLiant servers... 8 Deploying the solution components... 9 Pre-installation... 9 Updating firmware... 9 Configuring network connectivity Configuring storage Using the VMFS-5 filesystem for additional functionality Support for RDM LUNs in ESXi Configuring Fibre Channel connectivity Deploying ESXi Post-installation Planning the network environment for the host Configuring the HP 82E 8Gb/s HBA Using NPIV to identify HBA ports Configuring the storage array Using the OneCommand Manager plug-in for VMware vcenter Performance comparison Test environment Test methodology Summary Appendix A Installing the CIM provider and vcenter plug-in Installing the CIM provider Installing the vcenter plug-in Appendix B Sample configurations for ESXi 5.0 host and VMs Appendix C Configuring BfS For more information... 39

3 Emulex Solution Implementer s Series This document is part of the Emulex Solution Implementer s Series, providing Implementers (IT administrators and system architects) with solution and deployment information on popular server and software platforms. As a leader in I/O adapters Fibre Channel (FC), Ethernet, iscsi and Fibre Channel over Ethernet (FCoE) the Emulex technology team is taking a lead in providing guidelines for implementing I/O for these solutions. Executive summary With ESXi 5.0, VMware continues to raise the bar for hypervisor products, introducing many new features along with support for more and larger virtual machines (VMs). ESXi 5.0 does not utilize a service console OS; VMware agents and Common Information Model (CIM) providers run directly on the hypervisor layer (VMkernel). As a result, there will be no further releases of ESX. There are three options for communicating with VMkernel: VMware vsphere s enhanced command-line interface (vcli), vsphere PowerCLI or the vsphere Management Assistant (vma) virtual appliance. ESXi 5.0 provides many new features and enhancements in areas such as storage and networking. Indeed, VMware has indicated 1 that there are over 100 new features, claiming, for example, that ESXi 5.0 can run VMs that are four-times as powerful as those supported by earlier versions. However, as shown in Table 1, the new, larger VM will place heavy demands on data center infrastructure. Table 1. Resources supported by various VMware hypervisors Resource Requirements for 95% of applications ESX 1 ESX 2 VMware Infrastructure 3.0/3.5 vsphere 4 vsphere 5 CPU 1 2 CPUs 1 vcpu 2 vcpus 4 vcpus 8 vcpus 32 vcpus Memory <4GB (peak) 2GB/VM 3.6GB/VM 16 64GB/VM 256GB/VM 1,000GB/VM Network <2.4Mb/s <0.5Gb/s 0.9Gb/s 9Gb/s 30Gb/s >36Gb/s Storage <10,000IOPS <5,000IOPS 7,000IOPS 100,000IOPS 300,00IOPS 1,000,000IOPS Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

4 To help you transition to an infrastructure that can support the storage needed by this new generation of VMs, Emulex has validated the functionality and performance of ESXi 5.0 in conjunction with 8Gb/s Fibre Channel connectivity. The tested environment included the best-ofbreed HP ProLiant DL380 G7 server equipped with a dual-channel 8Gb/s Fibre Channel adapter produced for HP by Emulex. Storage was provided byhp 3PAR Utility Storage. In addition to suggesting that 8Gb/s SANs are better suited for the new release of vsphere, this implementer s guide outlines the process for deploying ESXi 5.0 with 8Gb/s Fibre Channel connectivity and outlines the results of performance tests carried out in such an environment. The testing demonstrated that an 8Gb/s Fibre Channel implementation based on the latest Emulex Fibre Channel adapter technology was able to support more VMs than a 4Gb/s implementation, while also reducing CPU utilization on the ESXi 5.0 host. Thus, as VM density increases, the additional burden placed on storage can be mitigated by migrating from 4Gb/s Fibre Channel to 8Gb/s. Intended audience: This document is intended for engineers, system administrators and VMware administrators interested in deploying ESXi 5.0 on an HP ProLiant server featuring an 8Gb/s Fiber Channel adapter. Testing performed in September-October 2011 is described. Introduction ESXi 5.0 supports an unprecedented number of VMs on a single host; moreover, these VMs can reach unprecedented size. As a result, Emulex expects to see more and more workloads being virtualized, as well as additional resources being assigned to existing VMs in order to meet the needs of particular workloads. As noted by VMware 2, the VM will only get bigger and faster Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

5 Figure 1 shows the progress that has been made in virtualizing business-critical applications. source is a bit blurred. Alex: Figure 1: the Figure 1. Comparing the percentages of business-critical application instances virtualized using VMware hypervisors VMware expects 3 more VMs to be deployed than ever before, with ESXi 5.0 allowing these VMs to grow as much as four times larger. With this increased density, virtualized environments must be able to provide additional network and storage resources in order to support the workloads. About this guide This implementer s guide describes how to configure an 8Gb/s Fibre Channel storage array network (SAN) with a ProLiant DL380 G7 server in a vsphere 5.0 environment. Guidelines and instructions are provided for configuring servers, adapters and storage using technical documentation provided by VMware and HP. While the key objective was to describe the configuration and validation of this end-to-end solution, this guide also outlines the following: Implementing boot from SAN (BfS) Using the Emulex OneCommand Manager plug-in for VMware vcenter Server 3 Based on interim results of VMware customer surveys performed in January 2010 and April Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

6 Solution components Emulex built the end-to-end test environment shown in Figure 2 in order to validate 8GB/s Fibre Channel connectivity with ESXi 5.0. Table 2 outlines the key components deployed in this environment. Table 2. Test environment Component Device Comments Tested server HP ProLiant DL380 G7 Virtualization host running ESXi 5.0; 10 identically-configured VMs Management server HP ProLiant DL160 Two VMs: vma (Linux appliance) vsphere Client and Emulex OneCommand Manager or the Emulex OneCommand Manager plug-in for VMware vcenter AD & DNS server Generic Microsoft Windows Server 2008 system Support for Microsoft Active Directory (AD) and Domain Name System (DNS) Storage HP 3PAR Utility Storage F200 Storage System (QL226B) Fibre Channel connectivity 8GB/s host bus adapter (HBA) 8Gb/s fabric switch HP 82E 8Gb Dual Port Fibre Channel HBA Brocade 24-port SAN switch 6 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

7 Figure 2. Lab Test environment If this is your first time installing VMware products on a ProLiant server, it is important for you to have a basic understanding of each of the solution components so that the terminology used in this guide is clear. ESXi 5.0 VMware s latest hypervisor, ESXi 5.0, extends the core capabilities of vsphere 4.1 and provides the foundation for a cloud infrastructure, whether public or private. Areas where you can expect to see improvements after deploying ESXi 5.0 include server consolidation, performance, management, provisioning, and troubleshooting. For more information on vsphere 5.0, refer to What s New in VMware vsphere 5.0 Platform, available at Platform-Technical-Whitepaper.pdf. 7 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

8 HP ProLiant servers HP ProLiant DL380 G7 continue to be the server of choice for many HP shops in the VMware space. These servers are used widely from small and medium-sized businesses (SMBs) to large data centers due to their high availability, scalability, CPU horsepower and memory capacity. In addition, these 2U rack-mount servers save space and power, making them ideal for large data centers moving to a cloud infrastructure. Using Fibre Channel for shared storage When deploying vsphere on ProLiant servers, you should always consider using a SAN so that you can take full advantage of the many hypervisor features and enhancements that require shared storage. While VMware supports most of the more common protocols, Fibre Channel is the predominant choice for shared SAN storage. Storage protocols continue to evolve, introducing options such as iscsi and NFS storage arrays; however, this guide focuses on Fibre Channel connectivity. Server sizing HP has developed an automated tool HP Sizing Tool for VMware vsphere that can help you size and scope a server for a particular vsphere deployment. Based on your responses to a questionnaire, this tool provides a quick, consistent method for identifying the best server for your environment. It also creates a bill of material. For more information on this sizer and other HP solutions for VMware, visit HP Storage HP 3PAR Utility Storage is a highly virtualized storage system built from the ground up to enhance the benefits of a VMware vsphere virtualization platform. This is accomplished by exceeding the storage demands of a server virtualization environment through architectural advantages that include wide-striping and a custom ASIC for true mixed workload support. HP 3PAR Utility Storage combines highly virtualized, autonomically managed and dynamically tiered storage with advanced internal virtualization capabilities to increase administrative efficiency, system utilization, and storage performance. As a result, HP 3PAR Storage boosts VMware vsphere ROI by enabling you to optimize your data center infrastructure, simplify storage administration and maximize virtualization savings. 8 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

9 Deploying the solution components Having introduced the key solution component, this implementer s guide now describes how to configure 8Gb/s Fibre Channel connectivity. Guidelines are provided for the following areas: Pre-installation Configuring network connectivity Configuring storage Deploying ESXi 5.0 Post-installation Pre-installation There are several steps to consider before applying power or installing an operating system on any system. First, you need to ensure rack space and appropriate power and cooling are available. Another key step is to ensure all hardware and software are at the latest firmware levels and download any necessary patches and drivers. The pre-installation section provides links to verify the latest firmware levels for both HP and VMware. The pre-installation process provides suggestions in pre-configuring the network for an ESX host, followed by suggestions for storage systems and storage area networking. As a best practice, HP recommends verifying with HP technical support that you are running the very latest HP firmware and drivers on components such as LAN on motherboard (LOM) and PCI adapters. Updating firmware You can review the latest HP and VMware recommended firmware levels at the following locations: HP: Visit and refer to the Certified ProLiants and Certified HP Storage links under Tools/Resources. VMware: Refer to the VMware Compatibility Guides at Before deploying ESXi 5.0 on a ProLiant server, you should determine the latest versions of the following firmware and, if necessary, update: ESXi host: o System BIOS o LOMs and PCI adapters Storage array and controllers SAN switches 9 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

10 Note Always contact HP support to verify the latest firmware updates and drivers. As always plan your deployment or upgrade before installing software. Read all the documentation provided by VMware and HP before starting. Planning will speed up the process particularly if you intend to deploy multiple servers. With pre-installation activities complete, you can now configure your networking. Configuring network connectivity Before installing ESXi 5.0, you need to understand the network requirements for the particular ESXi host and the connectivity supported by the physical server. For example, while many physical servers feature LOM or integrated network interfaces, ports are typically 1Gb/s, though newer models may provide 10Gb/s Ethernet (10GbE) ports. The ProLiant DL 380 G7 used in the test environment was equipped with two HP NC382i Integrated Dual Port PCI Express Gigabit Server Adapters (that is, four 1Gb/s ports). Thus, Emulex was able to configure the ESXi host with four network interface cards (NICs), as shown in Table 3. Table 3. Four-NIC configuration for ESXi 5.0 Physical NIC Virtual switch Port group(s) vmnic0 vswitch0 Management port and vmotion vmnic1 vswitch0 Management port and vmotion vmnic2 vswitch1 VM network vmnic3 vswitch1 VM network Combining management port traffic, which is relatively light, with vmotion traffic is acceptable in many four-nic deployments. Since vmotion traffic is heavier, it is not a good practice to combine this with VM traffic; in practice, you should consider separating vmotion and VM network traffic on different subnets. Since management port and vmotion traffic is placed on the same virtual switch in the test environment, it is a good practice to use a virtual LAN (VLAN) to enhance security and isolate management port and vmotion traffic. After configuring network connectivity, you can now configure storage. 10 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

11 Configuring storage This section provides information on the following topics: Using the new VMFS-5 filesystem to support additional storage functionality Support for RDM LUNs in ESXi 5.0 Configuring Fibre Channel connectivity o Implementing fabric zoning o Configuring the storage array o Configuring BfS Using the VMFS-5 filesystem for additional functionality Introduced with ESXi 5.0, the VMFS-5 filesystem provides support for VMFS volumes up to 64TB in a single extent. With VMFS-3, 32 extents would have been required to achieve 64TB. Note The volume creation wizard for VMFS-5 uses GUID Partition Table (GPT) format rather than Master Boot Record (MBR), which allows you to create VMFS volumes that are larger than 2TB. GUID refers to a globally-unique identifier. With the ability to create large VMFS volumes, you must now manage storage array queue depth as well as host queue depth. For example, the queue depth of the HP 82E 8Gb/s HBA is set by default to 30 and may be adjusted via Emulex OneCommand Manager, the OneCommand Manager plug-in for vcenter Server or vma. Other benefits delivered by the VMFS-5 filesystem include: Support for up to 130,000 files rather than 30,000 as before Support for 64TB physical-mode Raw Device Mapping (RDM) LUNs Virtual mode allows you to create snapshots, which is beneficial when a file exceeds 2TB For space efficiency, there can now be up to 30,000 8KB sub-blocks There is small-file support for files of 1KB or less; in the past, such files would have occupied entire sub-blocks As you plan the deployment of shared storage, take into consideration the benefits of VMFS-5. For example, if you are migrating from ESXi 4.x to ESXi 5.0, you may also wish to migrate to VMFS-5 to take advantage of the new features and enhancements. 11 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

12 Support for RDM LUNs in ESXi 5.0 While RDM LUNs are still supported in ESXi 5.0, Emulex suggests using them sparingly due to manageability issues. VMware testing (as shown in Figure 3) has demonstrated that RDMs deliver a negligible performance advantage. Figure 3. Comparing IOPS performance with different LUNs Configuring Fibre Channel connectivity The Emulex Fibre Channel driver is in-box, making it is easy to transition 8Gb/s Fibre Channel connectivity from an earlier platform. There is no need to install the driver during the deployment of ESXi 5.0. Thus, the configuration of 8Gb/s Fibre Channel connectivity via HP 82E 8Gb/s HBAs is a simple process in an ESXi 5.0 environment, with just the following stages: Implementing fabric zoning Configuring the storage array Configuring BfS Before you begin the configuration, it is a best practice to review the appropriate VMware Compatibility Guide to ensure firmware on the storage array is at the latest level, as shown in 12 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

13 Figure 4. Figure 4. Showing firmware levels specified in the VMware Compatibility Guide for HP 3PAR Storage IMPORTANT Whenever there is a need to update code or firmware, Emulex recommends backing up your data and configuration. 13 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

14 Implementing fabric zoning Zoning has become a standard component of VMware deployments; indeed, most if not all storage array vendors recommend zoning LUNs that are presented to ESXi hosts. Fabric zones can enhance manageability while providing support for advanced features such as vmotion and Fault Tolerance that require multiple hosts to access the same LUN. Zones can also enhance security. For example, consider what might happen if you were to connect a new server to the same SAN switch as an existing ESXi host. Without zoning or some other security measure, the new server would be able to access the same storage as the existing host and could potentially overwrite the filesystem, obliterating VM data and files. Thus, since the test environment features two 8Gb/s HBA ports, there should ideally be two or more fabric switches, each configured with a zone that includes the particular HBA port. Figure 5 shows the zoning used in the test environment. Figure 5. Zoning in the test environment, with four paths to the LUN 14 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

15 The test environment utilizes two Brocade 8Gb/s SAN switches and a total of four zones, as shown in Table 4. The zones were added to an alias zone configuration, which was then activated. Table 4. Zone configuration for HBA HBA Storage controller Alias Zone Zone configuration Switch Port 0 Port A1 Zone 1 ZoneSet 1 ZoneConfig_1 1 Port B2 Zone 2 ZoneSet 1 ZoneConfig_1 2 Port 1 Port A2 Zone 3 ZoneSet 2 ZoneConfig_1 2 Port B1 Zone 4 ZoneSet 2 ZoneConfig_1 1 This zone configuration will give the ESXi host four paths to a single LUN. At this stage of the deployment, no LUNs have been created; thus, LUNs cannot yet be bound to WWN ports on the HP 82E 8GB/s HBA. Setting the storage array HP 3PAR Utility Storage is very popular in VMware environments due to their extensive virtualization capabilities and ease of management. HP has documented deployment and tuning best practices for this simple-to-use yet robust and highly available array; for more information, refer to HP 3PAR Utility Storage with VMware vsphere Emulex followed HP s best practices when configuring the HP 3PAR Storage system for the test environment. The process included the following stages: Update storage array controller firmware and management software as needed Configure Virtual Domains Configure Virtual LUNs Assign host mode Create hosts Present LUNs to the hosts 15 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

16 Figure 6. Assigning the appropriate host mode as part of the storage array set-up If the correct zoning and host mode have been applied, LUNs will be visible to the assigned hosts. There should be four paths to each LUN for optimal performance and redundancy. Configuring BfS Enterprise server manufacturers such as HP continue to offer local disk storage; however, with the growth of virtualization and the increased use of BfS 4, server configurations are evolving. For example, HP offers a diskless server, which would allow you to deploy ESXi via a USB flash drive. BfS capability is often configured in an ESXi environment, where its benefits include: Enhanced manageability Faster deployment Easier backup 5 Enhanced disaster recovery capabilities The process for configuring BfS via an HP 82E 8GB/s HBA is simple and can be achieved in the following stages: Load the latest boot code to the HP 82E 8GB/s HBA Configure the ESXi host Specify the desired boot volume Place the HP 82E 8GB/s HBA first in the boot order This vendor-specific process is described in more detail in Appendix C Configuring BfS. Note If you plan to install ESXi 5.0 with local storage, Emulex recommends disconnecting the Fibre Channel cables from the SAN to prevent the OS from being accidentally installed on the SAN. Once storage has been configured and you have verified that the hardware has been certified by VMware 6 you can deploy ESXi Whereby the storage array owns the operating system LUN 5 Since the array owns the LUN, array-based copies can be made without server intervention. 6 Refer to the VMware Compatibility Guides at 16 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

17 Deploying ESXi 5.0 This section on deployment will cover the options for installing ESXi 5.0 to an HP ProLiant server. Since ESXi 5.0 has been designed for flexibility, you have a range of new deployment options, including the following: Interactive installation: Suggested for fewer than five hosts Scripted installation: Unattended deployment for multiple hosts vsphere Auto Deploy installation: Suggested for a large number of ESXi hosts with vcenter Server Custom installation: vsphere 5 Image Builder command-line interfaces (CLIs) provide custom updates, patches and drivers Emulex elected to use the interactive method, downloading an ESXi 5.0 image from the VMware website to local storage. The deployment process is fairly straightforward and, in many ways, identical to the deployment of earlier versions of ESXi. Since this process is detailed in VMware s technical documentation, it is not described in this guide. The main points to remember before beginning the deployment are as follows: Check all firmware on host and adapters and make updates as needed Check HBA drivers Plan the networking configuration Plan the Fibre Channel storage configuration and LUNs Decide on local or BfS storage Select the most suitable deployment option for your environment It should not take more than a few minutes to install ESXi 5.0. After you have configured the management network and, if appropriate, enabled lockdown mode, you can proceed with the post-installation process, which includes configuring the ESXi 5.0 host for 8 Gb/s Fibre Channel SAN connectivity. Post-installation After ESXi 5.0 has been deployed on the ESXi 5.0 host, you should review the installation and perform any necessary updates. Consider the following examples: A vendor may have recently released new firmware or drivers. Since it is not configured automatically unless you are using a scripted installation you should configure NIC teaming. 17 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

18 If you are using local storage, remember to reconnect the Fibre Channel cables to the SAN and then verify that the host can login to the fabric and view any LUNs that have been assigned You can now configure the ESXi 5.0 host and storage array for 8 Gb/s Fibre Channel SAN connectivity, which may involve the following activities: Planning the network environment for the host Configuring the HBA Using NPIV to identify HBA ports Configuring the storage array with features such as multipathing Note For host and VM configurations used in the Emulex test environment, refer to Appendix B Sample configurations for ESXi 5.0 host and VMs. 18 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

19 Planning the network environment for the host You should plan the network environment for the host in conjunction with your network administrator. Figure 7 shows the four-nic configuration planned for the Emulex test environment, with ports for the VMkernel, management network and VM network. Figure 7. Four-NIC network configuration for the ESXi host This configuration is commonly used with 8Gb/s Fibre Channel SANs. The following connectivity is provided: vmnic0 vmnic1: Redundant network adapters for the management port and vmotion vmnic2 vmnic 3: VM network traffic 19 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

20 Configuring the HP 82E 8Gb/s HBA If you are migrating to ESXi 5.0 or are installing ESXi 5.0 for the first time and have installed the HP 82E 8Gb/s HBA in an appropriate, full-length PCI slot, with the correct small form-factor pluggable (SFP) transceivers, configuration is simple. Since ESXi 5.0 already provides an in-box VMware driver for Fibre Channel, there is no need to install a driver. After the installation of ESXi 5.0, review the Configuration tab of vcenter Server; you will see your HBAs listed under Storage Adapters, as shown in Figure 8. Figure 8. In this example, ESXi has automatically recognized an 8Gb/s Fibre Channel HBA If necessary, you can use vma to remotely send commands to the ESXi host to verify that the driver has been installed. Using NPIV to identify HBA ports N-Port ID Virtualization (NPIV) 7 is supported in ESXi 5.0. This facility allows a single Fibre Channel HBA port to register with a fabric using multiple worldwide port names (WWNs), each having a unique entity. To learn more, refer to the VMware technical note, Configuring and Trouble-shooting N-Port ID Virtualization, which provides specific information on Emulex adapters. The technical note is available at 7 Maintained by American National Standards Institute (ANSI), Technical Committee T11 20 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

21 Configuring the storage array In addition to the VMFS-5 filesystem, new storage-related features introduced in ESXi 5.0 include: vsphere Storage APIs for Array Integration (VAAI) vsphere Storage APIs for Storage Awareness (VASA) Profile-driven storage Storage Distributed Resource Scheduler (SDRS) Storage vmotion Software Fibre Channel over Ethernet (FCoE) vsphere Storage Appliance (VSA) While these valuable new features are beyond the scope of this guide, be aware that you may be facing storage-related issues after you install ESXi 5.0. For example, after mounting a LUN and formatting it with VMFS-5, you may need to determine if additional, array-specific agents are required to support features such as VAAI or VASA. Note For more information on ESXi 5.0 features that are supported by a particular HP array, you are advised to consult HP technical support. In the test environment, Emulex mounted a LUN and formatted it with VMFS-5. The next concern was multipathing. Configuring multipathing By default with ESXi 5.0, HP 3PAR Storage uses Fixed path policy for active/active storage arrays. This policy maximizes bandwidth utilization by designating the preferred path to each LUN through the appropriate storage controller. According to HP documentation, HP 3PAR Storage also supports Round Robin path policy, which can improve storage performance by load-balancing I/O requests between active paths, sending a fixed number of requests through each in turn. Note The fixed number of I/Os is user-configurable. You are advised to consult HP technical support for their recommendations. 21 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

22 You might consider enabling the array s Asymmetric Logical Unit Access (ALUA) feature, which can improve storage performance in some environments. Using the OneCommand Manager plug-in for VMware vcenter Emulex OneCommand Manager for VMware vcenter integrates the in-depth management of HP 82E 8Gb Dual Port Fibre Channel HBAs into the vcenter management console. Note In order for the OneCommand Manager plug-in for VMware vcenter Server to function, you must also install the appropriate Emulex Common Information Model (CIM) provider on your ESXi 5.0 hosts. For more information on installing the CIM provider and plug-in, refer to Appendix A Installing the CIM provider and vcenter plug-in. Using vcenter to update Emulex adapters on multiple ESXi 5.0 hosts can be a daunting prospect for the VMware administrator. To ease this burden, Emulex has developed a plug-in for vcenter Emulex OneCommand Manager for VMware vcenter plug-in v1.1 that provides single-pane-of-glass manageability from within vcenter. 22 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

23 Figure 9 presents a view of an HP adapter provided by the plug-in. Figure 9. In this example, the OneCommand Manager plug-in is providing information about an HP 82E 8Gb/s HBA. The OneCommand Manager plug-in can manage a mix of Emulex-branded and Emulex OEM adapters (such as the HP 82E 8Gb/s HBA). While OneCommand Manager delivers a range of capabilities, this guide focuses on the plugin s ability to perform firmware updates on a per-port basis while adapters are online, without needing to reboot the host. In addition, new logging capabilities added in version 1.1 are outlined. Note To activate the new firmware, you must schedule downtime in order to re-boot the host. 23 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

24 Using the plug-in to perform firmware updates OneCommand Manager allows you to update the firmware on the HP 82E 8Gb/s HBA without you having to migrate VMs from the host and reboot the server. You can continue working as normal while OneCommand Manager updates the firmware on the host. At some point in the future the host will need to be rebooted so as to activate the firmware; however, you will able to schedule this reboot with the system and vcenter Server administrators. Some environments include not only Fibre Channel adapters but also adapters created by Emulex to support other connectivity. In this case, you could, for example, select the desired protocol to only update UCNAs on each host. Figure 10 shows firmware being updated on a particular host. Figure 10. Firmware being updated on localhost Best practice When using OneCommand Manager, Emulex suggests updating hosts in groups of five. 24 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

25 Using the plug-in to log events OneCommand Manager also provides the ability to create log files for activities associated with the HP 82E 8Gb/s HBA. After this feature is enabled, events are automatically logged; the associated files can be viewed via vcenter s Task & Events tab. Figure 11 shows event logging being enabled. Figure 11. Utilizing User Preferences to enable event logging for the HP 82E 8Gb/s HBA One application for event logging is to troubleshoot the status of a particular link a capability that was missing in earlier versions of OneCommand Manager; however, version 1.1 of the plug-in introduced tighter integration between the plug-in and vcenter. 25 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

26 In the example shown in Figure 12, the Task & Events tab lists event messages indicating that both HP 82E 8Gb/s HBA links (that is, both ports) are up. The WWNs of these ports are also provided. Figure 12. vcenter Server showing the results of enabling event logging for the HP 82E 8Gb/s HBA Performance comparison As VM density increases, the burden placed on storage by applications running on these VMs will also increase. As a result, Emulex has carried out a series of performance tests to demonstrate the ability of 8Gb/s Fibre Channel storage to sustain a significantly higher workload than 4Gb/s Fibre Channel without increasing CPU utilization on the host. Test environment The test environment used by Emulex included the following components: Tested server: ProLiant DL380 G7, configured as follows: o HBA: Either 8Gb/s (HP 82E 8Gb/s HBA) or 4Gb/s (HP PCIe 4Gb Host Bus Adapter 8 ) o RAM: 16 GB SAN o HP 3PAR Utility Storage o Memory Systems RamSan solid-state drives (SSDs), models 320 and 400 Each VM was configured with two 2GB RDM volumes on the SSDs; the HP 3PAR Storage was used to host datastores for the VMs. 8 Emulex LPe Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

27 Figure 13 shows how storage was configured in the test environment. Figure 13. Showing two RDM volumes configured on the RamSan SSDs for each of up to 10 VMs 27 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

28 Test methodology The Iometer tool was used to stress 4KB/s or 8KB/s Fibre Channel storage with the workloads generated by one or more VMs. Two scenarios were used: application testing and a backup reader test. Application testing The following block sizes were used in the application testing: 512B: To create a baseline for the testing 4KB and 8KB: To simulate typical workloads such as a mail server or online transactional processing (OLTP) database 64KB: To simulate a richer multimedia application such as video streaming Note This testing was designed to compare the capabilities of 4KB/s and 8KB/s Fibre Channel storage in a specific environment. Results in your environment may differ. Emulex initiated the testing with 512B blocks to establish a baseline, determining the number of I/Os needed to saturate the Fibre Channel connection between HBA and RDM volume. For subsequent block sizes, Emulex used Iometer to reduce the number of I/Os until, in most cases, the baseline was reached. 28 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

29 1 VM 4GB/s 2VMs 4Gb/s 4 VMs 4Gb/s 1 VM 4GB/s 2VMs 4Gb/s 4 VMs 4Gb/s 1 VM 4GB/s 1 VM 8Gb/s 2 VMs 8Gb/s 4VMs 8Gb/s 2VMs 4Gb/s 4 VMs 4Gb/s 1 VM 8Gb/s 2 VMs 8Gb/s 4VMs 8Gb/s 1 VM 8Gb/s 2 VMs 8Gb/s 4VMs 8Gb/s Solution Implementer s Series Figure 14 presents IOPS capacities for workloads generated by one, two or four VMs. 180, , VM 4GB/s 2VMs 4Gb/s 4 VMs 4Gb/s 140, , , , , , , B 4K 8K Figure 14. Comparing the IOPS delivered by 4Gb/s and 8Gb/s Fibre Channel configurations Significantly more IOPS capacity was available with 8Gb/s Fibre Channel. For example, with the commonly-used 8KB block size and a single VM, IOPS capacity was 100,000 with 8Gb/s Fibre Channel, compared to 50,000 with 4Gb/s Fibre Channel. 29 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

30 Emulex also considered I/O s per percent of CPU utilization which is IOMeter s CPU effectiveness result. This chart illustrates how efficient the I/O subsystem is in CPU utilization. For example, the HP 82E 8Gb/s Fibre Channel adapter using an 8KB block size test shows 128% improvement over HP s 4Gb/s Fibre Channel adapter. 8, , , VM 4GB/s 1 VM 8Gb/s 2VMs 4Gb/s 2 VMs 8Gb/s 4 VMs 4Gb/s 4VMs 8Gb/s 5, , , , , Figure 15. The chart demonstrates CPU effectiveness with 8Gb/s Fibre Channel vs. 4Gb/s Fibre Channel adapters. Video streaming and backup reader testing Emulex also compared the impact of a video streaming and backup on 4Gb/s and 8Gb/s Fibre Channel, again using Iometer to generate the workload. The following configurations were tested: Video streaming: Four VMs with 4Gb/s Fibre Channel 30 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

31 Four VMs with 8Gb/s Fibre Channel Back up reader: Four VMs with 4Gb/s Fibre Channel Eight VMs with 8Gb/s Fibre Channel Both 64KB for video and 256KB blocks for backup were read from the SSDs. As shown in Figure 16, throughput capacity was significantly higher with 8Gb/s Fibre Channel Throughput Gb 64K 8Gb 64K 4Gb 256K 8Gb 256K 0 64K 64K 256K 256K 4Gb 8Gb 4Gb 8Gb Figure 16. When comparing 8Gb/s Fibre Channel to 4Gb/s Fibre Channel, the 64KB throughput improved 126% and the 256KB throughput improved 205%. The HP 82E 8Gb/s HBA supports message signaled interrupts (MSI) and extensions (MSI-X), which are designed to improve host processor utilization and interrupt-handling efficiency. Thus, this HBA can reduce overhead in an MSI-enabled system such as the ProLiant DL380 G7 server. The latency recorded for average response time was 5 milliseconds (ms) when we tested both 64KB and 256KB block size for 8Gb/s Fibre Channel adapter compared to 20 ms on the 4Gb/s Fibre Channel adapter for the same block sizes. For more information, refer to HOWTO.txt;hb=HEAD. 31 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

32 As a result, the HP 82E 8Gb/s HBA helps to reduce backup times, as well as increasing throughput and reducing CPU utilization (as shown in Figure 17) CPU/e Gb 64K 8Gb 64K 4Gb 256K 8Gb 256K 0 64K 64K 256K 256K 4Gb 8Gb 4Gb 8Gb Figure 17. With MSI-X and 8Gb/s Fibre Channel, CPU utilization on the host was approximately 30% lower, even though twice as many VMs were being supported 32 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

33 Summary ESXi 5.0 clearly adds a broad range of features to the hypervisor; however, taking full advantage of these features requires an 8Gb/s SAN in your virtualization environment. Testing carried out be Emulex indicated that an 8Gb/s Fibre Channel implementation based on the HP 82E 8Gb/s HBA was able to support more VMs than a 4Gb/s implementation, while also reducing CPU utilization on the ESXi 5.0 host. Planning the deployment of an ESXi 5.0 host or the migration of an existing host to ESXi 5.0 is a simple process. Using 8Gb/s storage, the resulting infrastructure will be able to meet the demands of the larger VMs you can now create and will allow you to introduce additional business-critical applications. 33 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

34 Appendix A Installing the CIM provider and vcenter plug-in In order for the OneCommand Manager plug-in for VMware vcenter to function, you must also install the appropriate Emulex Common Information Model (CIM) provider on your ESXi 5.0 hosts. Note The VMware Knowledge Base provides an Emulex CIM provider overview for ESXi 5.0. Visit nguage=en_us&cmd=displaykc&externalid= The process is as follows: Note Ideally, the CIM provider should be installed before deploying any VMs on the particular host. The plug-in can be installed before or after deploying VMs. 1. Download the CIM provider package for ESXi 5.0 to any system that can communicate with the particular host via Secure Shell (SSH). The package is available from the Emulex Downloads site at 50/management.html. 2. Install the CIM provider on the ESXi 5.0 host (see Installing the CIM provider). 3. Reboot the host. 4. Install the vcenter plug-in on a remote system (see Installing the vcenter plug-in). 5. Register the host. 6. Use vsphere Client to login to vcenter. Installing the CIM provider 1. Open the secure shell (SSH) port on the ESXi 5.0 host. 2. Use an FTP client (such as WinSCP) to move the CIM package to directory /var/log/vmware/ on the ESXi 5.0 host. 3. Use an SSH client (such as PuTTY) to login to the vma. 4. Login to the ESXi host. 34 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

35 5. As shown in Figure A-1, install the CIM provider by typing the following: esxcli -s ipaddress software vib install -d /var/log/vmware/name-of-file.zip Figure A-1. Installing the CIM provider on the ESXi 5.0 host 6. After the CIM provider (VIB files) has been installed, reboot the host. 7. As shown in Figure A-2, verify that the CIM provider has been installed by typing the following: esxcli -s ipaddress software vib list Figure A-2. Verifying that the CIM provider has been installed 35 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

36 Installing the vcenter plug-in After successfully installing the CIM provider on the desired ESXi 5.0 hosts, you should now install the vcenter plug-in on a remote system. Next, use vsphere Client to login to vcenter to verify that the Emulex OneCommand tab has been added, as shown in Figure A-3. Figure A-3. The Emulex OneCommand tab has been added to vcenter Note There may be additional tabs if you have installed CIM providers for other management applications, such as storage. 36 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

37 Appendix B Sample configurations for ESXi 5.0 host and VMs This appendix presents the configurations used by Emulex in conjunction to compare the performance of 4Gb/s and 8Gb/s Fibre Channel adapters. Note Emulex used the Iometer I/O subsystem measurement and characterization tool as part of the performance testing. Configurations are provided for the following: ESXi 5.0 host see Table B-1 Microsoft Windows Server 2008 R2 VMs (VM1 VM10) see Table B-2 Table B-1. ESXi 5.0 host configuration Component Description Hypervisor ESXi 5.0 Build Host server HBA HP ProLiant DL380 G7 server HP 82E 8Gb/s HBA Local datastore Storage 1 Storage array SAN fabric switch HP 3PAR Utility Storage Brocade 8Gb/s Fibre Channel switch Table B-2. Typical VM configuration Hardware Memory Summary 2048 MB CPUs 2 Video card SCSI controller 0 Hard disk 1, 2 CD/DVD drive 1 Video card LSI Logic SAS Mapped Raw LUN ide1:0 Network adapter 1 VM Network 2 Floppy drive 1 SCSI controller 1, 2 Floppy0 Paravirtual 37 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

38 Appendix C Configuring BfS This appendix outlines the process for configuring BfS: 1. Collaborate with your SAN administrator on provisioning a boot LUN and presenting it to the ESXi 5.0 host. 2. Download the Universal Boot Code firmware for your adapter, available at ypeid=12169&prodseriesid= &taskid=135. For example, select HP 82E 8Gb Dual Port PCIe Fibre Channel Host Bus Adapter Cross operating system (BIOS, Firmware, Diagnostics, etc.). 3. Power on the server and, at the prompt, select Ctrl-E. 4. Install the boot code firmware. 5. Reboot the host and, at the prompt, select Ctrl-E. 6. Specify the adapter port from which the system will be booting. 7. Scan the array and select the boot LUN. 8. Save the settings and reboot the host. 9. Insert the ESXi 5.0 CD into the CD drive. 10. Initiate the install. 11. Select the appropriate disk on the SAN when asked where to install the media. 38 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

39 For more information Migrating to VMware ESXi Performance best practices for VMware vsphere 5.0 HP virtualization with VMware, including a section on ProLiant servers HP storage solutions for VMware HP offline bundle for ESXi 5.0 HP 3PAR Utility Storage with VMware vsphere ESXi-41-Migration-Guide-TWP.pdf es_vsphere5.0.pdf solutions/storage-vmware.html upport/softwaredescription.jsp?lang=en&cc=u s&prodtypeid=15351&prodseriesid= &switem=mtxdd492ace50c df8be&prodnameid = &swEnvOID=4115&swLang=8&taskI d=135&mode=4&idx=1 w-vsphere-3par-utility-storage.pdf To help us improve our documents, please provide feedback at implementerslab@emulex.com. Copyright 2012 Emulex Corporation. The information contained herein is subject to change without notice. The only warranties for Emulex products and services are set forth in the express warranty statements accompanying such products and services. Emulex shall not be liable for technical or editorial errors or omissions contained herein. LightPulse and OneCommand are registered trademarks of Emulex Corporation. HP is a registered trademark in the U.S. and other countries. VMware is a registered trademark of VMware Corporation , February Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

40 World Headquarters 3333 Susan Street, Costa Mesa, California Bangalore, India Beijing, China Dublin, Ireland+35 3 (0) Munich, Germany +49 (0) Paris, France +33 (0) Tokyo, Japan Wokingham, United Kingdom +44 (0)