EMC STORAGE WITH MICROSOFT HYPER-V VIRTUALIZATION

Transcription

1 White Paper EMC STORAGE WITH MICROSOFT HYPER-V VIRTUALIZATION Design and deployment considerations and best practices using EMC storage solutions EMC Solutions Abstract This white paper examines deployment and integration of a Microsoft Windows Server Hyper-V virtualization solution on EMC storage arrays, with details on integration, storage solutions, availability, and mobility options for Windows Server 2008 R2, Windows Server 2012, and Windows Server 2012 R2 Hyper-V. November 2013

2 Copyright 2013 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All trademarks used herein are the property of their respective owners. Part Number H12557 EMC Storage with Microsoft Hyper-V Virtualization 2

3 Table of contents Executive summary... 5 Business case... 5 Introduction... 6 Purpose... 6 Scope... 6 Audience... 6 Technology overview... 7 Microsoft Hyper-V... 7 Storage connectivity options for virtual machines Virtual machine direct connectivity using iscsi Virtual machine direct connectivity with virtual Fibre Channel SMB 3.0 File Shares Hyper-V Server managed connectivity Virtual Hard Disks Virtual hard disk types Windows Server 2012 R2 new VHD features Online VHD re-sizing Shared virtual hard disk Pass-through disks Storage connectivity summary Availability and mobility for virtual machines Windows failover clustering for Hyper-V servers Windows failover clustering for virtual machines Virtual machine live migrations within clusters Shared-nothing live migration Storage live migration Windows failover clustering with Cluster Shared Volumes Sizing of CSVs Site disaster protection with Hyper-V Replica Site disaster protection with Cluster Enabler Cluster Enabler CSV behavior EMC VPLEX VPLEX Local VPLEX Metro with AccessAnywhere VPLEX Geo with AccessAnywhere VPLEX with Windows failover clustering EMC Storage with Microsoft Hyper-V Virtualization 3

4 Manual or scripted disaster recovery with storage replication Microsoft integration with EMC storage technologies Microsoft System Center Virtual Machine Manager VNX operating environment file based SMI-S Provider Windows Server 2012 Offloaded Data Transfer ODX support requirements Using ODX for virtual machine deployments with SCVMM 2012 R Windows Server 2012 thin provisioning space reclamation EMC Replication Manager EMC Storage Integrator EMC Solutions Enabler Conclusion EMC Storage with Microsoft Hyper-V Virtualization 4

5 Executive summary Business case For many customers, there has been a growing need to provide ever-increasing physical server deployments to service business needs. This has led to several inefficiencies in operational areas, including the overprovisioning of server CPU, memory, and storage. Power and cooling costs, and the requirements for floor space in data centers, grow with each added physical server, whether the resources are overprovisioned or not. Large numbers of physical servers, and the inefficiencies of overprovisioning these servers, result in high costs and a poor return on investment (ROI). You can use Microsoft Windows Server 2008 R2, Windows Server 2012, and Windows Server 2012 R2 Hyper-V to consolidate multiple physical server environments to achieve significant space, power, and cooling savings, and maintain availability and performance targets. EMC storage arrays provide additional value by allowing you to consolidate storage resources, implement advanced high-availability solutions, and provide seamless multi-site protection of data assets. For consolidating data center operations, the Microsoft Hyper-V hypervisor provides a scalable solution for virtualization on the Windows Server environment. Large-scale consolidation saves money when you optimize and consolidate storage resources to a single storage repository. Centralized storage also enhances the advanced features of Hyper-V. EMC Storage with Microsoft Hyper-V Virtualization 5

6 Introduction Purpose This white paper examines deployment and integration of a Microsoft Windows Server Hyper-V virtualization solution on EMC storage arrays. The paper includes details of integration with storage solutions, and also includes availability and mobility options for Windows Server 2008 R2, Windows Server 2012, and Windows Server 2012 R2 Hyper-V. You can use EMC storage arrays for large-scale consolidation efforts that support thousands of connected hosts, tens of thousands of logical units, and advanced internal mechanisms such as Offloaded Data Transfer (ODX) and virtual provisioning. EMC storage arrays are a central part of Windows Server consolidation efforts and provide thin reclamation support, snapshot and clone operations, and multi-site replication solutions to provide disaster restart and recovery solutions. Scope Audience This white paper explains how to use Microsoft Hyper-V with EMC storage arrays to provide RAID protection and improve core system performance. This white paper also explains how to use complementary EMC technologies such as EMC Replication Manager, EMC Storage Integrator (ESI), and EMC Solutions Enabler for Hyper-V environments to improve dynamic placement capabilities for Hyper-V landscapes. This white paper is for administrators who use Microsoft Windows Server 2008 R2, Windows Server 2012, and Windows Server 2012 R2. This white paper is also for administrators, storage architects, customers, and EMC field personnel who want to understand the implementation of Hyper-V solutions on EMC storage platforms. EMC Storage with Microsoft Hyper-V Virtualization 6

7 Technology overview Microsoft Hyper-V Microsoft Windows Server 2008 R2, Windows Server 2012, and Windows Server 2012 R2 provide the Hyper-V server role on the applicable versions of Windows Server. When a Windows Server instance has the Hyper-V role installed, the original operating system instance is called the parent partition. Microsoft provides, supports, and recommends running a Hyper-V server in the minimal footprint of a Windows Server Core installation. Windows Server Installation Options, on Microsoft TechNet, provides more information about Windows Server installation options, including Server Core. Hyper-V products are only available for the 64-bit (x64) release of Microsoft Windows, and require that the server hardware environment supports hardware assisted virtualization (Intel-VT or AMD-V). When you install the Hyper-V server role, you also install the Windows Hyper-V virtualization hypervisor for the parent partition. Figure 1 shows the Hyper-V Manager Management Console (MMC) that you can use to define virtual machine instances. Figure 1. Hyper-V Manager You can use products such as Microsoft System Center Virtual Machine Manager (SCVMM) in more complicated Hyper-V deployments that include many physical servers and virtual machine instances. The SCVMM solution provides a comprehensive management framework with centralized command and control features. SCVMM also includes functionality in the Performance and Resource Optimization (PRO) subsystem, and storage integration based on the Storage Management Initiative Specification (SMI-S). Figure 2 shows these options. EMC Storage with Microsoft Hyper-V Virtualization 7

8 The PRO functionality has a dependency on Microsoft System Center Operations Manager (SCOM). You can use this functionality to build automatic and dynamic management capabilities into a Hyper-V landscape. Storage management integration has a dependency on a storage array with an SMI-S Provider. Such configurations allow dynamic placement of virtual machine resources based on the changing characteristics of the data center. Note: The Storage Automation with System Center 2012 and EMC Storage Systems using SMI-S white paper on EMC Online Support provides information about SCVMM 2012 integration with EMC storage using the SMI-S standard. Figure 2. SCVMM console storage fabric A virtual machine instance typically includes a configuration file defining the configuration of the virtual machine. The virtual machine includes processor count, memory configuration, network connectivity, and other hardware details, and one or more storage devices that represent the storage resources that are used by the operating system instance. These features are configurable through the virtual machine settings options in the Hyper-V MMC, as shown in Figure 3. EMC Storage with Microsoft Hyper-V Virtualization 8

9 Figure 3. Virtual machine settings With Windows Server 2008 R2, you can configure two types of storage devices for a virtual machine from the settings options. Provision a storage device as a Virtual Hard Disk (VHD) that is connected to one of the IDE or SCSI Controller adapters. You can also provision a device that is connected as a physical hard disk (also called a passthrough storage device). Windows Server 2012 includes VHD provisioning, pass-through support, and a new virtual Fibre Channel option. Virtual Fibre Channel creates synthetic Fibre Channel adapters that allow direct storage access using the Fibre Channel protocol. Virtual machine direct connectivity with virtual Fibre Channel on page 11 provides more information. EMC Storage with Microsoft Hyper-V Virtualization 9

10 Storage connectivity options for virtual machines Microsoft Hyper-V configurations support different types of deployment models for connectivity to storage arrays; however, there are two basic methods for a virtual machine to gain access to storage resources. Connectivity is either directly to the virtual host, or connectivity is provisioned through the Hyper-V server. The Hyper-V server manages the storage device that the virtual machines use for connectivity. For direct connectivity from the virtual machine, the virtual machine accesses blockbased storage through either an iscsi connection, or through a virtual Fibre Channel connection. In either case, the Hyper-V server does not physically manage the storage device. You can use network connectivity to access storage resources with shares, using the Server Message Block (SMB) protocol. With Windows Server 2012, you can also place virtual hard disk files onto SMB 3.0 shares for virtual machine use. Note: Although iscsi and SMB are types of direct connectivity, the network connectivity is really occurring over virtualized network adapters managed by the Hyper-V server. Virtual machine direct connectivity using iscsi Virtual machine instances running Windows Server can use storage provided directly to the virtual machine as an iscsi target. For this type of connectivity, the operating system of the virtual machine must implement the Microsoft iscsi Initiator software and must access network resources through a virtual network interface. As the virtual machine itself is directly accessing the iscsi storage device through the network, the operating system within the virtual machine is responsible for all management of the disk device and subsequent volume management. An iscsi target device must be appropriately configured for the virtual machine to access the iscsi devices. For channel redundancy, we recommend using EMC PowerPath or Microsoft Multipath I/O (MPIO) from the virtual machine instead of NIC teaming 1. We also recommend using jumbo frames for I/O intensive applications, by setting the Maximum Transmission Unit (MTU) to 9,000. The MTU should be the same for the storage array, network infrastructure (switch or fabric interconnect), Hyper-V server network cards servicing iscsi traffic, and on the virtual NICs for the virtual machine. The following example sets the MTU on a NIC in Server 2012 using PowerShell: set-netadapteradvancedproperty name iscsia -RegistryKeyword "*JumboPacket" -RegistryValue "9014" For clustered environments, disable the cluster network communication for any network interfaces that you plan to use for iscsi. As shown in Figure 4, you can disable it by opening the iscsi Properties dialog box for the iscsi network that was discovered by Windows failover clustering. 1 In this white paper, "we" refers to the EMC engineering team that validated the solution. EMC Storage with Microsoft Hyper-V Virtualization 10

11 Figure 4. Cluster network properties Note: The EMC Host Connectivity Guide for Windows, available on EMC Online Support, provides more details for configuring the Microsoft iscsi Initiator. One benefit of iscsi storage is the support for clustering within virtual machines. Shared storage clustering between virtual machines is supported with iscsi for both Windows 2008 R2 Hyper-V and Windows Server 2012 Hyper-V. Windows failover clustering for virtual machines on page 32 provides more information. For most configurations you must still provision a VHD device to support the installation of the virtual machine operating system. Hyper-V Server managed connectivity on page 18 provides details. Note: Third-party hardware iscsi solutions can support a boot from an iscsi SAN solution; however, these solutions are beyond the scope of this white paper due to the specific details required for each implementation. Virtual machine direct connectivity with virtual Fibre Channel With Windows Server 2012, you can use the virtual Fibre Channel feature to provide direct Fibre Channel connectivity to storage arrays from virtual machines, giving optimal storage performance and full protocol access. Virtual Fibre Channel also supports guest-based clustering on Hyper-V servers that are running Windows Server Virtual machines must be running Windows Server 2008, Windows Server 2008 R2, or Windows Server 2012 to support the virtual Fibre Channel feature. To support Hyper-V virtual Fibre Channel, you must use N Port ID Virtualization (NPIV) capable FC host bus adapters (HBA) and NPIV capable FC switches. NPIV assigns World Wide Names (WWN) to the virtual Fibre Channel adapters that are presented to a virtual machine. Zoning and masking can then be performed between the storage array front end ports directly to the virtual WWNs created by NPIV for the virtual machines. No zoning or masking is necessary for the Hyper-V server. For the initial configuration of virtual FC, you must create a Fibre Channel SAN within the Hyper-V Virtual SAN Manager, as shown in Figure 5. The Hyper-V SAN is a logical construct where physical HBA ports are assigned. You can place one or multiple HBAs within a Hyper-V SAN for port isolation or for deterministic fabric redundancy. Use the same virtual SAN configuration and naming convention on all Hyper-V servers in a EMC Storage with Microsoft Hyper-V Virtualization 11

12 clustered environment. This ensures that each node can take ownership and host a highly available guest with virtual Fibre Channel. Figure 5. Hyper-V virtual SAN manager After a virtual SAN is created, do the following to present virtual Fibre Channel (FC) controllers to the virtual machine: 1. From within the virtual machine settings (with the virtual machine powered off), click Add Hardware. 2. Click Fibre Channel Adapter. 3. Assign the virtual Fibre Channel adapter to a specific virtual SAN. The virtual adapters are discovered as Microsoft Hyper-V Fibre Channel HBA within the virtual machine operating system, as shown in Figure 6. EMC Storage with Microsoft Hyper-V Virtualization 12

13 Figure 6. Virtual FC adapter discovered within virtual machine We recommend that you add a minimum of two adapters on separate fabrics to ensure no single point of failure, the same as in a typical Fibre Channel topology. If you assign multiple virtual FC adapters to the same virtual SAN, each virtual adapter is pinned to a single physical adapter within the virtual SAN. If a virtual SAN has multiple physical adapters, you can assign each virtual adapter to a different physical adapter (in a round-robin fashion) within that SAN. Configure separate SANs to guarantee redundancy across physical components. Assign a minimum of two virtual adapters across these redundant virtual SANs. Zone and register each virtual adapter (both sets of WWPNs), with devices unmasked across redundant storage controllers. The overall recommended base topology is shown in Figure 7. EMC Storage with Microsoft Hyper-V Virtualization 13

14 Virtual FC Adapter A Virtual Machine Virtual FC Adapter B Virtual SAN A Virtual SAN B Physical Server(s) Physical FC Adapter A Physical FC Adapter B Fabric A Fabric B Storage Director A Storage Director B Figure 7. Virtual Fibre Channel topology Two NPIV based WWPNs, or Hyper-V address sets, are associated with each virtual FC adapter, as shown in Figure 8. Both of these WWPNs must be zoned, masked, or registered to the appropriate storage for live migration to work for that virtual machine. When powered on, only one of the two WWPN addresses are used by the guest at a specified time. If you request a live migration, Hyper-V uses the inactive address to log into the storage array and ensure connectivity before the live migration continues. After the live migration, the previously active WWPN becomes inactive. It is important to validate connectivity and live migration functionality before putting a virtual machine into production using the virtual Fibre Channel feature. EMC Storage with Microsoft Hyper-V Virtualization 14

15 Figure 8. Virtual Fibre Channel adapters You can also use the Get-VM PowerShell cmdlet to get WWPNs by viewing the FibreChannelHostBusAdapters property, or using the Get-VMFibreChannelHba command, as shown in the following WWPN PowerShell example: Get-VMFibreChannelHba -Computername MSTPM3035 -VMName FCPTSMIS ft SanName, WorldWidePortNameSetA, WorldWidePortNameSetB autosize SanName WorldWidePortNameSetA WorldWidePortNameSetB SAN_A C003FF22E51D0000 C003FF22E51D0001 SAN_B C003FF8A1F C003FF8A1F The virtual machine must manage multi pathing when you use the virtual Fibre Channel feature. You can use either native MPIO or EMC PowerPath from within the virtual machine to control load balancing and path failover where multiple virtual FC adapters or multiple targets are configured. Both Microsoft and EMC recommend that you use virtual Fibre Channel instead of pass-through devices when direct storage access is required by the virtual machine or required by the layered software. If components in the environment do not support EMC Storage with Microsoft Hyper-V Virtualization 15

16 NPIV and cannot use virtual FC, pass-through devices can still be used and are supported. SMB 3.0 File Shares SMB 3.0 is the current iteration of the SMB protocol, also known as Common Internet File System (CIFS). The SMB protocol is often used to provide shared access to files over a TCP/IP-based network in Microsoft Windows-based environments. The SMB 3.0 protocol, which is supported by Microsoft for Hyper-V, provides new core performance and high availability enhancement features. Server Message Block overview, on Microsoft TechNet, provides more details about SMB 3.0. SMB file shares can be presented directly to a virtual machine for storage use or, starting with Windows Server 2012, be used as a target to support virtual hard disks used by virtual machines. Both the EMC VNX and VNXe family of storage arrays have SMB 3.0 support in their latest software releases. Some of the SMB 3.0 features supported include Multichannel, Continuous Availability, Offload Copy (Windows Server 2012 Offloaded Data Transfer (ODX), and Directory Leasing. Note: EMC VNX Series: Introduction to SMB 3.0 Support and EMC VNXe Series: Introduction to SMB 3.0 Support, on EMC Online Support, provides more details about SMB 3.0 support for VNX and VNXe arrays. When configuring SMB 3.0 based storage for Hyper-V it is important to include the following steps: 1. Ensure the Hyper-V computer accounts, the SYSTEM account and all Hyper-V administrators have full control permissions to the appropriate file share folder. 2. Enable the SMB 3.0 Continuous Availability (SMBCA) feature. SMBCA is not enabled by default on either VNX or VNXe file system shares. 3. Enable CIFS Synchronous Writes. Synchronous writes are not enabled by default on either VNX or VNXe shares. To enable Continuous Availability on the VNX platform, use the CLI from the control station as in the following example: 1. From an SSH client (like Putty) connect to the VNX control station. 2. Run the server_mount command against the primary data mover that owns the file system. Note the file system and path name, SMB_FS on /SMB_FS for the specified example in Figure 9. Figure 9. VNX file system and path name EMC Storage with Microsoft Hyper-V Virtualization 16

17 3. Mount the file system with the Continuous Availability (CA) option: server_mount server_2 o smbca SMB_FS 4. Export the file system with the CA option server_export <server_number> -P cifs n <share_name> -o type=ca <mount_path> Example: server_export server_2 P cifs n SMB_Share o type=ca /SMB_FS Note: For VNXe, SMBCA can be enabled within Unisphere, under Advanced Options in CIFS Share Detail. For VNX, synchronous writes can be enabled within Unisphere, with the following steps: 1. From Storage Array > Storage Configuration > File Systems, click the Mounts tab. 2. Select the mount associated with the targeted file system, and then click Properties. 3. Select Set Advanced Options and ensure that Direct Writes Enabled and CIFS Sync Writes Enabled are selected. Notes: For VNX OE 8.x, we recommend not enabling direct writes. EMC VNX Unified Best Practices For Performance on EMC Online Support provides additional details. For VNXe, you can enable synchronous writes within Unisphere, under Advanced Attributes in Shared Folder Detail. In Windows, SMB shares can be used by specifying the Universal Naming Convention (UNC) path within PowerShell cmdlets, as in the following example. You can also use Hyper-V Manager, as shown in Figure 10. PowerShell example with SMB storage: New-VHD -Path \\SFSERVER00\SHARE00\VM00.VHDX -Dynamic -SizeBytes 100GB ComputerName Path VhdFormat VhdType FileSize : Size : MinimumSize : LogicalSectorSize : 512 PhysicalSectorSize : 4096 BlockSize : ParentPath : FragmentationPercentage : 0 Alignment : 1 Attached : False : EMCFT302 : \\SFSERVER00\SHARE00\VM00.VHDX : VHDX : Dynamic EMC Storage with Microsoft Hyper-V Virtualization 17

18 DiskNumber : IsDeleted : False Number : Figure 10. Universal Naming Convention to specify a VHD Hyper-V Server managed connectivity When you first deploy a virtual machine in Hyper-V, you must often provide the location for the VHD storage that represents the operating system image. When you format the volume to be used for virtual machine storage, we recommend using a 64 KB allocation unit size (AU). The 64 KB AU helps to ensure the VHD files within the file system are aligned with the boundaries of the underlying storage device. As shown in Figure 11, the initial configuration requires a Hyper-V management name, and also the location for the virtual machine configuration information. If you want to provide high availability for the virtual machine, then this location should represent a SAN device that is available to all nodes within the cluster. In most high availability cases, the storage location for the operating system VHD resides on a Cluster Shared Volume (CSV). EMC Storage with Microsoft Hyper-V Virtualization 18

19 Figure 11. Create a virtual machine Subsequent steps in the New Virtual Machine Wizard request sizing information for memory allocation and network connectivity, which are beyond the scope of this white paper. Microsoft Hyper-V online help provides information about these parameters. Use the New Virtual Machine Wizard to configure the Virtual Hard Disk (VHD or VHDX) for the operating system installation as shown in Figure 12. The default location for the VHD is based on the previous location specified in the Location field, and the VHD Name field is based on the name provided for the virtual machine. EMC Storage with Microsoft Hyper-V Virtualization 19

20 Figure 12. Default definition of the VHD Size the VHD appropriately for the operating system being installed. When configured through the wizard in this manner, the VHD created will be a dynamic VHD or dynamic VHDX file when using Windows Server You can manually configure VHD devices for the virtual machine so as to specify the VHD characteristics. To allow for manual configuration of VHD devices, select Attach a virtual hard disk later. After the New Virtual Machine Wizard has completed, select the Settings option from the Hyper-V Manager console for further modifications to the virtual machine configuration. The configuration is stored in an XML document located in the previously specified virtual machine directory. The name of the configuration file is based on a Global Unique Identifier (GUID) for the virtual machine. You can map manually configured VHDs to either the IDE or SCSI controllers defined in the virtual machine configuration. At least one such VHD must exist to host and install the operating system. Virtual Hard Disks You can define VHD devices that can be later repurposed to virtual machines. You can create VHD devices outside of a virtual machine by selecting New > Hard Disk within Hyper-V Manager, as shown in Figure 13. In clustered environments, you can launch the same hard disk creation wizard from the failover cluster manager by right clicking Roles and then selecting Virtual machines > New Hard Disk. EMC Storage with Microsoft Hyper-V Virtualization 20

21 Figure 13. Virtual hard disk creation Define and associate a VHD from within the Settings option for a virtual machine by selecting the controller (IDE or SCSI) to which a VHD will be associated. To define and map the VHD within the virtual machine, open the settings for the virtual machine as shown in Figure 14. By selecting the hardware device (SCSI Controller in this example), you can define the new VHD to be created and assigned to that controller. Figure 14. Virtual machine hard disk settings For Windows 2008 R2 and Windows Server 2012 VHDs, you must assign virtual machine boot disks to an IDE controller. Server 2008 R2 and Server 2012 support only two devices per IDE controller; because of this, you can configure only four IDE VHD devices for any specified virtual machine. If additional VHD devices are required, you must define them as SCSI controller managed devices. EMC Storage with Microsoft Hyper-V Virtualization 21

22 For Windows Server 2012 R2, you can configure two generations of virtual machines. Generation 1 virtual machines are compatible with the previous version of Hyper-V, and Generation 2 virtual machines add support for the following new functionality: Secure boot Boot from SCSI based VHDs Boot from SCSI based virtual DVD PXE boot using a standard network adapter UEFI firmware support Generation 2 virtual machines also remove support for the legacy network adapter and IDE drives. SCSI controllers support multiple disk devices per controller, and are a more scalable solution for configurations when multiple LUN devices or multiple VHD devices are required. Each virtual machine can have four virtual SCSI controllers, with 64 disks per controller. You can present 256 virtual SCSI disks to a virtual machine. For nonboot devices, use SCSI controllers for presenting additional storage to the virtual machine. For I/O intensive workloads, especially with Windows 2008 R2, you may need to allocate multiple virtual SCSI adapters to a virtual machine. With Windows 2008 R2, each virtual SCSI controller has a single channel, with a maximum queue depth of 256 per adapter. Additionally, a single virtual CPU is used for storage I/O interrupt handling. Due to these limits, you may need to use multiple virtual SCSI adapters to reach the IOPS potential of the underlying storage. Windows Server 2012 greatly reduced the need to present multiple virtual SCSI controllers to improve performance. Windows Server 2012 provides a minimum of one channel per virtual SCSI device/per controller. One channel is added for every 16 virtual CPUs presented to the virtual machine. The queue depth was also changed to 256 per device, and the I/O interrupt handling was changed to be distributed across all virtual CPUs presented to the virtual machine. Because of these changes, you usually need only a single virtual SCSI controller for a virtual machine with Window Server 2012 or Windows Server 2012 R2. Virtual hard disk types Two virtual hard disk formats are available natively with Hyper-V. For Windows Server 2008 R2, the VHD hard disk format is used. With Windows Server 2012 both the VHD and VHDX hard disk formats are supported. The VHD format supports files up to 2 TB in size, while the VHDX format supports files up to 64 TB in size. You can use three different types of VHD disks when you configure new or additional storage devices, as shown in Figure 15. The choice between a fixed size and dynamically expanding format is usually based on the storage utilization requirements, as there is a difference in how storage is allocated for these two types. For this reason, the two selections affect storage provisioning functionality such as that provided by virtual/thin provisioning technologies within storage arrays. EMC Storage with Microsoft Hyper-V Virtualization 22

23 Figure 15. Virtual Hard Disk Wizard A fixed size VHD or VHDX device is fully written to at creation time; as a result, when selecting this VHD type, all storage equal to the size of the VHD file is consumed within the targeted thin pools. The creation of the fixed device can also take a long time because of the requirement to write the full size of the file to the storage array. To help address the time it takes to create a fixed VHD, Windows Server 2012 has a feature called Offloaded Data Transfer, also referred to as ODX. ODX can offload the writing of repeating patterns to a storage device. If ODX is supported by the target storage array, the creation of fixed VHD files (either VHD or VHDX) offloads the series of contiguous writes to be handled by the storage array. This increases the speed at which the VHD is created. Another benefit of the ODX write offload capability, as implemented by both the VMAX and VNX storage arrays, is in virtual provisioning environments; the zeros that represent the fixed VHD are not allocated within the thin pool. This makes a fixed VHD file space efficient with virtual provisioning where ODX is available. The fixed VHD continues to show its full size within the file system. For example, if a 100 GB fixed VHD is created, 100 GB is consumed within the file system, but that space is not be consumed within the thin pool. You can have a fixed VHDX type that is not allocated within a thin pool, where ODX is disabled or not supported by the array. Windows Server 2012 contains a native thin reclamation capability based on the TRIM and UNMAP SCSI commands. This reclaim functionality is supported from within virtual machines using the VHDX disk format. If you present a fixed VHDX to a virtual machine, and that virtual machine is running Windows Server 2012, a file system format causes Windows to issue reclaim requests EMC Storage with Microsoft Hyper-V Virtualization 23

24 for the entire size of the underlying VHDX to the storage device. As a result, the space for the fixed VHDX is no longer allocated in the thin pool. Both the VMAX and VNX support the UNMAP functionality native to Windows Server Dynamically expanding VHD devices do not pre-allocate all storage that is defined for them, even if ODX is available in Server 2012 environments; however, these devices can suffer a slight degradation in performance because storage must be allocated when the operating system or applications within the virtual machine need more allocations. Also, with the dynamic VHD file format there are storage alignment concerns due to the internal constructs of the file as it grows. This causes additional storage performance overhead for both read and write operations. Alignment is not a problem with the Windows Server 2012 VHDX format. The VHDX internal constructs ensure 1 MB alignment on the underlying storage device. Microsoft also made significant improvements for the storage performance of dynamic VHDX files, compared to the legacy VHD format. For most workloads, dynamic VHDX files perform similarly to fixed VHDX files, except for sequential write workloads where the dynamic VHDX file must expand consistently into the file system. After you allocate dynamic VHDX space, there is no difference in performance when compared to a fixed VHDX file. When using dynamic VHD files, you can over provision a file system. A dynamic VHD file has both a current size and a potential size. The current size is the size it consumes in the file system, and the potential size is the size that is specified during creation, and which can be consumed by the virtual machine. An administrator can assign VHD files with potential sizes that total more than the size of the file system. This is not necessarily a problem, but be careful to monitor file systems and ensure adequate free space where dynamic VHD files are used. You can use fixed VHD files to avoid the possibility of over-allocation. You can use third-party tools in the public domain to create a fixed VHD device where, like a dynamic VHD, pre allocation is not executed. Be cautious when using public domain solutions. Although such solutions work, they may not be supported by Microsoft. One such tool is provided at Such third-party tools are unnecessary when ODX is available, or when reclamation support is available and the virtual machine is running Windows Server 2012, as the resulting fixed VHD file is space efficient for virtual provisioning. Based on the considerations in the previous paragraphs, we recommend the following when using VHD files: Windows 2008 R2: Where storage performance is important, use fixed VHDs over dynamic VHDs. If thin storage efficiency is more important than storage performance, use dynamic VHDs. Use fixed VHD files if the potential for over-allocating file system space is not desired. Windows Server 2012 and Server 2012 R2: Use the VHDX file format instead of the VHD format. EMC Storage with Microsoft Hyper-V Virtualization 24

25 Convert VHD files to VHDX files when migrating virtual machines to Hyper-V on Server 2012 if: Performance is of high importance for that file. Advanced functionality such as ODX and TRIM/UNMAP from within a virtual machine running Windows Server 2012 is required. Backwards compatibility with Windows 2008 R2 is not required. Use dynamic VHDX files for general-purpose workloads where initial write performance is not required. Use fixed VHDX files if performance is the most important factor. Use fixed VHDX files if the potential for over-allocating file system space is not desired. Use VHD files instead of pass-through or virtual Fibre Channel storage unless there are specific requirements for these technologies by applications within the virtual machine. Differencing disk is the third VHD format option. This disk device is configured to provide an associated storage area that is created against a source VHD. You can use this style of configuration when, for example, you create a gold master VHD, and you create multiple virtual machine instances. In such configurations, you must protect the gold master from being updated by any individual virtual machine. Each virtual machine must write its own changes. In this instance, the gold master VHD behaves as a read-only device, and all changes written by the virtual machine are saved to the differencing disk device. There will always be an association between the differencing disk and the gold master for, without the original gold master, the differencing disk only maintains changes, and does not represent a fully independent copy. Windows Server 2012 R2 new VHD features Windows Server 2012 R2 offers several new features specific to the VHD format, including online VHD re-sizing and shared VHD: Online VHD re-sizing In previous versions of Hyper-V, you had to power off the virtual machine before resizing a virtual hard disk. Windows Server 2012 R2 allows you to resize a virtual hard disk for the VHDX format, when presented through a virtual SCSI adapter and while the virtual machine is running. This feature is independent of any storage array specific functionality. However, when combined with the ability to expand storage pools and LUNs within EMC storage arrays, the feature offers an end to end capability for adding capacity when using virtual hard disks non-disruptively. Online Virtual Hard Disk Resizing Overview, in Microsoft TechNet, provides more details about this feature. Shared virtual hard disk Shared virtual hard disks enable multiple virtual machines to access the same VHDX file. The main benefit of this functionality is the support for shared storage within a virtual machine-based Windows failover cluster. Microsoft supports the use of shared VHDX files within Cluster Shared Volumes or within SMB file shares that support certain parts of the SMB 3.0 protocol. You can use EMC Storage with Microsoft Hyper-V Virtualization 25

26 shared virtual hard disks with CSVs on EMC block-based storage. VNX or VNXe SMB 3.0 based file shares do not currently support the use of shared VHDs. You can enable shared VHDs from Hyper-V manager or PowerShell. You must power off the virtual machine to modify the required setting. You can use Hyper-V manager, from within the settings of the virtual machine, to enable virtual hard disk sharing, as shown in Figure 16. Figure 16. VHD sharing option With PowerShell, you can enable hard disk sharing either when you add the virtual disk to the virtual machine or after you add the disk. To enable hard disk sharing when adding the virtual disk to a virtual machine: Add-VMHardDiskDrive -VMName VM1 -Path C:\ClusterStorage\Volume1\Shared.vhdx -ShareVirtualDisk To enable hard disk sharing on a virtual disk already added to a virtual machine: $Drive = get-vm VM1 Get-VMHardDiskDrive Where {$_.Path -like "C:\ClusterStorage\Volume1\Shared.vhdx"} Set-VMHardDiskDrive $Drive -SupportPersistentReservations $true Virtual Hard Disk Sharing Overview, in Microsoft TechNet, provides more details about this feature. Pass-through disks Because of the way that I/O is generated to a VHD device located on a volume managed by the parent partition is processed, several levels of indirection are imposed. The virtual machine operating system services I/O within the virtual machine and passes I/O to the storage device. In turn, as the VHD is physically owned by the parent partition, the parent must now receive and drive the I/O to the physical disk that it owns. This multi-level indirection of I/O does not provide the best EMC Storage with Microsoft Hyper-V Virtualization 26

27 performance, although the overhead is relatively small. The best performance occurs when there are the fewest levels of indirection. For storage devices presented to a virtual machine, the best performance occurs when you use virtual Fibre Channel devices, pass-through devices, or iscsi devices directly to the virtual machine. As previously mentioned, we recommend that you use virtual Fibre Channel instead of pass-through devices where it is supported in a specified environment. Pass-through devices must be configured as offline to the parent partition and are therefore inaccessible for any parent managed functions, such as creation or management of volumes. These offline disk devices are then configured as storage devices directly to the virtual machine. You can use the disk management console or, alternatively, the DISKPART command line interface, to transition SAN storage devices between online and offline status. The SAN Policy set, which is accessed through the DISKPART command line interface, manages the default storage devices state. You can use the Hyper-V MMC to configure pass-through devices as shown in Figure 17. Disks can be allocated against a SCSI controller that is configured for the virtual machine. Each SCSI controller can map up to 64 pass-through devices, and up to four discrete SCSI controllers may be configured to an individual virtual machine. This provides support for up to 256 SCSI devices. Figure 17. Configuration of pass-through disk devices for a virtual machine EMC Storage with Microsoft Hyper-V Virtualization 27

28 After you configure the required disk devices as pass-through devices to the virtual machine, the operating system of the virtual machine detects and displays them as shown in Figure 18. In this instance, the virtual machine has been configured with a VHD device that is used as a boot device. The Virtual HD ATA Device is the boot device. The EMC SYMMETRIX SCSI Disk Device identifies the two pass-through devices, as this is the detected storage device from the Windows Server 2008 R2 operating system of the virtual machine. Figure 18. Pass-through storage for an EMC Symmetrix VMAX array You must configure storage devices that have been configured as pass-through devices to a virtual machine in the same way as is typical for storage devices to a physical server. Administrators should follow the recommendations provided for a physical environment, which can include the requirements to align partitions on applicable operating systems. Windows Server 2008 R2 and Windows Server 2012, do not require manual partition alignment, as partitions are automatically aligned to a 1 MB offset. When you create a New Technology File System (NTFS) volume, follow Microsoft SQL Server and Microsoft Exchange Server recommendations for such tasks as selecting an allocation unit size of 64 KB when formatting volumes. You can deploy virtual machine instances that use pass-through devices as the boot device for the operating system disk device. You must define the pass-through device before installing the operating system of the virtual machine and before selecting the pass-through disk (configured through the IDE controller) as the install location. In clustered environments, ensure that the proper resource dependencies are in place for pass-through devices with their respective virtual machine, as shown in Figure 19. The pass-through disk should be within the virtual machine group or role. The virtual machine resource and virtual machine configuration resource must also be made dependent on the pass-through devices. The Update- ClusterVirtualMachineConfiguration PowerShell cmdlet can be used to help set the proper dependencies for a clustered virtual machine. EMC Storage with Microsoft Hyper-V Virtualization 28

29 Figure 19. Pass-through disk dependencies Storage connectivity summary EMC storage arrays provide and support all forms of storage connectivity required by Windows Server 2008 R2 and Windows Server 2012 Hyper-V. You can deploy any form of Hyper-V Server managed by, or directed to virtual machine connectivity, and can even combine multiple forms of connectivity to satisfy application-level requirements. Each form of storage connectivity provides different management or operational features. For example, storage that is provisioned directly to a virtual machine, using virtual Fibre Channel, iscsi connectivity, or pass-through disks, restricts that storage volume only to virtual machines. Conversely, storage allocated in VHD devices created on volumes within the Hyper-V server allows a single LUN to be collocated among any number of virtual machines. The various VHD devices are also shared with, and located on the parent-managed volume. When you use a common volume to collocate VHD devices, you can also affect some high-availability or mobility solutions, because a change to the single LUN affects all virtual machines located on the LUN. This can affect configurations using failover clustering. However, the implementation of Clustered Shared Volumes (CSV) with Windows Server 2008 R2 and Windows Server 2012 failover clustering addresses the need for high availability of consolidated VHD deployments. When you collocate VHD devices onto a single storage LUN, consider how to address the cumulative workload. In cases where an application such as Microsoft SQL Server or Microsoft Exchange Server is deployed within a virtual machine, use sizing best practices to ensure that the underlying storage is able to support the anticipated workload. When collocated VHD devices are placed on a common storage volume (LUN), provision the device to ensure that it can satisfy the cumulative workload of all applications and operating systems located on the VHDs. When storage has been EMC Storage with Microsoft Hyper-V Virtualization 29

30 under-provisioned from a performance perspective, all collocated applications and virtual machines can be adversely affected. Availability and mobility for virtual machines After initial deployment of a virtualized infrastructure, you must often provide high availability for the services running within the environment. In its simplest form, you can provide a high-availability solution by configuring multiple Windows servers into a failover clustering configuration. This style of configuration can cluster up to 16 physical servers with Windows 2008 R2 and up to 64 physical servers with Windows The virtual machines that are configured on shared SAN storage then become resources that can be moved amongst the nodes. Windows failover clustering for Hyper-V servers You can implement Windows failover clustering for use with Hyper-V virtual machines in the same way as implementing the Windows cluster environment for other applications such as SQL Server or Exchange Server. Virtual machines become another form of application that failover clustering can manage and protect. Use the failover cluster management wizard to configure a new application that converts an existing virtual server instance into a highly available configuration. Use the option to configure a virtual machine as shown in Figure 20. Shut down the virtual machine to configure it for high availability, and locate all storage objects, including items such as ISO images that are mounted to the virtual machine, to SAN storage. Failover clustering with Windows 2008 R2 assumes that access to storage objects from all nodes within the cluster is symmetrical. This means that all drive mappings, file locations, and mount points are identical, and during configuration, checks are made to ensure that this condition is met. With failover clustering with Windows Server 2012, you can have asymmetrical storage configurations, where the same storage is not connected to all nodes in the cluster. Such configurations are possible in many geographically dispersed cluster scenarios. In this case, the cluster validation wizards only validate storage against nodes in a common site. Wizard failure results when mandatory requirements are not met. You will receive warnings when failover clustering is not able to verify some of these aspects, or when failure is likely. Read the warnings for information about how to fix the problems. EMC Storage with Microsoft Hyper-V Virtualization 30

31 Figure 20. High Availability Wizard After you import a virtual machine into failover clustering, manage and maintain the virtual machine through the failover cluster management interface. Avoid starting and stopping the virtual machine outside of the control of failover clustering. If the virtual machine shuts down outside of the control of failover clustering, the clustering software assumes that the virtual machine has failed and restarts the virtual machine. Failover Cluster manager, where necessary, launches the required virtual machine management interfaces. Use failover clustering to manage all availability options and state changes for the virtual machine. When you import a virtual machine instance into a high-availability configuration, the machine must include all related storage disk devices so that you can manage the virtual machine correctly. The High Availability Wizard fails if it is unable to include all storage configured for the virtual machine within the cluster environment. Configure all shared storage correctly across the cluster nodes. When you add disk storage devices, correctly configure the devices as shared storage within the cluster. The primary goal of Windows Server failover clustering is to maintain availability of the virtual machine when the virtual machine becomes unavailable due to unforeseen failures; however, this protection does not always maintain the virtual machine state through such transitions. As an example of this style of protection, consider the case of a physical node failure where one or more virtual machines were running. Windows failover clustering detects that the virtual machines are not operational and that a node is no longer available and attempts to restart the virtual machines on a remaining node within the cluster configuration. EMC Storage with Microsoft Hyper-V Virtualization 31

32 Windows failover clustering for virtual machines Availability for the virtual machine resources is ensured through the use of Windows failover clustering at the parent level; however, protection at the virtual machine level may not provide high availability for the applications running within the virtual machines. For example, a server instance cannot start if a virtual machine instance has corrupted files. The high-availability protection for the virtual machine can ensure that the virtual machine is running, but cannot ensure that the operating system itself, or the applications installed on the server, are accessible. Windows failover clustering checks at the application level to ensure that services are accessible. For example, a clustered SQL Server instance continually undergoes Look Alive and Is Alive checks to ensure that the SQL Server instance is accessible to user connections. Implementing clustering within the virtual machines can provide this additional level of protection. You cannot configure a Failover Cluster within virtual machines that are running Windows Server 2008 R2 or Windows Server 2012 using virtual disks or pass-through disks. This limitation is because of the filtering of the necessary SCSI-3 Persistent Reservation commands. However, you can form Windows Cluster configurations with virtual machines that are running Windows Server 2008 R2 with iscsi shared storage devices. In such configurations, the iscsi initiator is implemented within the child virtual machines, and the shared storage is defined on the iscsi LUNs. With Windows Server 2012 you can use both iscsi and virtual Fibre Channel as shared storage within a virtual machine cluster. You can also use SMB file share storage with certain clustered applications, such as SQL Server. If you use SMB file share storage, you should also use SMB 3.0 based file shares. With Windows Server 2012 R2, you can also use VHDs as shared storage between virtual machines that run Windows failover clustering. Windows Server 2012 R2 new VHD features on page 25 provides more information about the shared virtual hard disk feature. Virtual machine live migrations within clusters Movement of virtual machines within a cluster was different for systems before Windows 2008 R2. When an administrator or an automated management tool requested a move, the virtual machine state was saved to a disk and then resumed after disk resources were moved to the target node. This move, or quick migration operation, took so long that outages often occurred, even though the virtual machine state would then resume. With Windows Server 2008 R2 and Windows Server 2012 for failover cluster nodes, you can use the live migration functionality available with the clustering environment. Live migrations move virtual machines transparently between nodes. Unlike quick migration move requests, there is no outage for a client application, and the migration between nodes is completely transparent. To achieve this level of client transparency, live migrations copy the memory state representing the virtual machine from one server to another so as to mitigate any loss of service. Live migration configurations require a robust network configuration between the nodes within the cluster. This network configuration optimizes the memory copy between the nodes and enables an efficient virtual machine transition. For such live migration configurations, you must have at least one dedicated 1 Gb (or greater) network between cluster nodes to enable the memory copy. We also recommend that EMC Storage with Microsoft Hyper-V Virtualization 32