Virtualized High Availability and Disaster Recovery with Microsoft Hyper-V

Transcription

1 With the System Storage DS5000, System x 3850M2, and BladeCenter H Authors: Scott Smith i, David West ii, Reviewed by: David Hartman iii Document version: 2.3 Document date: November 7, 2008 Page 1

2 Table of Contents Introduction... 3 The Solution... 3 Solution Overview... 3 Solution Requirements... 4 Hyper-V with Failover Clustering Requirements... 4 DS 5000 Enhanced Remote Mirroring Requirements... 5 Hyper-V Features... 5 DS 5000 Features... 6 System x 3850M2 and Blade Center Features... 7 System x 3850 M2 and x3950 M2 Key Features... 7 BladeCenter Key Features... 8 Hyper-V Installation and Configuration... 9 To Install the Hyper-V Role:... 9 To Create a Virtual Network: Cluster Installation and Configuration Installing Windows Server Failover Clustering Validate the Cluster Configuration Creating a Cluster Virtual Machine Installation and Configuration Virtual Machine Creation and Configuring Automatic Start Action Virtual machine creation Reconfigure the automatic start action for the virtual machine Make the Virtual Machine Highly Available Configure the Virtual Machine Test a Planned Failover Test an unplanned failover DS 5000 Storage Configuration LUN Replication with Enhanced Remote Mirroring Overview of Enhanced Remote Mirroring Enabling and Configuring Remote Mirroring Remote Mirror Failover and Role Reversal Remote Mirror failback process Conclusion Page 2

3 Introduction Today s virtualization solutions help customers reduce datacenter complexity and management costs, providing dynamic and flexible configurations across multiple resources. Companies of all sizes are interested in energy and space efficiency, more centralized management, and flexible disaster recovery options to reduce their total cost of ownership. With Microsoft Hyper-V and Windows Server 2008, organizations are now even better equipped to streamline their IT infrastructure and take advantage of the more efficient storage and server solutions IBM has to offer. High-availability and disaster-recovery (HA/DR) are two key areas where virtualization can also offer significant flexibility and cost savings. The Solution By implementing Hyper-V for high availability and disaster-recovery scenarios, using the IBM System Storage DS5000, System x3850 M2 and BladeCenter servers, elegant and sophisticated solutions can be achieved. This reference architecture outlines one possible solution for intra-site high availability and inter-site disaster recovery scenarios. This solution contains the following features: Windows Server 2008 and Hyper-V with failover cluster support which provides efficient high availability. Quick migration of Hyper-V guest machines between IBM HS21XM Blades with automatic fail-over. Optimized disaster recovery with the flexibility of virtualization and efficiency of the IBM BladeCenter, System x3850 M2 and the DS5000 modular storage system. Reliable replication of data between sites with the IBM DS5000 and Enhanced Remote Mirroring (ERM). A robust, yet easy to manage and configure HA/DR scenario. Solution Overview The configuration includes the simulation of two geographically dispersed sites, with local site HA provided by two failover clustered HS21XM Blade servers hosting multiple Hyper-V virtual machines (guests). The DR simulation consists of Enhanced Remote Mirror LUN replication between the two sites, with the capability of bringing the Hyper-V guests online at the second site if the main datacenter becomes unavailable. For this proof-of-concept, 2 guests ran on one node of the failover cluster, and one guest ran on the second node to validate failover. The given solution could of course host many more guests in a production environment, dependent upon resource availability. Page 3

4 Enhanced Remote Mirroring is a premium feature of the IBM System Storage DS5000. Data replication and control is handled by the array controllers over dedicated replication links with negligible impact on source host applications. Advanced features such as ordered-writes and consistency group support help ensure that complex application LUN layouts and database integrity are preserved. Figure 1. HA and DR solution overview Solution Requirements This solution demonstrates Hyper-V running on System x hardware (HS21 XM blades, and the 3850 M2), backed by two IBM DS5000 storage controllers running Enhance Remote Mirroring. This is a disaster recovery solution that limits potential downtime to the few minutes it takes to break the mirrored drive sets and bring the failed guests back online on your DR server (in this case, the 3850 M2). Below are the system requirements for implementing such a solution. Hyper-V with Failover Clustering Requirements Microsoft Windows Server 2008 x64 has Hyper-V built-in to the operating system. The parent partition must run the x64 version of the operating system. The requirements to implement this solution are as follows: Servers that support Windows Server 2008 x64. In this solution, two HS21 XM Blades were used for clustered servers and a 3850 M2 was used as the DR server. Virtualization extensions and execute disable bits enabled in the BIOS Fiber HBA for attachment to shared storage. QLogic HBAs were used in HS21-XM and an Emulex HBAs were used in 3850 M2. Page 4

5 The parent partition requires a minimum of 512 MB of memory which is the minimum amount of memory needed to support Windows Server 2008, however 2GB (or more) is recommended. o At this time Hyper-V does not support memory over-commitment. This means there needs to be enough physical memory in the system to support the parent partition and all running virtual machines. Two Ethernet network interfaces and two network switches. o One (or more) public network interfaces, to be used by the cluster as well as virtual machines. o One private network interface to be used by the cluster. Active Directory environment DS 5000 Enhanced Remote Mirroring Requirements The DS5000-specific requirements when running ERM are as follows: Two DS5000 storage systems, with LUNs configured the same size on each, and running identical firmware versions. A SAN environment is required to isolate host traffic, controller pairs, and dedicated ERM connections. Activation and management of Remote Mirror relationships must be done over an Ethernet connection, with all DS5000 storage systems able to communicate with the management work station. The ERM feature requires a Premium Feature License for each DS5000 storage unit. One available host port on each DS5000 system, which becomes dedicated to mirroring traffic once ERM is activated. Hyper-V Features Hyper-V provides key functionality such as scalability, high performance virtualization services, reliability, symmetric multi-processing support, and manageability. Hyper-V also provides customers an ideal platform to improve IT efficiencies and hardware utilization through consolidation of production servers into virtualized guests. Additionally, Hyper-V provides increased business continuity by facilitating the quick migration of severs among multiple hardware computing resources for rapidly relocating and/or repurposing servers in your IT environment. Hyper-V features include: Native x64 Hypervisor (parent partition) with all device drivers supported by Windows Server 2008 x64 also supported under Hyper-V. Support for both x86 and x64 virtual machines running concurrently. Up to 4 virtual CPUs per virtual machine (Standard Edition) Up to 64GB of RAM per virtual machine Virtual machine snapshots, to capture the state of a running virtual machine. This allows you to easily revert back to any previous snapshot as needed. Page 5

6 Windows Server 2008 x64 supports the Hyper-V role in either Full Install (GUI), or Core Install (Command Line). Hyper-V leverages Microsoft Cluster Services to make virtual machines clustered resources that can utilize Quick Migration for failover support. The Microsoft Management Console (MMC) 3.0 interface for familiar & robust out-ofbox management tools. Microsoft System Center Virtual Machine Manager for enterprise-wide Hyper-V management. Operating systems supported under Hyper-V: o Windows Server 2008 x86 and x64. o Windows Server 2003 x86 and x64. o SLES 10 o DS 5000 Features Windows Server 2000 / Windows XP / Windows Vista The IBM System Storage DS5000 sets new standards for performance, scalability, reliability, availability, and flexibility for midrange storage systems. As IBM s most powerful midrange storage system, the DS5000 is the ideal platform for a virtualized environment that can keep pace with your business growth. Organizations can buy only the capacity needed initially, and can then dynamically upgrade and reconfigure additional capacity and features later to meet changing business requirements, all without incurring any system downtime. The DS5000 delivers industry-leading performance and is equally adept at supporting transactional-applications, such as databases and OLTP; throughput-intensive applications, such as HPC and rich media, and concurrent workloads which are also ideal for consolidation and virtualization. With its relentless performance, and architected to provide the highest reliability and availability, the DS5000 storage system is comfortable supporting the most demanding service level agreements (SLAs). Page 6

7 With flexibility and performance headroom ideal for virtualized environments, the DS5000 offers the following key features: Flexible host interface options which are 8 Gb/s Fibre Channel and 10 Gb/s iscsi ready. Sixteen 4 Gb/s Fibre Channel drive interfaces for supporting up to 256 drives in initial release, with future support for 448 FC/SATA drives, using EXP5000/EXP810 drive expansion units. Up to 16 GB of dedicated data cache (8 GB per controller) in initial release, with future support for 32 GB. Dedicated cache mirroring channels and persistent cache backup in the event of a power failure. Support for RAID 6, 5, 3, 10, 1, 0. Two performance levels (base and high) with ability to field-upgrade. Base model is the DS5100 High-performance model is the DS5300. Remote Volume Mirroring and FlashCopy premium features for Volume Shadow Copy (VSS) supported backups and flexible DR scenarios. Break-through performance levels over 5x greater than the DS4800. System x 3850M2 and Blade Center Features System x 3850 M2 and x3950 M2 Key Features The IBM System x3850 M2 and x3950 M2 servers provide a scalable, cost-effective and highly flexible x64 platform. With the ability to scale-up while maintaining balanced performance between processors, memory, and I/O, these servers can easily accommodate business expansion and the resulting need for additional application space required in highly virtualized environments. True 2 to 16-socket scalability up to 64 cores (96 with Intel Dunnington). Revolutionary Intel Xeon dual-core, quad-core, and six-core MP 7400 Series processors. Up to 1TB of registered DIMM memory for better workload density and up to 20 30% less power consumption than competitors fully buffered DIMM technology.* IBM Memory ProteXion with redundant bit-steering offers twice the memory resilience of the competition. 4th generation snoop filter 4 times larger than the competition s best. Page 7

8 IBM Predictive Failure Analysis, not just on hard drives and memory but, unlike competitors, also on processors, power supplies, fans, and voltage regulator modules. 40% lower memory latency than the nearest competition. NUMA architecture for optimal scale-up support and performance. More flexible memory configurations than competitors, at significantly lower costs. System x engineers work closely with Microsoft engineers to maximize overall system performance, create synergy amongst complimentary technologies, and ensure optimal compatibility. The System x3950 was in fact featured at WinHEC 2008 during several keynote sessions focused on the performance and scalability enhancements when running the next generation of Windows (Win7); a direct result of ongoing work between Microsoft and IBM. BladeCenter Key Features The IBM BladeCenter H delivers high performance, extreme reliability, and ultimate flexibility for even the most demanding IT environments. In 9U of rack space, the BladeCenter H chassis can contain up to 14 blade servers, 10 switch modules, and four power supplies to provide the necessary I/O network switching, power, cooling, and control-panel information to support the individual servers. The chassis supports up to four traditional fabrics using networking switches, storage switches, or pass-through devices. The chassis also supports up to four high-speed fabrics for support of protocols like 4x InfiniBand or 10 Gigabit Ethernet. The built-in media tray includes light path diagnostics, two front USB inputs, and an optical drive. The BladeCenter H offers the following features: Dense, space-saving 9U chassis. Up to 14 blades in a chassis. IBM Cool Blue technology. Page 8

9 Energy-efficient design. IBM Open Fabric. Powerful solutions management software. High-availability midplane. Hot-swappable, redundant management, switch, power-supply and blower modules. New high-speed switch module support. New high-speed bridge module bays. Advanced server management. IBM Remote Deployment Manager. Light path diagnostics self-diagnosis panel. Predictive Failure Analysis technology. 9.5 mm UltraSlim DVD. 3-year customer replaceable unit and onsite limited warranty. Hyper-V Installation and Configuration In this procedure, you will install the Hyper-V role on both servers. This procedure assumes that Windows Server 2008 x64 has been installed on the server(s), as well as the appropriate BIOS settings have been enabled (Virtualization Extensions, and Execute Disable bit). Pre-requisites: Windows Server 2008 x64 has been installed on the server(s). The Virtualization Extensions and Execute Disable Bit settings have been enabled in BIOS (the Execute Disable Bit is disabled by default on some systems). All drivers are the latest available drivers for the given system (per the IBM website). The MPIO driver for the SAN has been loaded (even if single path). Apply the Hyper-V RTM patches (x64) to your server(s). These are available on the Microsoft website: To Install the Hyper-V Role: 1. Use the Start button to open Server Manager. 2. In the Roles Summary area of the Server Manager main window, click Add Roles. 3. Under Select Server Roles page, select Hyper-V. 4. On the Create Virtual Networks page a network adapter can be selected now to create a Virtual Network Switch or this can be done later using the Hyper-V MMC. If there are only two NICs in the system this adapter will also be used for the public network side of the cluster. Page 9

10 5. On the Confirm Installation Selections prompt, click Install. 6. The computer must be restarted to complete the installation. Click Close to finish the wizard, and then click Yes to restart the computer. 7. After you restart the computer, log on with the same account you used to install the role. After the Resume Configuration Wizard completes the installation, click Close to finish the wizard. To Create a Virtual Network: You will need to perform this step on both physical computers if you did not create the virtual network when you installed the Hyper-V role. This virtual network provides the highly available virtual machine with access to the physical network. If there are only two physical NICs in the system being configured, this NIC will also serve as the Cluster s Public network interface. There are three types of virtual network types. External - allows guests to communicate with any system on the network. Internal - limits the guests to communicating only with parent partition. Private - limits the guests to communicating only with each other. 1. Open Hyper-V Manager. 2. Select Virtual Network Manager from the Actions menu. 3. Under Create virtual network, choose External. 4. Select Add to open the New Virtual Network page. 5. Assign a name for the new network. Make sure you use exactly the same name on both servers running Hyper-V. 6. Under Connection Type, verify External is still selected for network type, and choose the physical network adapter that will be assigned to this Virtual Network Switch. 7. Click OK. Cluster Installation and Configuration In this procedure the Microsoft Clustering component will be added to both servers. This procedure assumes Windows Server 2008 x64 has been installed on the server(s). Pre-requisites: Windows Server 2008 x64 has been installed on the server(s). All drivers are the latest available drivers for the given system (per IBM support). The MPIO driver for the SAN has been loaded. Page 10

11 Any servers that Windows Clustering will be installed on must be members of an Active Directory Domain. Installing Windows Server Failover Clustering 1. Select Start, click Administrative Tools, and then click Server Manager. 2. If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue. 3. In Server Manager, under Features Summary, select Add Features. 4. In the Add Features Wizard, click Failover Clustering, and then click Install. 5. Follow the instructions in the wizard to complete the installation of the feature and close the wizard when it has completed. 6. Repeat the process for the second server. Validate the Cluster Configuration Before you create the cluster, a full validation test of your configuration is recommended. This validation helps you confirm that the configuration of your servers, network, and storage meets a set of specific requirements for failover clusters. Pre-requisites: Failover clustering has been successfully added to the servers. Two network connections have been established between the two servers (one will be used for the private cluster network, and the other for public communication). Minimum of two LUNS created and visible to both servers (but offline on both). Cluster Quorum should have its own LUN. Each guest should have a dedicated LUN, and both servers can see the LUNs. To validate the failover cluster configuration 1. Open the Failover Cluster snap-in, select Start -> Administrative Tools-> Failover Cluster Management. 2. If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue. 3. In the center pane of the Failover Cluster Management tool under Management, select Validate a Configuration. 4. Follow the instructions in the wizard to identify the two servers, and run all tests to fully validate the cluster prior to creating the cluster. Page 11

12 5. While still on the Summary page, click View Report and read the test results. Or, to view the results of the tests after you close the wizard, open the file SystemRoot\Cluster\Reports\Validation Report date and time.htm where SystemRoot is the folder in which the operating system is installed (for example, C:\Windows). 6. As necessary, make changes to the configuration and rerun the tests. Figure 2. View of Failover Cluster Management MMC with highly available Hyper-V guests. Creating a Cluster In this procedure the actual cluster will be created. This procedure presumes that the cluster validation tool has been successfully run and passed as described above. To create a cluster, you run the Create Cluster wizard. Running the Create Cluster wizard: 1. Select Start -> Administrative Tools -> Failover Cluster Management. Page 12

13 2. If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue. 3. Select Create a cluster from the Failover Cluster Management tools center pane (under Management). 4. Follow the instructions in the wizard to specify: The servers to include in the cluster. The name of the cluster. Any IP address information that is not automatically supplied by your Dynamic Host Configuration Protocol (DHCP) settings (it is recommended that a cluster be given a static IP address). 5. Upon completion of the wizard, the Summary page appears. Select View Report to view a report of the tasks the wizard performed. Virtual Machine Installation and Configuration This procedure will step through the creation of a Virtual Machine under Hyper-V, setting this Virtual Machine to be a clustered resource, installing an operating system on the virtual machine, and validating cluster failover of the virtual machine. Pre-requisites: All targeted servers have Windows Server 2008 x64 installed with Hyper-V, and Failover Clustering roles and features added as described above. The shared storage must be online for the physical server that the virtual machine will be installed on. Virtual Machine Creation and Configuring Automatic Start Action In this step, you create a virtual machine and reconfigure the automatic action that controls the virtual machine's behavior when the Hyper-V Virtual Machine Management service starts. The New Virtual Machine Wizard will be used to create a virtual machine. Important: The virtual machine must use shared storage as the location of the virtual hard disk file in order to make the virtual machine highly available on the cluster. You must create the virtual machine on the physical computer node which owns the storage. Virtual machine creation 1. Logon to the server that owns the shared storage. 2. Select Start -> Administrative Tools, and then start Hyper-V Manager. 3. From the Action pane, click New -> Virtual Machine. 4. Select Next from the New Virtual Machine Wizard. Page 13

14 5. On the Specify Name and Location page, specify a name for the virtual machine, such as Sever1, and select Store the virtual machine in a different location, and then type the path or select Browse and navigate to the shared storage. 6. On the Memory page, set the amount of memory (in MB) required for the operating system that will run on this virtual machine. For example, specify 1024 MB to provide 1GB of virtual memory. 7. On the Networking page, connect the network adapter to the virtual network that was created earlier. 8. On the Connect Virtual Hard Disk page, click Create a virtual hard disk. If you want to change the name, type new a name for the virtual hard disk. Click Next. 9. On the Installation Options page, click Install an operating system from a boot CD/DVD-ROM. Under Media, specify the location of the media, and then click Finish. 10. It is helpful to have a previously copied ISO image of the OS install onto the system. This can then be selected and installed directly from ISO image residing on the host hard drive. Important: Do not start the virtual machine at this point. The virtual machine must be turned off so that you can configure it as highly available. Reconfigure the automatic start action for the virtual machine This step will reconfigure the automatic start action of the virtual machine to Do Nothing. This is because the management of a clustered virtual machine should be through the cluster service. Note that any virtual machines running on a clustered server should be moved offline (shutdown) or migrated to another node before shutting down the selected physical node. To reconfigure the automatic start action for the virtual machine: 1. In Hyper-V Manager, under Virtual Machines, right-click Failover Test and then click Settings. 2. In the left pane, click Automatic Start Action. 3. Under What do you want this virtual machine to do when the physical computer starts?, Select Nothing and then click Apply. Make the Virtual Machine Highly Available 1. To make the virtual machine highly available, you run the High Availability Wizard. 2. Open the failover cluster management tool by selecting Start -> Administrative Tools -> Failover Cluster Management. Page 14

15 3. If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue. 4. Right-click Services and Applications and select Configure a Service or Application. 5. The High Availability wizard opens. Select Next. 6. On the Select Service or Application page, select Virtual Machine from the list and then select Next. 7. On the Select Virtual Machine page, check the name of the virtual machine that you want to make highly available and then select Next. 8. Confirm your selection and then click Next again. 9. The wizard configures the virtual machine for high availability and provides a summary. To see the details of the configuration, click View Report. To close the wizard, select Finish. 10. Verify that the virtual machine is now highly available, by checking in either one of two places in the console tree: a) Expand Services and Applications. The virtual machine should be listed under Services and Applications. b) Expand Nodes. Select the node on which you created the virtual machine. Under Services and Applications in the Results pane (the center pane), the virtual machine should be listed. 11. To bring the virtual machine online, under Services and Applications, right-click the virtual machine and then click Bring this service or application online. This action will bring the virtual machine online and start it. Configure the Virtual Machine In this step, you install Windows Server 2008 on the virtual machine you created previously. After the operating system is installed, the integration services need to be added, which improve performance and integration with the physical computer. Note: If you are installing a different operating system, integration services may not be available. To install the operating system 1. Open Hyper-V Manager. Under Actions, click Manage virtual machine. 2. From the Virtual Machines section of the results pane, right-click the name of the virtual machine you created previously and select Connect. The Virtual Machine Connection tool will open. 3. Proceed through the installation. Page 15

16 Note When input needs to be provided to complete the process, move the mouse cursor over the image of the setup window. After the mouse pointer changes to a small dot, click anywhere in the virtual machine window. This action "captures" the mouse so that keyboard and mouse input is sent to the virtual machine. To return the mouse input to the physical computer, press Ctrl+Alt+leftarrow and then move the mouse pointer outside of the virtual machine window. To login into a virtual machine use Ctrl+Alt+End. 4. After the operating system is set up, you are ready to install the integration services. From the Action menu of Virtual Machine Connection, click Insert Integration Services Setup Disk. If Autorun does not start the installation automatically, you can start it manually. From a command prompt, type: %windir%\support\amd64\setup.exe. Test a Planned Failover To test a planned failover, you use Failover Cluster Management to move this service or application to another node. Steps to test a planned failover 1. From the console tree, select Services and Applications and then point to Failover Test. 2. Right-click the virtual machine, point to Move this service or application to another node, and click the name of the other node. Verify that the move succeeded by inspecting the details of each node. Test an unplanned failover To test an unplanned failover, you stop the Cluster Service. You can also shut down the node, and verify it moves to the other node automatically. Steps to test an unplanned failover 1. To open the failover cluster snap-in, click Start, click Administrative Tools, and then click Failover Cluster Management. 2. From the console tree, select Nodes and then right-click the node that runs the virtual machine. 3. Select More Actions and then click Stop Cluster Service. 4. Click Stop the Cluster Service to confirm the action. 5. The cluster service should stop on the selected node, and the virtual machine will be moved to the other node. Page 16

17 Figure 3. View of VM3 hosted on Blade 1 prior to system shut down. Page 17

18 Figure 4. View of VM3 failed-over to Blade 1 after Blade 2 is shut down. DS 5000 Storage Configuration For this solution, each of the two DS5000 s were configured with six 300 GB LUNs. On each storage system, three of the LUNs were for locally hosted Hyper-V guests, and 3 were dedicated to LUN replication. Figure 5 below shows each side of the solution hosting 3 active guests, and 3 replicated guests. Page 18

19 Figure 5. DS5000 LUN layout, and ERM replication configured The Emulex HBAs on the 3850M2 DR server were configured for multipathing using IBM DS Storage Manager MPIO compliant drivers. The.vhd Virtual Machine files were hosted on the LUNs mapped to the physical server. LUN Replication with Enhanced Remote Mirroring This section describes the concepts of Enhanced Remote Mirroring (ERM/Remote Mirroring) then goes into detail on how to enable and configure data replication between the DS5000 storage systems. We also explain the different mirroring methods and the steps to recover data after an outage. ERM is an enhanced design of the previous Remote Volume Mirroring (RVM) feature available on the System Storage DS4000 series. Page 19

20 Overview of Enhanced Remote Mirroring ERM provides multi-site DR redundancy by establishing two copies of a data set in two different physical locations, on two or more different storage servers, and enabling the second storage system to take over control during a site disaster. Figure 6 below shows a typical scenario which was simulated by our proof of concept. ERM is configured at the individual LUN level, and this solution uses two-way replication, enabling redundancy and failover for both sites. Figure 5. Enhanced Remote Mirror site configuration The available mirroring methods include the following: Metro Mirroring for synchronous replication. The host application does not complete a write request until the write request has been executed on both (local and remote) storage controllers, ensuring data is synchronized between both sites. This mode corresponds to the former RVM functionality. Global Copy for asynchronous replication, without write consistency. All write requests from a host are written to the primary (local) storage system and immediately reported as completed to the host system. Global Copy does not ensure that write requests performed to multiple drives on the primary site are later processed in the same order on the remote site, which means no write consistency check is performed. Write-consistency must be handled at the application level. Global Mirroring for asynchronous replication with write consistency. This is similar to Global copy, except it ensures that the write requests are carried out in the same order at the remote site, providing ordered-write consistency. The new enhanced features of ERM include: Dynamic Mode Switching, which allows an established mirror relationship to dynamically change modes at any time. Page 20

21 Secondary logical drives are accessible in read-only mode during replication. This allows the secondary LUNs to DR host mappings to be configured in advance, for faster recovery time, while prevent data corruption. The ability to suspend or Resume mirroring, with write I/O held in a Mirroring Repository Volume until replication resumes. Test mirror communication before replication begins, using the Storage Manager GUI. ERM Mirror repository logical drive This is a special logical drive in the storage system that stores mirroring information. Two repository drives (one for each controller) are automatically created when the ERM feature is activated. These drives are visible in Figure 7 below. The repository drive stores information such as remote write requests that are pending for the secondary logical drive. After confirmation of a write request, the corresponding entry in the repository drive is removed. Figure 7. Mirror Repository logical drives. Enabling and Configuring Remote Mirroring In order to use the ERM feature, the requisite Premium Feature License must be purchased for each DS5000 storage unit. This license key activates this feature on each DS5000. Page 21

22 SAN environment required To use ERM it is mandatory to attach the storage servers to a SAN fabric, using Fibre Channel switches. The ERM Fibre Channel connection must be dedicated for replication between the DS500s. The ports used for ERM cannot receive I/O from any host. It is possible to accomplish this with one switch, if both sites are within FC distance limits, or for proof of concept work. The SAN zones must separate the host ports from the storage system mirroring ports and also separate the mirroring ports between the redundant controllers. This means the mirroring ports from the A pair (primary and secondary A controllers) must be in a separate zone from the mirroring ports from the B pair (primary and secondary B controllers). See figure 8 below. IMPORTANT: After ERM is enabled, the DS5000 dedicates the last controller host port (normally port 8) to ERM use only. This is the port you must use for the ERM connection. The port is no longer available for standard host connections. The purpose of this is to isolate replication traffic from host server traffic. Figure. SAN switch zoning example, separating host and replication traffic. Page 22

23 Ethernet management required Enabling, activating, and managing ERM must be done via Ethernet management (out of band management). It cannot be controlled with in-band management over Fibre connections. Configuring storage system network connections are beyond the scope of this document. Detailed information on this and other configuration steps can be found in the DS5000 documentation, located here: ibm.com/systems/support/supportsite.wss/selectproduct?taskind=7&brandind= &fa milyind= &typeind=0&modelind=0&osind=0&psid=sr&continue.x=1 Steps to enable ERM with premium feature key 1. From the Subsystem Management window click Storage Subsystem Premium Features - Enable. 2. The b window opens. Chose the folder where you stored the numerical *.key file, then click Next. 3. The Enable Premium Feature confirmation dialog appears. Click Yes to confirm. Steps to activate ERM 1. From the primary DS5000, select the Storage Subsystem - Remote Mirroring - Activate pull-down menu option, or select Remote Mirroring - Activate from the right-mouse pop-up menu. 2. The Activate Remote Mirroring Wizard - Introduction dialog opens. In this dialog box, you choose where to allocate capacity for the mirror repository logical drives. 3. You can choose to create these on either an un-configured array or an existing array with sufficient free space. After creating or selecting the array, click Next. 4. After choosing where to create the mirror repository drives, the Activate Remote Mirroring Wizard Preview window is presented. Read the information and click Finish to continue. 5. The Activate Remote Mirroring Wizard Completed message is displayed. Click OK to close the dialog box. 6. Two mirror repository logical drives, one for each controller, are created as a resource for the controller managing the Remote Mirror. The last host port on each controller is now dedicated to Remote Mirroring communication. The dedicated host ports will not accept read/write requests from a host. Steps to create an ERM mirror relationship between storage systems After ERM is enabled and activated, you must create Remote Mirror relationships. Before you create mirrors, the logical drive must exist on both the primary and secondary storage systems. The secondary logical drive on the remote storage system must be at least the same capacity as the primary drive. Page 23

24 7. Select a logical drive in the Logical View of the designated primary storage system and then select the Logical Drive - Remote Mirroring - Create pull-down menu option or select Create Remote Mirror from the right-mouse menu. 8. This launches the Create Remote Mirror introduction window. Read the configuration warnings in the window and click Next if the stated prerequisites have been met. 9. This brings up the Create Remote Mirror Wizard - Selecting a Storage Subsystem for the Secondary Logical Drive. Select the storage system that contains the secondary logical drive you want to include in the mirrored pair, and click Next to continue. 10. Select a secondary logical drive for the Remote Mirror pair. The logical drives shown in the table are sorted by size, starting with the logical drive closest to the primary logical drive's size. Logical drives with identical size will be sorted alphabetically. 11. Select the secondary logical drive to be included in the mirrored pair. You can select only one. Click Next to continue. Note: Selected primary and secondary logical drives can have different RAID levels, segment sizes, and caching parameters. 12. The next window prompts you to set synchronization priority. Use this window to set the rate at which the owner of the primary logical drive will synchronize data with the secondary logical drive. The owner of the primary logical drive uses the synchronization priority, but a synchronization priority is also set on the secondary logical drive at this time (in case it is promoted to a primary logical drive, during a role reversal). This setting applies to both primary and secondary drives. Click Next to continue. 13. The write mode settings window appears. Select the mode according to the mirroring requirements of your environment: For Metro Mirror select Synchronous write mode. For Global Mirror select Asynchronous write mode and make sure the box labeled Add to write consistency group is checked. For Global Copy select Asynchronous write mode and make sure the box labeled Add to write consistency group is not checked. Click next to continue 14. The Preview window is the last step before the Mirror is created. You must type Yes in the box in order to click Finish and create the Remote Mirror relationship. 15. The Creation Successful message dialog is displayed. Select whether to continue and create another Remote Mirror or close. NOTE: The secondary logical drives are write protected, however they can be assigned to hosts as read only volumes. Mapping these secondary volumes to DR hosts in read-only mode after initial creation can make the DR failover process simpler and faster. Page 24

25 The time it takes for the Remote Mirror relationship to be created and fully synchronized depends upon the size of the volumes. You can view the status in the Storage Manager window. The drives show a clock icon while synchronizing. View detailed status information Select either the Logical Drive - Properties pull-down menu option, or Properties from the right-mouse pop-up menu to view the Logical Drive Properties dialog. For any drives that are part of a Remote Mirroring relationship, there will be a Mirroring tab available on this dialog box that shows mirroring-related information, including a synchronization progress bar (figure 9 below). Figure 9. ERM synchronization status window. Page 25

26 View of DR site with healthy ERM replication Figure 10 below is a view of the DR site DS5000 with ERM activated in both directions. There are 3 LUNs hosting Hyper-V guests at the DR site with those LUNs replicating to the production, primary site. There are also 3 LUNs from the production site that are being replicated to the DR site. Figure 10. View of the DR site with ERM replication running in both directions. Remote Mirror Failover and Role Reversal Once data has been mirrored from a primary storage system to a remote or secondary storage system, primary and secondary drives can have their roles reversed so that the mirrored data can be accessed from the remote storage system. To simulate failure of the production site during the proof-of-concept test, we shut down the DS5000 storage system serving as the primary site. Role reversal can be used for planned fail over, such as maintenance or tesing activities, or during an actual disaster. Page 26

27 The failover procedure is the same for a planned failover as for an actual disaster, however the failback process is different for a disaster. In a disaster, there would be a communication problem between the secondary and primary systems, which would prevent a clean demotion of the primary logical drive. This condition results in an error message, however you are allowed to proceed with the promotion of the secondary logical drive even though this will lead to a Dual Primary Remote Mirror status condition once the primary site comes back online. The Storage Manager Recovery Guru provides steps to resolve this condition. Failover procedure by reversing roles 1. Select the secondary logical drive in the Logical View, then select either the Logical Drive - Remote Mirroring - Change - Role to Primary pull-down menu option, or Change - Role to Primary from the right-mouse pop-up menu. 2. The Change to Primary dialog is displayed, select Yes to proceed. 3. The secondary logical drive is then promoted to be the primary logical drive in the Remote Mirror relationship. 4. If the controller owner of the primary logical drive can be contacted, the primary logical drive is automatically demoted to be the secondary logical drive in the Remote Mirror. 5. If the owner of the primary drive cannot be contacted, such as during a primary site disaster, the following error message appears: Figure 11. Expected error when reversing roles during primary site outage 6. Click Yes to continue. The volumes will display in Storage Manager as unsynchronized since the secondary storage can not communicate with the primary storage. Page 27

28 The view in figure 12 below shows the status of the two storage systems during the simulated disaster, with the DR DS5000 LUNs hosting all LUNs as primary and available volumes with mirroring suspended. The DR volumes and their Hyper-V virtual machines remain unaffected and online, although mirroring to production is suspended. The replicated production volumes have had their roles changed from secondary to primary, and are no longer write protected. Hyper-V is able to mount the.vhd files on these volumes and bring up the Production guests (VM 1,2,3). Figure 12. DR site with replicated production LUNs active and hosting Hyper-V guests. Page 28

29 Figure 13. Hyper-V Management console with Production guests, (VM1,2,3) running on the DR site Remote Mirror failback process The failback procedure after the production site is back online consists of breaking the mirror, recreating it, and reversing roles back to production site as primary. You must break the mirror relationship to resolve the dual-primary condition. This assumes the DS5000 was recoverable from the main site, and when it comes back up, both systems are in a primary role state. 1. Right click the logical drive that is being mirrored and select Remove Mirror Relationship. 2. Follow the earlier steps in this document to create the ERM mirror relationship again between each of the LUNs. This will initially result in the DR site having primary drives and the production site hosting secondary drives. 3. Wait for the mirror synchronization process to complete. 4. Reverse the roles on the production site so the drives become primary. Select the secondary logical drive in the Logical View, then select either the Logical Drive - Remote Mirroring - Change - Role to Primary pull-down menu option, or Change - Role to Primary from the right-mouse pop-up menu. This completes the failback process, with both environments in their original pre-disaster configuration. Page 29

30 Conclusion Microsoft s Hyper-V ushers in a new era of server virtualization for customers, offering new opportunities for increased resource utilization, ease of management, and improved ROI. Hyper-V is fully integrated with Windows Server 2008, and the management of the virtual environment is fully integrated in MMC and other familiar Microsoft management tools. IBM has worked closely with Hyper-V development team to ensure our products are optimized for Hyper- V deployments. Together with the new System Storage DS5000, the ability for companies of all sizes to implement a highly-available and disaster-tolerant computing environment has never been easier. Page 30

31 Copyright 2008 by International Business Machines Corporation. This document could include technical inaccuracies or typographical errors. IBM may make changes, improvements or alterations to the products, programs and services described in this document, including termination of such products, programs and services, at any time and without notice. Any statements regarding IBM s future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document is current as of the initial date of publication only, and IBM shall have no responsibility to update such information. Performance data for IBM and non-ibm products and services contained in this document was derived under specific operating and environmental conditions. The actual results obtained by any party implementing any such products or services will depend on a large number of factors specific to such party s operating environment and may vary significantly. IBM makes no representation that these results can be expected or obtained in any implementation of any such products or services. THE INFORMATION IN THIS DOCUMENT IS PROVIDED AS-IS WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR INFRINGEMENT. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Any reference to an IBM program or product in this document is not intended to state or imply that only that program or product may be used. Any functionally equivalent program or product, that does not infringe IBM s intellectual property rights, may be used instead. It is the user s responsibility to evaluate and verify the operation of any non-ibm product, program or service. The provision of the information contained herein is not intended to, and does not grant any right or license under any IBM patents or copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY U.S.A. IBM, the IBM logo, System x, and System Storage are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries or both. Microsoft, Windows, and Hyper-V are trademarks of Microsoft Corporation in the United States, other countries or both. i IBM Systems & Technology Group, Kirkland, WA, smithsc@us.ibm.com ii IBM Systems & Technology Group, Kirkland, WA, davwest@us.ibm.com iii IBM Systems & Technology Group, Raleigh, NC, dhartma@us.ibm.com Page 31