Hyper-V Live Migration over Distance

Transcription

1 Hyper-V Live Migration over Distance Reference Architecture Guide By Hitachi Data Systems in collaboration with Microsoft, Brocade and Ciena June 2010

2 Summary Hitachi Data Systems, Microsoft, Brocade and Ciena are partnering to architect a robust business continuity and disaster recovery solution using best-in-class technologies for implementing Microsoft Hyper-V Live Migration over Distance. A comprehensive business continuity and disaster recovery plan mandates the deployment of multiple data centers located far enough apart to protect against regional power failures and disasters. Synchronous remote data replication is the appropriate solution for organizations seeking the fastest possible data recovery, minimal data loss and protection against database integrity problems. However, application performance is affected by the distance and latency between the data centers, which might restrict the location of the data centers. Deploying best-in-class storage replication technologies with the Hitachi Universal Storage Platform family, Hitachi Storage Cluster, Hyper-V Live Migration over Distance and data center interconnect products from Brocade and Ciena creates a highly available and scalable solution for business continuity where synchronous replication is a requirement. This document defines a tested reference architecture that supports Live Migration over Distance. For best results use Acrobat Reader 8.0.

3 Feedback Hitachi Data Systems welcomes your feedback. Please share your thoughts by sending an message to Be sure to include the title of this white paper in your message.

4 Table of Contents Proactive Data Center Management... 1 Solution Overview... 2 Hyper-V Live Migration over Distance Requirements... 4 Solution Components... 4 Tested Deployment... 9 Storage Configuration... 9 Storage Area Network... 9 Wide Area Network Private Fiber Network Operating System Management Software Deployment Considerations Storage Bandwidth Storage Replication Paths Storage Redundancy Storage System Processing Capacity Professional Services Lab Validated Results Conclusion Appendix A: Bill of Materials Appendix B: References Hitachi Brocade Ciena... 21

5 Hyper-V Live Migration over Distance Reference Architecture Hitachi Data Systems, Microsoft, Brocade and Ciena are partnering to architect a robust business continuity and disaster recovery solution using best-in-class technologies for implementing Microsoft Hyper-V Live Migration over Distance. A comprehensive business continuity and disaster recovery plan mandates the deployment of multiple data centers located far enough apart to protect against regional power failures and disasters. Synchronous remote data replication is the appropriate solution for organizations seeking the fastest possible data recovery, minimal data loss and protection against database integrity problems. However, application performance is affected by the distance and latency between the data centers, which might restrict the location of the data centers. The main drawback to synchronous replication is its distance limitation. Fibre Channel, the primary enterprise storage transport protocol, is limited only by its physical layer flow control mechanism. However, application response time becomes a problem as propagation delays lengthen with increased distance. Propagation delays can significantly slow servers by forcing them to wait for confirmation of each storage operation at local and remote sites. This means that the practical distance limit for synchronous replication is about 200 kilometers, or 125 miles, depending on the application response time tolerance and other factors. Deploying best-in-class storage replication technologies with the Hitachi Universal Storage Platform family, Hitachi Storage Cluster, Hyper-V Live Migration over Distance and data center interconnect products from Brocade and Ciena creates a highly available and scalable solution for business continuity where synchronous replication is a requirement. This document defines a tested reference architecture that supports Live Migration over Distance. Planning and implementation of this solution requires professional services from Hitachi Data Systems Global Solutions Services. This white paper is written for storage and data center administrators charged with disaster recovery and business continuity planning. It assumes the reader has general knowledge of Microsoft Failover Clustering, local and wide area networking and storage area networks. Proactive Data Center Management Implementing this solution enables you to proactively manage your data center, for example, in the event of a known impending disaster like a hurricane, to balance workloads among multiple data centers, or to ease consolidation of data centers. This solution provides flexibility and agility for a high-availability computing environment. It enhances the benefits derived through server virtualization: Perceived zero data center downtime for maintenance Perform virtually any data center maintenance task during normal working hours without affecting end users by simply moving the affected applications either within the data center or to a remote site. Workload balance Dynamically and non-disruptively move workloads between data centers. 1

6 Disaster avoidance Move applications in the case of an impending disaster. Data center consolidation and migration Move workloads between data centers. Ease of management Relieve storage and data center administrators of the need to learn many different tools or products. Hitachi Storage Cluster and Hyper-V live migration use standard cluster management interfaces to execute all operations such as application failover and failback between sites. Live migration is performed using the Failover Cluster GUI, Virtual Machine Manager (VMM) or Powershell scripting. Solution Overview A highly available, highly scalable Hyper-V failover cluster that supports Live Migration over Distance requires highly available, highly scalable storage, storage fabric and networks. Because this solution uses Hitachi Storage Cluster, Hitachi TrueCopy Synchronous software and Hyper-V Failover Clusters, virtual machines can be migrated between storage systems across distance with minimal intervention. This solution supports up to 16 Hyper-V host nodes in a multi-site failover cluster, with high availability achieved at the local and remote sites with redundant physical paths enabled via multiple host bus adapters (HBAs) from the servers. Proper zoning within the storage fabric and the use of multipathing software allows for continued operation in the event of a hardware component failure. Redundant high-speed network and fabric interconnects enable continued operation in the event of a hardware failure, ensuring high availability and performance across geographically separated sites. For this reference architecture, a distance of 200 kilometers was tested by using Fibre Channel spools between the DWDMs at each site. This reference architecture uses the Hitachi Universal Storage Platform VM as the storage platform, Hyper-V Failover Clustering supporting live migration of VMs, and a Brocade network and fabric architecture to provide connectivity across data centers. Note that although this reference architecture was tested on a Universal Storage Platform VM, it can be deployed on a Universal Storage Platform V as well. Figure 1 illustrates the reference architecture described by this white paper. 2

7 Figure 1. Reference Architecture In this reference architecture, each site hosted eight Hyper-V servers connected to the Universal Storage Platform VM via a Brocade DCX Backbone director at the local site and a DCX-4S director at the remote site. To support the storage replication and rapid movement of VMs between the local and remote sites, Hitachi Storage Cluster was implemented. A 1/10GbE Brocade TurboIron switch and a 10GbE NetIron XMR router at each site, along with a Ciena 4200 DWDM, provided the network infrastructure to support both storage replication traffic and Hyper-V Live Migration over Distance traffic. To support the storage replication and Live Migration over Distance traffic bandwidth and latency requirements, both Fibre Channel over IP ISL links along with native Fibre Channel inter-switch links (ISLs) were configured. ISL links are inter-switch links that connect the switches into a switched fabric. This was done for the following reasons: To provide redundancy in case of an ISL link failure with either the Fibre Channel over IP ISL links or the Fibre Channel ISL links, each ISL link was comprised of multiple physical links. In the case of failure of a physical link within an ISL, the availability of a particular ISL link is not affected. The Fibre Channel FCIP link contained three physical 1Gb links and the Fibre Channel ISL links contained two 4Gb physical links for availability and performance. To validate this reference architecture for storage replication traffic using Fibre Channel over IP. To validate this reference architecture for Fibre Channel ISL links over distance. Testing used SQL server transaction workloads along with Iometer workloads to validate this solution in terms of bandwidth and latency capabilities. This ensured that live migrations occurred in a timely fashion and with no perceivable outage to end users. 3

8 A total of 3Gb of bandwidth was configured to support the Fibre Channel FCIP links and validation of throughput and latency was performed on these links. In the lab, Hitachi Data Systems was able to move, on average, 400MB/s of write traffic across the replication links with average response times less than 20ms. Brocade s hardware compression on the Fibre Channel over IP ISLs within the DCX directors accounted for the increased throughput. A total of 8GB of bandwidth was configured to support the Fibre Channel ISL links and validation of throughput and latency was performed on these links. Testing showed that we were able to move on average 480MB/s of write traffic across the replication links with average response times less than 20ms. Additional bandwidth was still available across the Fibre Channel ISL links, but increasing the link utilization requires additional resources within the Universal Storage Platform VM storage systems, which were not available in the test environment. This solution validated that multiple parallel live migrations completed successfully across distance between nodes in the Hyper-V cluster. Because live migration is restricted to pairs of nodes in the cluster, this reference architecture s 16-node cluster design is limited to eight parallel live migrations. Hitachi Data Systems successfully conducted eight simultaneous migrations in its testing of this reference architecture. For more information about validation testing, see the Lab Validation section. Hyper-V Live Migration over Distance Requirements Hyper-V Live Migration over Distance has the following infrastructure requirements: An IP network that can support the bandwidth requirements of the virtual machines that will be migrated. This requirement can vary based on the number of modified pages that might need to be moved across the IP network for a particular virtual machine. A Fibre Channel IP or Fibre Channel network that can support the bandwidth and latency requirements for storage replication across distance. The source and destination Hyper-V host are required to have a private live migration IP network on the same subnet and broadcast domain. The IP subnet that is utilized by the virtual machines must be accessible from both the local and remote servers. When live migrating a virtual machine between the local and remote site, the virtual machine must retain its IP address so that TCP communication continues during and after the migration. The following variables affect live migration speed: The number of modified pages on the VM to be migrated; the larger the number of modified pages, the longer the VM remains in a migrating state Available network bandwidth between source and destination physical computers Hardware configuration of source and destination physical computers Load on source and destination physical hosts Available bandwidth (network or Fibre Channel) between Hyper-V physical hosts and shared storage Solution Components The following sections describe the components that make up this solution. Hitachi Universal Storage Platform VM This solution used two Hitachi Universal Storage Platform VMs as reliable, flexible, scalable and cost effective storage systems for the Live Migration over Distance architecture. The Hitachi Universal Storage Platform VM brings performance and ease of management to organizations of all sizes that are dealing with an increasing number of virtualized business-critical applications. It is ideal for a failover clustering and storage replication environment that demands high availability, scalability and ease-of-use. 4

9 This reference architecture uses Hitachi Dynamic Provisioning software to provision virtual machines. The Universal Storage Platform VM with Hitachi Dynamic Provisioning software supports both internal and external virtualized storage, simplifies storage administration and improves performance to help reduce overall power and cooling costs. Although this solution was tested on a Universal Storage Platform VM, it is also appropriate for use with Universal Storage Platform V. Hitachi TrueCopy Synchronous Software Hitachi TrueCopy Synchronous software allows you to create and maintain duplicate copies of all user data stored on a Hitachi Universal Storage Platform storage system for data duplication, backup and disaster recovery scenarios. Data is replicated from a primary Universal Storage Platform storage system to a secondary Universal Storage Platform storage system either in a local data center or across geographically dispersed data centers. With TrueCopy Synchronous software, the remote copy of the data is always identical to the local copy, which allows for fast restart and recovery of the data at the remote site. During normal TrueCopy Synchronous software operations the primary volumes remain online to all hosts and continue to process both read and write operations. In the event of a planned or unplanned outage, write access to the secondary copy of the data can be invoked to allow recovery or migration with complete data integrity. Hitachi Storage Cluster Hitachi Storage Cluster for Microsoft Hyper-V is a business continuity and disaster recovery solution for Hyper- V virtualized environments. Hitachi Storage Cluster enables the replication of virtual machines and their associated data either locally or across geographically dispersed sites. Hitachi Storage Cluster provides for automated or manual failover and failback of virtual machines and data resynchronization using either live or quick migration. Data replication and control are handled by the Hitachi Storage Cluster software and the storage system controllers. This fully automated process has little effect on the applications running in the VM guest partitions. Consistency groups and time-stamped writes ensure database integrity. This solution consists of LU replication between the two geographically dispersed sites, with automated failover of VMs resources to the secondary site in the event that the local site goes down, or a failover is initiated manually either through Virtual Machine Manager (VMM) or the Failover Cluster GUI. Through the implementation of a generic resource script in the cluster resource group for a particular VM, Hitachi Storage Cluster ensures that the replication traffic is moving in the correct direction based on the owning node of VM in the cluster. In the event of a failover HSC controls cluster access to the disk resources by ensuring that the disks are ready for access and controlling the order by which resources come online within the cluster group. Virtual machines run as cluster resources within the Hyper-V cluster. If a node within the cluster that is hosting the virtual machine fails, the virtual machine automatically fails over to an available node. The virtual machines can be quickly moved between cluster nodes to allow for planned and unplanned outages. With Hitachi Storage Cluster, the replicated LUs and the virtual machines are automatically brought online. 5

10 Figure 2 illustrates how multiple virtual machines and their associated applications can be made highly available using Live Migration over Distance with Hitachi Storage Cluster. Figure 2. Hitachi Storage Cluster for Hyper-V Microsoft Windows Server 2008 R2 Windows Server 2008 R2 adds powerful enhancements to Hyper-V including increased availability, improved management and simplified deployments. Windows Server 2008 R2 provides a new feature called live migration, which allows for the movement of virtual machines across physical hosts in the datacenter with no perceived downtime by the application and its users. In this reference architecture, live migration is used to move virtual machines across extended distance to physical hosts located in a geographically separate datacenter. Live migration and quick migration both move running virtual machines from one Hyper-V physical host to another. Quick migration saves, moves, and restores a VM, which results in some downtime to the end user. Live migration uses a different process for moving the VM to another Hyper-V host: 1. Transfer all VM memory pages from the source Hyper-V host to the destination Hyper-V host over the network. While these pages are being transferred, all modifications to the VMs memory are tracked. 2. Transfer memory pages that were modified while Step 1 was executed to the destination Hyper-V host. 3. Move storage resources to the destination Hyper-V host computer. 4. Bring the destination VM online on the destination Hyper-V host computer. 6

11 Live migration produces significantly less downtime than quick migration for the VM being migrated. This makes live migration the preferred method when users require uninterrupted access to the VM that is being migrated. Because a live migration will complete in less time that the TCP timeout for the migrating VM, users will not experience any outage even during steps 3 and 4 of the migration. Because live migration moves virtual machines over the Ethernet network, the following networking features within Microsoft Windows Server 2008 R2 enhance Live Migration over Distance: The ability to specify which physical network at the NIC level that live migration uses when moving a virtual machine s configuration and memory pages across the network. For this reference architecture, the 10GbE Brocade network hosts the live migration network traffic for both performance and throughput reasons. Support for Jumbo Frames allows for larger payloads per network packet which improves overall throughput and reduces CPU utilization for large transfers. This reference architecture uses Broadcom NICs that support Jumbo Frames. Support for VM Chimney, which allows a virtual machine to offload its network processing load onto the NIC of the physical Hyper-V host computer. This improves CPU and overall network throughput performance and is fully supported by live migration. This reference architecture uses Broadcom NICs that support VM Chimney. For more information about Hyper-V Live Migration, see the Hyper-V Live Migration Overview and Architecture white paper. Microsoft Virtual Machine Manager 2008 R2 Virtual Machine Manager 2008 R2 (VMM) is Microsoft s management solution for the virtualized data center. VMM enables the consolidation of multiple physical servers onto Hyper-V host servers, allowing them to run as guest virtual machines. VMM also provides for the rapid provisioning of virtual machines, and unified management of the virtual infrastructure through one console. This reference architecture uses Microsoft Virtual Machine Manager 2008 R2 to manage live migration. This reference architecture uses VMM to manage the virtual machines in the failover cluster and also to initiate and track live migration times. Brocade Data Center Fabric Manager (DCFM) is a comprehensive network management application that enables end-to-end management of data center fabrics. Virtualization requires the careful management of the application stack rather than hardware components to enable seamless mobility of applications and data throughout the data center fabric. Brocade fabric management software centralizes the management of large, multi-fabric or multi-site storage networks, improving visibility throughout the data center fabric and the virtual connections between servers and storage. With enterprise-class reliability and scalability and advanced features such as proactive monitoring and alert notification, Brocade management solutions help optimize storage resources in virtual environments and maximize the performance of the data center fabric. Brocade DCX Backbone and DCX-4S Directors Brocade DCX and DCX-4S Backbone directors are a new class of fabric infrastructure platform. The DCX and DCX-4S fit the requirements for extraordinary performance (bandwidth, port count, and power efficiency), nondisruptive scalability, continuous availability and end-to-end security, as required by this architecture. The DCX and DCX-4S improve infrastructure utilization, simplify provisioning, capacity planning and management and reduce infrastructure costs. The Brocade FX8-24 extension blade used in this solution provides Fibre Channel over IP connectivity between sites. Brocade Host Bus Adapters The Brocade 8Gb Host Bus Adapters (HBAs) used in this reference architecture lay the foundation for extending fabric intelligence to servers and through the network to virtual machines, applications, and services, enabling end-to-end storage network management. This approach provides tighter integration across the enterprise, including both physical and virtual infrastructure. Brocade HBAs provide robust and powerful storage connectivity for virtual servers, helping to ensure that flexibility does not come at the price of performance, reliability, or scalability. They also provide up to 8Gbit per second performance along with greater data protection for both physical and virtualized environments. The flexible Brocade architecture simplifies 7

12 management of virtual connections, providing the ability to guarantee service levels, monitor I/O history and isolate traffic per virtual machine. Brocade NetIron Routers The Brocade NetIron XMR series of routers features Brocade Direct Routing (BDR) technology for full forwarding information base (FIB) programming in hardware, together with hardware-based, wire-speed access control lists (ACLs) and policy-based routing (PBR) for robust, high performance IPv4, IPv6 and Layer 3 VPN routing. NetIron XMR routers provide the high availability, advanced failure detection and network traffic protection and restoration schemes required to support Hyper-V Live Migration over Distance. The routers provide complete hardware redundancy with resilient software featuring hitless failover and hitless software upgrades with graceful restarts for maximizing router uptime. The multi-service IronWare operating system, powering the NetIron XMR routers, offers advanced capabilities for rapid detection and bypass of link and node failures. Brocade TurboIron Switches This reference architecture uses the Brocade TurboIron 24X switch because it is a compact, high-performance, high-availability and high-density 10GbE switch that meets the mission-critical data center requirements for Live Migration over Distance. With an ultra-low-latency, cut-through, non-blocking architecture, the TurboIron 24X provides cost-effective server connectivity. The TurboIron 24X can support 1GbE servers until they are upgraded to 10GbE-capable network interface cards (NICs), simplifying migration to 10GbE server farms. The servers deployed in this architecture were equipped with 1GbE network cards. The TurboIron 24X was deployed to save rack space, power and cooling in the data center while delivering 24-hour-a-day, seven-day-a-week service through its high-availability design. The TurboIron 24X can be with internal power redundancy features, which are usually available only in a modular chassis form factor. Every TurboIron 24X has a single AC power supply, but for this reference architecture an additional AC power supply was added for redundancy. The AC power supplies are hotswappable and load-sharing with auto-sensing and auto-switching capabilities, which are critical for power redundancy and deployment flexibility. Ciena 4200 DWDM This reference architecture uses Ciena s Virtual Optical WAN architecture with the CN 4200 and FlexSelect Advanced Services Platform and associated modules to transpond or multiplex client ports over the network. This Ciena Virtual Optical WDM provides the highest bandwidth and lowest latency available today over Fibre Channel networks. This reference architecture uses the Optical Transport Network (OTN/709) to support long distance virtual machine migration and storage replication. With inherent Layer 1 protection and deterministic performance, OTN is more reliable and easier to manage than native Ethernet solutions. This solution provides the most efficient and flexible consumption of metro bandwidth for connectivity services with right-sized transport tunnels. This approach enables flexible configurations to allocate only as much capacity to each application and protocol channel as needed within a wavelength. Ciena s OTN implementation also benefits SAN fabric connectivity. Ciena can accommodate Fibre Channel 8Gb or 10Gb ISLs on individual 10Gb wavelengths. However, Ciena recommends a more efficient transport using 4Gb Fibre Channel links because three 4GbFibre Channel ISLs can be efficiently transported using one 10GB wavelength. The optical-fiber-based bandwidth solutions deployed in this reference architecture are ideal for business continuity applications by providing the ideal transport for storage networking and migration of virtual machines between data centers. 8

13 Tested Deployment The following sections describe the tested deployment of Hyper-V Live Migration over Distance in the Hitachi Data Systems laboratory. Storage Configuration Hitachi TrueCopy Synchronous software requires the use of a Hitachi Universal Storage Platform V or Universal Storage Platform VM at the local site that contains the primary volumes and a Universal Storage Platform V or Universal Storage Platform VM at the remote site for the secondary volumes. Testing conducted to develop this reference architecture used a Universal Storage Platform VM. The Universal Storage Platform VM storage system at the local site is known as the Main Control Unit (MCU) and the Universal Storage Platform VM at the remote site is known as the Remote Control Unit (RCU). Remote paths connect the two Universal Storage Platform VM storage systems over distance. The tested deployment used two 4Gb Fibre Channel remote copy connections. Table 1 lists the configuration specifications for the two Universal Storage Platform VMs deployed in this reference architecture. Table 1. Deployed Storage System Configuration Component Storage system Details Hitachi Universal Storage Platform VM Microcode level RAID group type RAID-5 (3+1) Cache memory Front-end ports TrueCopy ports Drive capacity Drive type and number 128GB 32 Fibre Channel ports 2 4GB Fibre Channel ports 300GB 24 Fibre Channel 15K RPM Number of Dynamic Provisioning pools 1 Number of drives in pool 24 Number of RAID-5 (3+1) groups in pool 6 Number of LUs in pool 88 Size of each virtual LU 50GB Number of VMs deployed 8 Number of LUs per VM 10 Storage Area Network For this solution, Hitachi Data Systems connected the Hyper-V servers and the Hitachi Universal Storage Platform VM through Brocade DCX enterprise-class directors, a DCX Backbone at the local site and a DCX-4S at the remote site. Multiple links between the directors across distance were created to form inter-switch links (ISLs). To support the storage replication traffic, two 4Gb Fibre Channel ISLs were trunked together to create a single logical trunk that provides up to 8Gb/sec throughput. This was done to evenly distribute traffic across all the ISLs, to ensure high availability and reliability if an ISL link with the trunk fails and to provide load balancing by using Dynamic Path Selection (DPS). 9

14 In addition, to provide additional throughput, availability and performance for the storage replication traffic, multiple Fibre Channel over IP links were configured between the directors across distance. Fibre Channel over IP trunking was implemented to combine multiple FCIP links into a high bandwidth FCIP trunk spanning multiple physical ports to provide load balancing and network failure resiliency. For more information, see the Wide Area Network section of this paper. This solution uses two redundant paths from each Hyper-V host to the Universal Storage Platform VM. Each Hyper-V host had dual-path Brocade HBAs configured for high availability. Microsoft s MPIO software provided a round-robin load balancing algorithm that automatically selects a path by rotating through all available paths, thus balancing the load across all available paths, optimizing IOPS and response time. Figure 3 illustrates the storage area network configuration for the stretched 16-node Hyper-V failover cluster used for this reference architecture. Figure 3. Live Migration over Distance SAN Configuration Table 2 lists the Brocade director configuration and firmware levels for the HBAs and the directors deployed in this solution. Table 2. Deployed Brocade Configuration and Firmware Levels Device Number of Fibre Channel Ports Number of GbE Ports Notes Brocade DCX Backbone 22 3 Firmware b Brocade DCX-4S 22 3 Firmware b Brocade HBA N/A Storport Miniport Driver Firmware

15 Wide Area Network Multiple networks were deployed in the reference architecture to support the high speed performance and throughput requirements of live migration. The following sections describe the architecture for extending the LAN network over distance and network architecture deployed for supporting storage replication over distance. LAN Extension over Distance The Brocade TurboIron 24X 1/10GbE switches at the local and remote sites were connected to the NetIron XMR 10 GbE layer 2/3 switches and then onto the Ciena CN 4200 DWDM. During live migration of a virtual machine, IP traffic for migrating the virtual machine flows over the 10GbE network, as shown in Figure 4. Figure 4. Live Migration Network Storage Replication over the Network For this reference architecture, both Fibre Channel over IP links and Fibre Channel links were configured to support storage replication across distance. The Brocade FC8-24 extension blade provides the Fibre Channel and IP connections for distance routing for the Fibre Channel over IP and Fibre Channel links. 11

16 Figure 5 shows the network configuration implemented for both the IP network that supports cluster management traffic and live migration traffic and the Fibre Channel over IP and Fibre Channel links that support storage replication. Figure 5. Storage Replication Network 12

17 Private Fiber Network The Ciena CN 4200 switches deployed at both the local and remote sites were configured to transport the 10GbE connections from Brocade s NetIron routers, while also transporting multiple 4GB Fibre Channel ISLs from the DCX Fibre Channel directors as shown in Figure 6. Two principle modules were configured in the CN 4200, the F10-T and the FC4-T, to support transport of the IP and Fibre Channel traffic. The Ciena 4200 DWDM network saves scarce Fibre Channel resources and avoids the cost of leased lines to support this architecture. Figure 6 shows the DWDM deployed for this reference architecture. Figure 6. DWDM Configuration The F10-T Transponder module provides transponding and regeneration of various 10Gb signals. For this solution, the 10GbE connections from the NetIron XMRs connected directly into the F10-T. The FC4-T Muxponder aggregates up to three 4Gb Fibre Channel ports across a 10Gb wavelength. The FC4- T is a Fibre Channel aggregation card that can be provisioned to handle FC400 clients. For this architecture, the Fibre Channel links from the Brocade DCXs were plugged directly into the FC4-T. Operating System Microsoft Windows Server 2008 R2 was deployed on 16 Hyper-V servers across geographically dispersed datacenters, with eight servers deployed on the local site and eight servers on the remote site. Multiple network connections were deployed to support the cluster management network and the live migration network. Table 3 lists the deployed server hardware and software. 13

18 Table 3. Deployed Servers and Operating Systems Make and Model Role Quantity Operating System Dell 2950 Hyper-V host server 16 Windows 2008 R2, 4 x Quad-Core AMD Opteron processor 1.9 GHz, 12 GB RAM. HP DL585 Domain controller and DNS 1 Windows 2008 R2, 2 x Intel Xeon 1.9G Hz, 2GB RAM. HP DL585 Management server for Hitachi Storage Navigator software and Brocade DCFM 1 Windows 2008 R2, 2 x Intel Xeon 1.9G Hz, 2GB RAM Management Software This section describes the software deployed to support the Hyper-V Live Migration over Distance architecture. Table 4 lists the software used in this reference architecture. Table 4. Deployed Management Software Software Version Microsoft Virtual Machine Manager 2008 Release 2 Hitachi Storage Navigator 7.0 Hitachi Performance Monitor 7.0 Microsoft MPIO Brocade Datacenter Fabric Manager Deployment Considerations The following sections describe key considerations for planning a deployment of this solution. Storage Bandwidth To maintain a continuous replica copy, the bandwidth available across the TrueCopy links must be greater than the average write workload that occurs during any Recovery Point Objective (RPO). This means that if an organization wants to maintain an RPO of twenty minutes, then the twenty-minute interval with the greatest write activity must be identified. With that the bandwidth required to keep up with this traffic can be calculated. Storage Replication Paths In many respects, replication traffic is processed much like any other workload on the storage subsystem. Write I/O on the primary storage array is transferred across the replication links to a secondary storage array. For Hitachi TrueCopy Synchronous software, this traffic uses SAN Fibre Channel connections between SCSI initiator ports (MCUs) at the local storage system to SCSI (RCU) ports at the remote storage array. These Fibre Channel paths have a specific bandwidth capacity and multiple connections might be required to ensure sufficient capacity. It is also important to ensure that the Brocade directors are properly sized to carry the storage replication traffic over the ISL links. Storage Redundancy It is important to identify redundancy requirements, and to ensure that multiple links are available over the network to carry replication traffic in case of failure. 14

19 Storage System Processing Capacity TrueCopy software operations incur overhead on the storage system. Additional processor cycles are required to service the additional workload for remote copy operations. On Hitachi storage systems, this additional workload occurs within the processors for the front-end director s ports. Allocate sufficient front-end director ports to accommodate the production write workload alongside additional requirements for handling remote copy operations. Professional Services Planning and implementation of this solution requires professional services from Hitachi Data Systems Global Solutions Services. Hitachi Remote Copy Planning and Design Service This required service assists you with costly bandwidth decisions and distance-data-replication challenges. Using Hitachi Data Systems remote replication best practices, consultants produce a detailed study of the current workload environment and make bandwidth recommendations necessary to support and improve the customer s remote-copy environment. This service provides customers with a high-level design for the remote-replication solution, as well as a detailed analysis of workload and performance characteristics to help support potentially expensive bandwidth decisions. This service includes the following reports: Bandwidth recommendation High-level strategy for implementing distance data replication Hardware and software audits of the host and storage environments identified for inclusion in the replication environment Workload characteristics Objectives for the replication environment Documentation of mechanisms to support the customer s replication objectives Strategic recommendations Copy-group configuration recommendations Hitachi Storage Cluster for Microsoft Environments Service This required service allows customers to improve mission-critical availability and reliability in their Microsoft Windows Server 2008 Hyper-V environments by leveraging Microsoft clustering with Hitachi replication software. This turnkey solution provides the following services: Discovery and assessment of your existing Microsoft and replication environments Implementation planning Process-based automation of Hitachi TrueCopy Remote Replication software Production implementation assistance, including Microsoft Hyper-V host configuration Testing Knowledge transfer (including hardware and software configuration, command scripts, host operations, and control mechanisms) 15

20 Lab Validated Results The goals of the joint testing were to measure what the throughput and response times were across the Fibre Channel over IP and Fibre Channel extended distance links, measure the duration of a live migration between sites and test the live migration of an application between sites while clients were accessing the application. For the last goal, testing used SQL Server with clients accessing an AdventureWorks database. The throughput and response time testing consisted of migrating live VMs under a test workload from a source server on the local site to a server on the remote site. Hitachi Data Systems initiated the test by using a load generator to direct reads and writes to the VMs running on the local host. The size of the memory allocated to the virtual machine was also modified to understand the effect on migration times as the memory size increased. After the targeted loading level was attained on the local host, live migration to the remote host was initiated manually using VMM. Hitachi Data Systems monitored the tests until live migration completed successfully. This included verifying that migration completed successfully without VM outage and that application operation was uninterrupted during the live migration. VMM started and monitored the migration during each test. In each test, one VM was migrated from the source to the destination server and was also tested migrating two VMs in parallel from the source to the destination server. For each server pair, migration was successful in both directions. Table 5 lists the test results reported by VMM. The live migration times increased as the I/O workload and the size of the memory allocated to the virtual machine increased. All response times were less than 20ms for writes. Table 5. Fibre Channel over IP Links for I/O Profile 75% Write, 25% Read (50% Random, 50% Sequential) Total IOPS Read (MB/sec) Write (MB/sec) Response Time (ms) VM Memory Size (GB) Live Migration Time :01: :02: :02: :02:47 Table 6 lists the test results. The live migration times increase as the I/O workload and the size of the memory allocated to the virtual machine increases. All response times were less than 20ms for writes. Table 6. Fibre Channel Links for I/O Profile 75% Write, 25% Read (50% Random, 50% Sequential) Total IOPS Read (MB/sec) Write (MB/sec) Response Time (ms) VM Memory Size (GB) Live Migration Time :01: :01: :02: :03:14 16