OPEN DATA CENTER ALLIANCE USAGE Model: Long-Distance Migration Rev. 2.0

Transcription

1 OPEN DATA CENTER ALLIANCE USAGE Model: Long-Distance Migration Rev. 2.0

2 Table of Contents Legal Notice...3 Executive Summary...4 Purpose...5 Taxonomy...6 Assumptions...7 What Does Long Distance Mean?...8 Long-Distance Migration Usage Scenarios...9 General Migration Classes...9 Representative Migration Usage Scenarios...9 Units of Long-Distance Migration...10 Network Concept for Migration Considerations...12 General Migration Workflow...13 Schema for Usage Scenarios...16 Usage Scenario Group 1 - Relocate...17 Usage Scenario Group 2 - Extend...18 Usage Scenario Group 3 - Sustain...19 Key Performance Indicators...20 Contributors Axel Knut Bethkenhagen, BMW Mustan Bharmal, T-Systems International GMBH Sudip Chahal, Intel IT Alan Clarke, SUSE Ravi A. Giri, Intel IT Bernd Henning, Fujitsu Eric Kristoff, ODCA Infra Workgroup Tobias Kunze, Red Hat Ben MP Li, Deutsche Bank Geoff Poskitt, Fujitsu Peter Pruijssers, Atos Erik Rudin, Science Logic Avi Shvartz, Bank Leumi Ryan Skipp, Deutsche Telekom Catherine Spence, Intel IT Mick Symonds, Atos Arivou Tandabany, Telstra Hans van de Koppel, Capgemini Stephanie Woolson, Lockheed Martin Service Tiers...21 RFP Requirements...23 Solution Providers...23 Service Providers...23 Summary of Industry Actions Required...24 Appendix 1: Example Use Cases...25 Use Case 1 Data Affinity...25 Use Case 2 Follow-the-Sun...25 Use Case 3 Disaster Recovery...25 Use Case 4 Follow-the-Moon...25 Use Case 5 Cloud Bursting...26 Use Case 6 Data Center Downtime/Closure...26 Appendix 2: Suggested Future Enhancements

3 Legal Notice This Long-Distance Migration document is proprietary to the Open Data Center Alliance (the Alliance ) and/or its successors and assigns. NOTICE TO USERS WHO ARE NOT OPEN DATA CENTER ALLIANCE PARTICIPANTS: Non-Alliance Participants are only granted the right to review, and make reference to or cite this document. Any such references or citations to this document must give the Alliance full attribution and must acknowledge the Alliance s copyright in this document. The proper copyright notice is as follows: Open Data Center Alliance, Inc. ALL RIGHTS RESERVED. Such users are not permitted to revise, alter, modify, make any derivatives of, or otherwise amend this document in any way without the prior express written permission of the Alliance. NOTICE TO USERS WHO ARE OPEN DATA CENTER ALLIANCE PARTICIPANTS: Use of this document by Alliance Participants is subject to the Alliance s bylaws and its other policies and procedures. NOTICE TO USERS GENERALLY: Users of this document should not reference any initial or recommended methodology, metric, requirements, criteria, or other content that may be contained in this document or in any other document distributed by the Alliance ( Initial Models ) in any way that implies the user and/or its products or services are in compliance with, or have undergone any testing or certification to demonstrate compliance with, any of these Initial Models. The contents of this document are intended for informational purposes only. Any proposals, recommendations or other content contained in this document, including, without limitation, the scope or content of any methodology, metric, requirements, or other criteria disclosed in this document (collectively, Criteria ), does not constitute an endorsement or recommendation by Alliance of such Criteria and does not mean that the Alliance will in the future develop any certification or compliance or testing programs to verify any future implementation or compliance with any of the Criteria. LEGAL DISCLAIMER: THIS DOCUMENT AND THE INFORMATION CONTAINED HEREIN IS PROVIDED ON AN AS IS BASIS. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, THE ALLIANCE (ALONG WITH THE CONTRIBUTORS TO THIS DOCUMENT) HEREBY DISCLAIM ALL REPRESENTATIONS, WARRANTIES AND/OR COVENANTS, EITHER EXPRESS OR IMPLIED, STATUTORY OR AT COMMON LAW, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, VALIDITY, AND/ OR NONINFRINGEMENT. THE INFORMATION CONTAINED IN THIS DOCUMENT IS FOR INFORMATIONAL PURPOSES ONLY AND THE ALLIANCE MAKES NO REPRESENTATIONS, WARRANTIES AND/OR COVENANTS AS TO THE RESULTS THAT MAY BE OBTAINED FROM THE USE OF, OR RELIANCE ON, ANY INFORMATION SET FORTH IN THIS DOCUMENT, OR AS TO THE ACCURACY OR RELIABILITY OF SUCH INFORMATION. EXCEPT AS OTHERWISE EXPRESSLY SET FORTH HEREIN, NOTHING CONTAINED IN THIS DOCUMENT SHALL BE DEEMED AS GRANTING YOU ANY KIND OF LICENSE IN THE DOCUMENT, OR ANY OF ITS CONTENTS, EITHER EXPRESSLY OR IMPLIEDLY, OR TO ANY INTELLECTUAL PROPERTY OWNED OR CONTROLLED BY THE ALLIANCE, INCLUDING, WITHOUT LIMITATION, ANY TRADEMARKS OF THE ALLIANCE. TRADEMARKS: OPEN CENTER DATA ALLIANCE SM, ODCA SM, and the OPEN DATA CENTER ALLIANCE logo are trade names, trademarks, and/or service marks (collectively Marks ) owned by Open Data Center Alliance, Inc. and all rights are reserved therein. Unauthorized use is strictly prohibited. This document does not grant any user of this document any rights to use any of the ODCA s Marks. All other service marks, trademarks and trade names reference herein are those of their respective owners. 3

4 OPEN DATA CENTER ALLIANCE USAGE Model: Long-Distance Migration Rev. 2.0 Executive Summary Historically, workload migration over any distance has been an immensely complex, time consuming, and disruptive activity. With the adoption of virtualization at the core of modern infrastructure services, the complexity of the underlying infrastructure may have increased, but the potential to eliminate time and disruption from migration activities is finally possible, although that potential is currently somewhat constrained by relatively short distances and low latencies. The drivers for long-distance migration capabilities are really no different from those that led to the current state of technology, which can be considered in terms of three foundational classes: migration, extension, and sustaining. This document presents practical usage models based on these classes, along with success and failure scenarios for each. Finally, a discussion of service provider requirements and an industry call to action is presented. This version of the usage model extends the prior version of this document in the following ways: The name change to Long-Distance Migration includes the whole system and not just the workload. The reference base of platform as a service (PaaS) and compute infrastructure as a service (ClaaS) now includes storage and network. Usage scenarios have been updated to include data migration activities. The focus is still on a single system migration for this iteration as opposed to a complete computing landscape. The scope has expanded to include any cloud provider, public or private. This document serves a variety of audiences. Business decision makers looking for specific solutions and enterprise IT groups involved in planning, operations, and procurement will find this document useful. Solution providers and technology vendors will benefit from its content to better understand customer needs and to tailor service and product offerings. Standards organizations will find the information helpful in defining end-user relevant and open standards. 4

5 Purpose There are several motivations for long-distance migration. Business drivers typically fall into one of several categories. 1 Business continuity. Migrate or replicate systems to establish an orderly evacuation of a data center that has experienced or anticipates experiencing failures, security breaches, or other disruptions. This encompasses both disaster recovery and disaster avoidance. Seasonal migrations in high-risk areas would also be included here. Resource locality. In some cases it is impossible, impractical, or too costly to bring data or other resources close to the compute environment. Therefore the application and services move closer to the needed resources. This move could be driven by data volume, device requirements, power costs, data residency requirements, or other legal/regulatory restrictions. Data affinity refers to when computing moves to the data location, because the volume of data is too large to move to where the compute is taking place. Follow-the-moon migrations take power and cooling costs into consideration to move the system to where costs are least expensive. Follow-the-sun. Move computing close to users based on geographic locations, time zone considerations, and other factors. This is performed on a proactive and scheduled basis. Follow-the-sun is a special variant of resource locality where the users are the immovable resource around which the system revolves. Lazy-follow-the-sun refers to a follow-the-sun model that has no hard cut-off time for users accessing the source system so therefore an overlap exists during the migration process. Dynamic scaling. Also referred to as data center expansion or cloud bursting. Dynamic scaling is the need to dynamically and elastically acquire and dispose of capacity. We include both real-time, on-demand, and proactively scheduled scaling. The benefit is that the clients (cloud subscribers) should no longer have to build for peak demand. Data center migration. Migrate to a new facility because the cloud subscriber is temporarily or permanently changing facilities. For example, this migration could be driven by a data center consolidation program or by the termination of a relationship with a given cloud provider. Note: The compute and cloud marketplaces are currently out of the scope of this usage model. 1 Travostino, F., et al., Seamless Live Migration of Virtual Machines Over the MAN/WAN. (2005). 5

6 Taxonomy Table 1 lists the standard terms and definitions used in this document. Table 1. Terms and definitions. Actor Cloud Provider Cloud Service Broker Cloud Standards Body Cloud Subscriber Data Center Operator Description An organization providing compute, storage, and network services (such as infrastructure as a service, platform as a service, or software as a service) and charging cloud subscribers based on their actual proportional usage (pay per use) of the allocated resources. A (public) cloud provider provides services over the Internet. A cloud subscriber could also be its own cloud provider. Cloud providers can be considered to be in two groups in the context of system movement. Source cloud provider - The site or data center where the cloud application or system is originally located Target cloud provider - The site or data center where the cloud application or system is to be moved or copied An entity that manages the use, performance, and delivery of cloud services and negotiates relationships between cloud providers and cloud subscribers. A cloud service broker has no cloud services of its own. An entity responsible for setting and maintaining the cloud orchestration standards considered in this usage model. These standards include the following: Global standards bodies - Typically technology orientated Country standards bodies - Related to compliance and local rules Industry sector standards bodies - Pertinent to a specific sector, such as medical Company standards bodies - Local to an enterprise, such as security Open standards bodies - Driven by open source bodies or communities with significant momentum A person or organization that has been authenticated to a cloud and maintains a business relationship with a cloud, for use of the resources provided thereby according to the defined terms of the agreement. A person or organization operating a generic IT or enterprise data center for its own services and/or as a service for third parties. Term Description Interconnectability The parallel process in which two coexisting environments can communicate and interact. Interoperability Refers to how cloud-based applications and services coexist and interact with each other throughout various data center environments. The two key aspects are interconnectability and portability. Long Distance For the purposes of this usage model, long distance is defined as greater than 20 kilometers of conductor between disparate data centers (cloud provider sites). Inter-site latency is assumed to be at least 10 milliseconds or worse. Migration - At Rest At-rest migration allows the transfer of a fully-stopped virtual machine instance or a machine image from one provider site to another site, or between disparate providers. It also may include migrating applications, services, and their contents from one site (or provider) to another. By definition, only one site may accept connections for the subject workload at a time. This is also referred to as cold migration. 2 Migration - Live Live migration approximates continuous operation of a set of processes even as they are being sent to another physical location. This may also be referred to as hot migration. 3 The user of the workload is not impacted, with no down time in service. Portability The serial process of moving a system from one cloud environment to another. Traffic Trombone The network traffic between disparate data centers caused by separated workload components during a live migration. 4 2 National Institute of Standards and Technology (NIST), NIST Cloud Computing Reference Architecture. (2001). CloudComputing/ReferenceArchitectureTaxonomy/NIST_SP_ _-_ pdf 3 Travostino, F., et al., Seamless Live Migration of Virtual Machines Over the MAN/WAN. (2005). 4 VMware vfabric and the Potential Impact on Data Centre Network Design The Network Trombone. 6

7 Figure 1 illustrates the key actors that participate in a long-distance migration. Cloud Subscriber Source Cloud Provider System Source Cloud Service Movement Distance > 20km Target Cloud Provider System Target Cloud Figure 1. The key actors participating in a long-distance migration. Assumptions The following assumptions apply to the usage scenarios contained herein: Where cloud provider and cloud subscriber are separate legal organizations, there is a properly executed service agreement similar to the Commercial Services Framework from the Open Data Center Alliance (ODCA). Cloud providers implement the ODCA Service Catalog Usage Model. 5 Compliance with ODCA VM Interoperability Usage Model, 6 including import and export of virtual machine (VM) packages per the Open Virtualization Format (OVF). For extension and sustaining foundational class migrations, we assume the applications built on cloud provider services follow the practices of the ODCA white paper Developing Cloud-Capable Applications. 7 Note: Migration class migrations do not depend on this assumption. Service level agreements (SLA), operating level agreements, and relevant controls are specified between the cloud provider and the cloud subscriber. These agreements and controls may include the following: Type of migration capabilities required At-rest migration versus live migration Availability Security root of trust Consistency of management Security and compliance Carbon measurement Geographic hosting requirements System owner, and roles and responsibilities of relevant parties Roll-back criteria Duration of roll-back availability All clocks are synchronized to Coordinated Universal Time (UTC). Cloud systems can be stopped for at-rest migration. A cloud system can be described in a manifest. The manifest is simply a list and dependency mapping of all relevant components and requirements. (A manifest implementation format is beyond the scope of this usage model). Unless specified otherwise, all migrations occur between disparate data centers of one or more cloud providers. Migrations are initiated based on the cloud subscriber s intent, either explicitly by the subscriber or automatically, based on a subscriber-defined policy. Unless specified otherwise, connectivity between source and target data centers is over the public Internet

8 What Does Long Distance Mean? A key factor in IT landscape design, and especially transition, is whether the source and target locations are close enough that they can reliably communicate synchronously, and thus be treated as one location. The determining factor is the latency (time taken) for any I/Os or messages between the location: A figure of less than 10 msec, one way, is often seen as a threshold, but it depends on the application and usage. How that latency time arises is a factor of the laws of physics (that is, the speed of light in a fiber), engineering (how switches and protocol converters work) and configuration (how they are set up, at both ends of the link). Very often, the geographical distance is used as a shorthand to encapsulate this constraint, although it is only one of the factors. The term long distance in this document is used to indicate that the environments involved cannot be treated as if they were in one virtual data center : the distance is such that the latency prevents their transparent synchronous use. Thus, special arrangements have to be made to transfer them, especially if they are to remain in use during the transfer. There is not one fixed definition of what this distance actually is: a lowest common denominator value is usually required, to be sure that the most sensitive application can still function in a heterogeneous environment. It seems to be generally accepted that the practical limit for most purposes is around a 40-kilometer (km) cable length, which equates to around a 25-km geographical distance. Note that there is not one, hard distance limitation: It depends very much on the usage and behavior of the application system(s) being used. Factors to be considered in the configuration of clouds in multiple data centers include: Ownership and configuration. If both locations are owned or managed by the same organization the user or a cloud provider the connections can be transparent and optimized; otherwise other elements such as firewalls may be required, which introduce further latency. Synchronous or asynchronous connections. Very high bandwidth links can be made available, but there are still physical limitations as to the time in which data can be transferred, the latency matters, and the capacity to transfer volumes of data. For short distances, the connection can be used synchronously, so that the center in which any one system resides is treated as irrelevant for these purposes. Over that distance, the delays may influence the design of the system landscape and asynchronous connections are required, with the limitations on use that entails (for example, the potential for lost data). Storage. The prime consideration regarding performance, continuity, and disaster recovery is typically the data storage instead of the processing capacity. The use of this distance facility also extends to the need for backup and archiving. Various configurations can be constructed, using a combination of clouds, and single and twin centers, with a possible further long-distance facility for disaster recovery purposes. The current trend for business-critical systems seems to be toward synchronously connected twin centers for high availability, with a third center at a greater distance for backup, archiving, and disaster recovery. Data can be synchronously replicated within one site, between two sites, and backed up asynchronously to a third, all without host involvement. Ideally, storage is architected as a service in its own right; this can strongly influence the use and deployment of such data centers. Information Lifecycle Management (ILM) storage tiers determine which facilities are used for which data and how many copies are held where. There are a number of options regarding the technologies used to connect the centers together, with consequent levels of feasible usage. Dark fibers leave most options open as to usage; alternatively, especially between different organizations, a higher-level protocol service may be deployed. Various connection technologies can be used to provide synchronous connection up to a distance of around a 40-km cable length; the geographical distance will be less, because the cables cannot always follow straight lines and especially because dual diverse routes are required, so that one event cannot sever all connections. Beyond that distance, different laser technologies and signal repeaters may be needed, and the feasibility really does depend on the nature of the application(s) involved and whether they are susceptible to transaction time lengthening due to the extra latency that is inevitable. To use such facilities across and between cloud providers and/or data centers, consideration also has to be given to network (IP) and storage (SAN) addressing schemes. They have to be unified if the systems are to be deployed and used transparently. 8

9 Long-Distance Migration Usage Scenarios General Migration Classes There are three generic classes from which specific migration usage scenarios can be built. These classes can help address specific business usage scenarios not yet encountered, and they provide a way to generalize implementations. These generic migration classes are as follows: Relocate. Implies the cloud application or system can be at only one place at a time. It can be thought of as moving the workload. Extend. Extends the cloud application or system into new places. It can be thought of as growing the system. Sustain. Sustains the cloud application or system. It can be thought of as preserving or keeping the system. Relocating allows a cloud application or system to move from one location to another. Pure migration implies a serial sequence: being operational at only one place at a time. It does include geographic diversity and potential fault detection, but it does not include elements of load distribution or horizontal scalability. Relocation can address disaster recovery sustainability with planning, pre-placement, and failure detection mechanisms. Extending a cloud application or system extends a function into additional geographic and/or network topological locations: The computing runs in both locations in parallel. It includes elements of geographic diversity, horizontal scalability, and a load distribution mechanism (that is, a global load balancer). Increased capacity, throughput, and performance are common goals of workload extension. Extend could also be used to gradually vacate unneeded sites as well. Sustaining a cloud application or system provides for continuity of operation during and after an event. Elements of sustainability include geographic diversity, horizontal scalability (support for multiple instances), and a load distribution mechanism (that is, a global load balancer). Increased availability is the goal. Sustaining is analogous to the Internet routing around damage and can address disaster avoidance requirements. There may be some commonality or overlap between the extending and the sustaining migration classes. Representative Migration Usage Scenarios In the context of the migration classes, there are many situations that require the movement of systems or selected components of systems among locations. Based on the migration class and the potential level of impact of the migration to the user (a possibly prolonged service interruption on the one extreme and no service interruption on the other extreme), most of the possible usage scenarios fall into groups. These groups are not exclusive and overlap may exist. The diagram in Figure 2 shows these groups. Continuous Availability of Service to Cloud Subscriber during Service Migration Migration Class Sustain Extend Relocate Cloud Subscriber Service Downtime, to enable Service Migration Data Center Closure Data Affinity Asynchronous Disaster Recovery Data Center Consolidation Usage Scenario Group 1 Follow the sun Cloud bursting Follow the moon Usage Scenario Group 2 Lazy follow the sun Lazy follow the moon Franchise create system copy Service Scaling Impact to Cloud Subscriber s Service Availability Synchronous Disaster Recovery Data Center Migration Continuous Business Availability Usage Scenario Group 3 Figure 2. Migration scenarios. 9

10 This document does not address each of the possible individual usage scenarios, but instead it provides a schema and addresses the three obvious groups of usage scenarios in the context of that schema. The movement of systems in these scenarios could be within a corporation or between external cloud services, but essentially the movement is greater than 20 km (that is, more than 20 km between the source and target locations). Each of the business drivers, explained in the Purpose section, can be addressed by at least one of the migration classes. Table 2 lists which migration class could be used to address each business driver. Where a given migration class can be applied to a business driver, a sample usage scenario is indicated. Empty cells indicate where a migration may not be applicable. Table 2. Examples of migration usage scenarios mapped to migration classes. Motivation Relocate Migration Class Extend Sustain Business Continuity Disaster recovery Disaster avoidance Resource Locality Data affinity Follow-the-moon Follow-the-Sun Follow-the-sun Lazy-follow-the-sun Dynamic Scaling Cloud bursting Data Center Migration Data center closure Specific examples for each of the situations in Table 2 are addressed in Appendix 1: Example Use Cases. There is some overlap in which more than one migration class can meet a given business driver. Depending on the specific business requirements of a given situation, there may be more than one solution. System migration is a fundamental requirement in many different kinds of business situations. It is impossible to enumerate each potential such usage scenario. The six use cases described in Appendix 1: Example Use Cases are representative of commonly observed business requirements. There are, of course, other variations. Cloud subscribers can articulate these in a common manner using the workflow steps described in the General Migration Workflow section below. All of the use cases are applicable to private, community, public, and hybrid clouds. Units of Long-Distance Migration Migration of work across clouds relies on the encapsulation of movable elements, which can be described in layers. Each layer is an independent set of items such as application processes, data, configuration information, and state. For IaaS, the abstraction includes VMs. PaaS contains higher-order abstractions of the OS and middleware layers, which can include items such as database, object storage, and message queuing. Four distinct layers of movable elements have been identified for this usage model as shown in Table 3. Table 3. A layered model for a migrating system. Layer Scope Relevant Standards 3 - Migration Protocol Procedures, processes, methodologies, and checklists, including Start/Halt/Ack/Stop/Failure/Finish 2 - Business; Non-Functional 1 - Application and System Management Appliance metadata, disaster recovery, high availability, advanced security configurations, compliance requirements, automated scaling parameters, carbon footprint requirements Software binaries, platform-as-a-service application stack, cloud broker service or appliance, monitoring, logging, auditing, licensing, support 0 - Base Runtime Virtual appliance including system configurations, additional metadata, and session-related data None Topology and Orchestration Specification for Cloud Applications (TOSCA) Cloud Infrastructure Management Interface (CIMI) Service Provision Markup Language (SPML) System for Cross Domain Identity Management (SCIM) Open Virtualization Format (OVF) 10

11 At the lowest layer, the base runtime centers on the virtual appliance. A virtual appliance is a preinstalled, preconfigured OS and software stack encompassing one or more VMs. Each VM is an independently installable runtime entity comprising an OS, applications, and other applicationspecific data, as well as a specification of the virtual hardware that is required by the VM. Metadata beyond what is contained in the virtual appliance usually includes items such as the following: Network. Load balancers (yes/no), connections to other systems inside and outside the cloud. Security. VLAN definitions, authorization and authentication data, firewall rules, and encryption keys. Compute. Whether memory overcommit is permitted and use of hardware features such as CPU attributes or flash memory. Data transfer. Direct file transfer/replication, or database copy. (Data transfer/replication is not always part of the OVF format). Note: The data transfer or copy itself usually is not part of the metadata. In addition, as part of the base runtime, active session data may also be migrated. Examples include session cookies that reside outside the compute container and HTTP cache gateways. The next layer up from the base runtime is the application and system management layer. This layer includes the software framework that may be expressed as software binaries or as a PaaS application stack. Software binaries include executable files, data and configurations that are moved via an installation script as opposed to an appliance. This approach utilizes a scripted methodology that includes selected data copy at the application layer. Another way to transfer software applications is through a cloud broker service or appliance. Cloud broker appliances are intermediary systems that incorporate an agent in both the source and target clouds. A brokering function handles the orchestration and affects the migration. Examples of cloud broker appliances include transfer systems from companies such as CA and VMware. The app and management layer includes all runtime information needed to migrate into the target cloud provider s management systems. This includes capabilities such as monitoring, auditing, logging, error handling, and help-desk functions. There is a technical integration component, which may include service name and parameters. There is also a data integration component, which provides the context for transparent migration with respect to seamless service levels (SLAs). For instance, the migration includes all data needed to ensure application error handling is handled in the same way in the target cloud with the same SLA. This may involve setting parameters when the service is selected from the service catalog in the target cloud or by explicitly setting SLAs. The business and non-functional requirements layer includes all information related to a higher order of system operation and quality of service. Appliance metadata defines data elements migration, especially managing the master data in terms of who owns the master copy of the data as it moves among clouds. These data elements must be synchronized during high availability and disaster recovery scenarios and incorporate conflict resolution strategies. In addition, it must be possible to determine the original piece of data and audit changes. Other elements in the business and non-functional requirements layer are as follows: High-availability settings for active/active, active/passive, and global load balancing Resource utilization limits for metering Definition/topology of other systems that must be running prior to starting a new session The OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) promotes the concept of a service template that specifies the topological structure and orchestration characteristics to provision services. 8 The topmost layer in the model is the migration protocol layer. It specifies the content and protocol to establish connections between the source and target cloud providers, which enables the service lifecycle. This provides a configuration to transact the following: Lifecycle stages. Start migration, suspend (halt), resume, roll back, stop (success/failure), end of migration. Data associated with enacting the lifecycle. Error conditions, timeout, success messages. 8 OASIS Topology and Orchestration Specification for Cloud Applications Version March OASIS Committee Specification

12 Network Concept for Migration Considerations There are a number of key network aspects that must be considered to enable successful long-distance migration. Enable bulk data transfer from source to target System interaction with non-migrated workloads User access Management and administration of the migrated system When considering a system to be migrated, the network is one of the most common areas in which problems may occur. In order to effectively identify the network requirements of a system for migration, it is helpful to consider the network from different viewpoints. Table 3 describes some of these important viewpoints. Table 3. Different network viewpoints. Bulk Data Transfer from Source to Target System Integration with Non-Migrated Workloads User Access Management and Administration of the Migrated System Consideration Calculate the required network bandwidth and duration for the data transfer phase of the migration (live or offline). Define the network links required back to other non-migrated systems, and the inter-system dependencies and access requirements. Define the service access schema, addressing, and load balancing. Also identify network and service access control, directory services, routing and alternate routes, and intrusion detection and prevention mechanisms. Consider encryption criteria, and traffic auditing between the source and target locations. Define the service administration responsibilities according to the selected scenario and how the responsible parties will access the systems to be administered. Explanation Determine whether there is sufficient bandwidth to deal with the data migration within the given time, considering the defined data volume (synchronously or more usually over long distances asynchronously). Map the system requirements within the business transaction chain, any latency impacts on those transactions, and key dependencies on system and data resources. Consider where a WAN outage can result in overall business transactions ceasing although both source and target systems are running normally and available at their respective locations. Identify what addressing schema will be applicable (especially if alternate cloud provider IP addresses are used), and how the DNS, DHCP, routing, load balancing, authentication, and access configurations are to be defined. Then review how auditing and controls will be implemented. This is especially important when administration responsibility is shared across sites for selected elements. Create clearly defined boundaries of responsibility, clear audit points, and consider what access, authentication, and credential controls will be applied. For example, VPNs, directory services, Internet Download Manager and Intrusion Prevention Management, multi-factor authentication, and encryption. 12

13 General Migration Workflow The migrate, extend, and sustain classes each have distinct workflow steps, while also having many steps in common. Variations on workflow implementation can be used to address niche cloud subscriber requirements not considered herein. Table 4 defines some addition terms. Table 4. Additional terminology. Term Source Target Control Definition The site or data center where the cloud application or system is originally located. The site or data center where the cloud application or system is to be moved or copied. Shorthand for which site or data center is, at that moment, the primary location for the subject system. This could be the source, the target, or a load balancing facility. A general workflow framework is illustrated in Table 5 and specific usage scenario examples follow in Figure 3. Here workflow refers to the migration process itself. One way to think of the general workflow is as a set of migration building blocks that can be assembled for a given business usage scenario. Proper implementation of the migration workflow is dependent upon good orchestration solutions, which are preferably standardized among cloud providers. Note: In this context, the word maybe means that the migration step depends on specific business requirements. Table 5. A general workflow framework. Workflow Step Number Migration Step Relocate Applicable to Migration Class Extend Sustain 1 Prepare source for migration. Yes Yes Yes 2 Marshal resources and set up environment at the target. Yes Yes Yes 3 Provision processes at the target. Yes Yes Yes 4 Provision data at the target. Yes Maybe Yes 5 Copy state to the target. Maybe No Yes 6 Confirm successful process and data provisioning. Yes Yes Yes 7 Assign control to the target. Yes No Maybe 8 Destroy or disengage the source. Yes Maybe Maybe 9 Accept new connections at the target. Yes Yes Yes 10 Balance connections between the source and the target. No Yes Maybe 13

14 Figure 3 generically illustrates the high-level workflow for a system migration over distance. It is important to note that one of the first steps to review is the business requirement that must be met select the applicable usage scenario that will drive the migration. Based on the usage scenario, it is then possible to select the relevant system elements and scope of the data or system to be migrated and the methodology to be applied, along with the potential service impacts to the service consumers. Depending on the usage scenario, the process could be iterative or one-off (for example, follow-the-sun versus disaster recovery). Also associated with the usage scenario is the service and operational responsibility, shown at the top of Figure 3. This is the point at which service and operational responsibility is either transferred or becomes shared among the sites and providers. SCP responsible for VM operation TCP responsible for VM operation VM Running State VM Stopped State VM Running State Source Cloud Provider (SCP) Running state VM inventory configuration Stopped state VM inventory configuration Create VM inventory and configuration running and stopped states Determine VM operational requirements in normal operation and for elastic / burst capacity capability requirements Change VM state to stopped and clone VM Clone of the original VM is made (for migration purposes) so if the migration process fails, the original VM is still present to continue normal operations Original VM and all data deleted Resources released, cloud subscriber billed for resources consumed, and so on Request Migrate Cloud Subscriber (CS) Start Migration Process Find TCP and access Service Catalog Match VM operational requirements to target CSP s Service Catalog No Can TCP support VMs operational requirements? Yes Reserve VM resources (tier-based) Migrate or Build New VM? Confirmation of VM deletion received New No Yes Target Cloud Provider (TCP) Create, publish, and maintain Service Catalog Service Catalog VM resource reservation request Create new VM Match current VM configuration and operational requirements Convert cloned VM to match target platform requirements Responsibility for conversion of source VM is with TCP (not SCP) Move converted cloned VM to target TCP s platform Migrate VM data to new VM Execute VM Acceptance into Service and Operation Process Deploy new VM onto TCP platform Is migrated VM operational? Criteria must be defined to prevent continuous loop back if conversion fails so VM operation returns to SCP Resource charging by TCP to CS begins End Migration Process Figure 3. The high-level workflow for a system migration over distance. 14

15 Along with workflow, and especially in the context of regular system movement as required for usage scenarios that support constant service movement (for example, follow-the-sun or follow-the-moon), data lifecycle management is integral to the workflow and planning. This is particularly relevant as new data and records are created and added to the whole moving system or parts thereof (creating a data federation) at each next service location throughout the cycle (for example, three locations in 24 hours, each stage being approximately eight-hours long). The normal data lifecycle stages for a data record are represented in Figure 4. Destroy Create Initiate Cloud Subscriber Transfer Archive Cloud Subscriber Media/Online Archive Data Lifecycle Cloud Provider Store Share Use Migrate ( * ) Cloud Subscriber Figure 4. The data lifecycle stages for a data record. Figure 5 explains in more detail the Migrate (*) stage identified in Figure 4. As multiple locations (and/or providers) come into scope the complexity increases, and the responsibility divides among more parties. The overall responsibility and ownership of data resides with the subscriber but operational responsibility lies with the various providers, and for a federated dataset, this must be well understood, mapped, and accounted for. The migrated system therefore gains more and more complexity, depending on the usage scenario. A straight migrate or sustain requirement for purposes of disaster recovery remains relatively straightforward, while a service extension, burst, or partial migration that splits system elements and associated data (especially if repeatedly, as in the case of (lazy) follow-the-moon or followthe-sun), must be carefully managed in order to retain overall system integrity, original master data, and overall system recoverability. The diagram shown in Figure 5 represents this consideration, with the data stages from the previous diagram, integrated into a migration scenario. For these migration scenarios, the associated system metadata and management/controls should carefully consider how to direct and retain control of the system and its elements. Target Cloud Provider Migrate Initiate Integrate Create Store Use Prepare Source Cloud Provider Migrate Migration Protocol Migration Protocol Source Cloud Provider Migrate Prepare Use Initiate Integrate Create Store Target Cloud Provider Migrate Figure 5. The migrate stage. 15

16 Schema for Usage Scenarios A number of concepts are important to consider when defining applicable usage scenarios against organizations business requirements. Each usage scenario has its own complexities, and each complexity has to be recognized and understood, and accommodated into the orchestration of the migration. Table 6 lists some of the most important aspects of each complexity. The column titles at the top of the table map to the impact to the user, according to the illustration in Figure 2. The row titles on the left side of the table map to important aspects to consider against each use case. The three usage scenarios described later can be considered as overlays on this table, selecting relevant characteristics per scenario from the table. Table 6. Important aspects of usage scenario complexities. Partial Service Disruption or Service Disruption Transaction Reroute No Service Disruption Data Movement Complete data migration Selected data movement Data duplication System Resource Movement According to a Layered Model Transaction Movement Global Workload Balancer Role Complete systems resource movement from source to target location Movement or duplication of selected system resources from source to target locations Complete duplication of system resources between source and target locations Complete movement to target site New transactions route to target site Transactions are routed to available sites according to a defined algorithm No need Workload allocated based on a defined algorithm License Migration License migration Add a temporary license during migration. For a short period of time, the software is licensed at both locations simultaneously. The license at the source location is released after the transfer is completed. Degree of Service Impact Degree of User Impact Service downtime during migration Accepted service outages Service available from selected locations during migration Defined access and performance limitations Service Levels Accepted service outages Defined access and performance limitations System Capacity Compliance Environmental Impact Including Carbon Footprint Sufficient capacity for the defined system Applicable conversion of compliance requirements from source to target locations Reporting of selected metrics converted from source to target; for example, carbon footprint or power usage effectiveness (PUE) Capacity for the source system at the source site and a limited capacity duplicate system at the target site Compliance with both source and target location requirements Reporting of all required metrics between source and target; for example, carbon footprint or PUE Full duplication and balancing of workload between source and target locations Global license Constant service availability and performance during migration Constant access and performance Constant access and performance Duplicate capacity spread across both source and target sites Compliance with both source and target location requirements Reporting of all required metrics between source and target; for example, carbon footprint or PUE Note: No Service Disruption does not necessarily mean that an active session (that is, the end user is using the system when a migration starts) will not be terminated during the migration. In that case, the end user may need to log on to the application in the new site. 16

17 Usage Scenario Group 1 - Relocate The example for usage scenario group 1 is motivated by a requirement to move services and operations between locations, with defined user impact, extending to defined service non-availability. This could range from business drivers, such as the cost of the source site, being significantly higher than the cost of the target site, or for purposes of data center consolidation. This situation could also apply to cases where other resources are required, such as specific devices or low-cost power. Below are the proposed high-level steps for using the relocate class to implement migration. 1. Construct a manifest of all system components and requirements at the source site. This could be done by either the cloud subscriber or cloud provider, depending upon their contract and SLA. This manifest will be used in the orchestration of the remaining steps. 2. Identify suitable target locations, according to business requirements and the system requirements identified in the previous step. 3. The cloud provider deploys all required software applications and system components to the target site. 4. The cloud provider replicates the selected system components and metadata to the target site. 5. The cloud provider executes an offline migration of the system. 6. Once the migration is confirmed to be complete and correct, traffic will be redirected to the alternate (target) site. 7. All quality-of-service (QoS), performance, security, and availability characteristics at the target site must be at parity with the source, unless otherwise agreed upon by the cloud subscriber. 8. All system resources are freed up and returned to the available pool (or decommissioned), and data is deleted at the source site, according to business requirements. Steps 3 through 7 should be treated as an atomic event. That is, they should be guaranteed either to occur completely or have no effect. Migration failure during any of those steps should cause a full rollback and continuation to operate with the in-place resources. Success scenario 1 The system is moved to the target within the acceptable defined service interruptions to system execution. Failure condition 1 The migration was interrupted or not possible. The migration did not complete successfully, and therefore the cloud application or system is unable to enter a running state at the target. A roll back to the source site and returning an error code are expected to allow for retry or failure event notice. Failure condition 2 The migrated system cannot be started and does not work as intended. The migration completed successfully. However, the cloud application or system is unable to enter a running state because of various failure conditions. A roll back to the source site and returning an error code are expected to allow for rollback and retry, or failure event notice. Failure condition 3 The migrated system does not perform as expected. The migration completed successfully, and the system entered a running state at the target site. However, behavior and/or user experience is incorrect because of various failure conditions. The cloud subscriber raises an incident to the cloud provider. Failure condition 4 The cloud provider cannot meet the usage scenario goals without introducing unacceptable risks such as network broadcast storms or network limitations, loss of data, or significant unplanned outages. The cloud subscriber or provider raises an incident to the other parties, as appropriate. 17

18 Usage Scenario Group 2 - Extend The example for usage scenario group 2 is motivated by a requirement to move selected compute operations to alternate identified locations, with limited user transaction and service impacts. This could also apply to cases where other resources are required, such as specific devices or low-cost power. Below are the proposed high-level steps for using the extend migration class. 1. Construct a manifest of all system components and requirements at the source site. This could be done by either the cloud subscriber or cloud provider, depending upon their contract and SLA. This manifest is used in the orchestration of the remaining steps. 2. Identify suitable target locations, according to business requirements and the system requirements identified in the previous step. 3. The cloud provider deploys all required software applications, metadata, and system components to the target site. 4. The cloud provider replicates storage to the target site. 5. The cloud provider executes a migration of the designated subset of system components. 6. Once the migration is confirmed complete and correct, the cloud provider adds the target location to the load-balanced group, with rules reflecting the cloud subscriber s business priorities for the source/target split. 7. All QoS, performance, security, and availability characteristics at the target must be at parity with the source, and be maintained at the source, unless otherwise agreed upon by the cloud subscriber. 8. Repeat steps 3 through 6 according to cloud subscriber requirements as often as required until defined system elements have been migrated from the source to the target (and back again if required for a follow-the-sun or follow-the-moon scenario). This could occur over an extended period of time, and master data management must be taken into account. Steps 3 through 7 should be treated as an atomic event. That is, they should be guaranteed either to occur completely or have no effect. Migration failure during any of those steps should cause a full rollback and continuation to operate with the in-place resources. Success scenario 1 The system (or defined part thereof) is successfully added to the target without disruption to the state at the source. The target is successfully added to the load-balance group. Failure condition 1 The addition was interrupted or not possible. The addition did not complete successfully, and therefore the system is unable to enter a running state at the target. A roll back to the source site and returning an error code are expected to allow for retry or failure event notice. Failure condition 2 The added system subset cannot be started and does not work as intended. The addition completed successfully. However, the system subset is unable to enter a running state because of various failure conditions. A rollback and returning an error code are expected to allow for rollback and retry or failure event notice. Failure condition 3 The added system subset does not perform as expected. The addition completed successfully, and the system subset entered a running state at the target site. However, system subset behavior and/or user experience is incorrect because of various failure conditions. The cloud subscriber raises an incident to the cloud provider. Failure condition 4 The cloud provider cannot meet the usage scenario goals without introducing unacceptable risks, such as network broadcast storms or network loops, or service and user access impacts. The cloud subscriber or provider raises an incident to the other parties, as appropriate. 18