An Oracle White Paper November Oracle VM 3: Integrating Oracle VM into a Disaster Recovery Solution using SAN

Transcription

1 An Oracle White Paper November 2012 Oracle VM 3: Integrating Oracle VM into a Disaster Recovery Solution using SAN

2 Contents Introduction... 1 Part 1: Solution Architecture and Concepts... 2 Installing and Configuring Oracle VM Existing Business Continuity Solution is a Requirement... 2 Solution Flexibility... 3 Flexibility through independent Oracle VM Managers... 3 Flexibility through independent server pools... 3 Flexible active/standby solutions with a single server pool at one site4 Flexible active/active solutions with a single server pool at each site 4 Flexible active/active solutions with multiple server pools at each site5 Flexible hybrid solutions including non-dr server pools... 6 Key concepts... 7 Overview of the Active/Standby Example Used in this Paper... 7 Site Considerations... 9 Required Hardware... 9 Homogeneous Vs. heterogeneous hardware between sites... 9 Network Considerations Global Network Name Space Oracle VM Network IDs Using network metafiles to sync Oracle VM network IDs Revealing Oracle VM network IDs Virtual NICs Storage Considerations Pool file system Physical disks Storage repositories Virtual disks Data Replication Software Simple names for physical disks... 18

3 Triggering a DR Failover Develop Meaningful Conventions Oracle VM Version Considerations Product Licensing Conclusion Part 2: Integrating Oracle VM into a DR Environment Process Overview Before Beginning the Installation and Configuration Process Servers for Site B Storage for Site B Pool file system Storage repository Oracle VM Guests for validation Step 1: Install and Configure Oracle VM at Site A Step 2: Install Oracle VM Servers at Site B Step 2.1: Install Oracle VM Server on Site B servers Step 3: Replicate Site A Network IDs to Site B Step 3.1: Determine which metafiles to copy Step 3.1: Copy the metafiles to Site B Oracle VM Servers Step 4: Install Oracle VM Manager at Site B Step 4.1: Get the UUID of the Site A Oracle VM Manager Step 4.2: Install Site B Oracle VM Manager using Site A UUID Step 5: Discover Oracle VM Servers at Site B Step 6: Configure network interfaces for Site B Step 6.1: Ensure all Ethernet interfaces are available Step 6.2: Configure bonded interfaces Step 6.3: Create VNICs for Site B Step 7: Configure Oracle VM management networks Step 8: Configure Oracle VM Guest networks Step 8.1: Remove the virtual machine channel from each network. 29

4 Step 8.2: Add the virtual machine channel to each network Step 8.3: Finish adjusting any other networks Step 9: Register Storage at Site B Step 10: Create Server Pool at Site B Step 11: Validate Server Pool at Site B Step 12: Implement Storage Replication between Sites Conclusion Part 3: Executing a Failover from Site A to Site B The Process Flow Step 1: Prepare Site A for failover Step 2: Present replicated storage to Oracle VM Servers Step 3: Discover replicated storage on Oracle VM Servers Step 3.1: Rescan for disks on Oracle VM Servers Step 3.2: Add the Site A repository to the Site B OCFS2 cluster Step 3.3: Refresh the disk containing the storage repository Step 4: Present replicated storage repository Step 5: Ensure networking is correct for Oracle VM Guests Step 6: Ensure storage is correct for Oracle VM Guests Step 7: Start Oracle VM Guests on Site B servers Step 7.1: Migrate Oracle VM Guests Step 7.2: Start Oracle VM Guests Conclusion References Pertinent Oracle white papers Book references Web references... 41

5 Introduction Oracle offers a complete product portfolio that includes applications, middleware, database, operating systems, servers, storage, and even thin clients along with across the stack virtualization and clustering technologies that are optimized to work together. Oracle VM 3 is simply one piece of a very complex puzzle that must be integrated along with other disaster recovery software and applications into an overall business continuity solution for your enterprise. This technical white paper focuses on Oracle VM Server for x86, explains how to incorporate Oracle VM into an existing or planned disaster recovery solution in your data center. The solution presented in this paper is designed to use Fibre Channel (FCP) or iscsi SAN storage as the protocol being used to present physical disks to the server pools for storage repositories, applications and application data. Either SAN or NAS (NFS) can be used to integrate Oracle VM into multisite disaster recovery solution, but only SAN (FCP or iscsi) is discussed in this paper. The decision regarding which SAN storage protocol to use is completely dependent on the storage infrastructure available in your particular data center. Our goal is to provide knowledge and insights into Oracle VM that will help you design a solution that fits with the unique requirements of your software and hardware platform. This paper assumes the reader is knowledgeable and experienced with Oracle VM 3, data replication, business continuance and disaster recovery concepts. Topics such as high availability, data replication, backup and recovery strategies are beyond the scope of this document since computing environments and business continuity strategies vary so widely in approach and complexity. This paper is divided into three parts. The first part explains the architecture and design considerations for storage, network and product installation. The second part of the paper explains in detail how to prepare and install Oracle VM 3 for an active/standby solution using SAN. The third part of the document explains in detail how to execute a failover from the active site to a standby site. 1

6 Part 1: Solution Architecture and Concepts Oracle VM presents a disaster recovery solution that is flexible enough to allow easy integration with the varying degrees of complexity and technologies used in today s data center to build computing environments for business systems. There is no single approach or method for designing a mission critical disaster recovery solution presented in this paper since every data center environment is going to be slightly different: different servers, different backup solutions, different storage vendors, unique networking infrastructures, different application stacks, etc. Oracle VM provides the tools needed to adapt the product to your particular environment. The solution presented in this white paper includes some step-by-step instructions that must be followed in order to integrate Oracle VM into an overall disaster recovery project under any circumstance, but the examples and screen shots are simple in nature and are not representative of a robust network or storage solution. This document is meant as a guideline rather than rigid instructions for designing and implementing your mission critical Oracle VM deployment: you must take the information presented here and adapt it to fit your unique requirements. Installing and Configuring Oracle VM 3 Setting up and configuring Oracle VM 3 server pools is beyond the scope or purpose of this document. Please use the Getting Started guide that is an integral part of the Oracle VM Manager interface to learn how to configure an Oracle VM server pool with Oracle VM Guests. You should not attempt to integrate Oracle VM into a disaster recovery solution if not already very familiar with installing, configuring and managing Oracle VM. Important user guides for getting started with Oracle VM can be found on Oracle Technology Network for Oracle VM documentation. Existing Business Continuity Solution is a Requirement Oracle VM is simply a small piece of a complete business continuity structure. All other aspects of a disaster recovery solution are assumed to be in place before integrating Oracle VM into the overall architecture. At the very least the following components of a complete business continuity solution should be in place: A proven method for backing up and recovering Oracle VM Servers A proven method for backing up and recovering Oracle VM Manager & storage repositories A proven method for backing up and recovering applications and data within the Oracle VM environment A proven method for replicating Oracle VM storage repositories, applications and data between sites A proven fault tolerant, highly available computing environment needs to be in place. The goal is to eliminate single points of failure for power, networking, servers, storage and applications at each site. A plan for continuous maintenance and site synchronization for the overall solution A plan for periodic auditing and validation of the overall solution 2

7 Solution Flexibility Oracle VM 3 is flexible enough to be integrated into almost any DR solution envisioned by Oracle customers. As indicated in the introduction, Oracle VM is only a small piece of a complete business continuity solution. Each hardware and software vendor will have solutions, user guides and technical white papers explaining how integrate their particular products into an overall disaster recovery plan. Figure 1 below illustrates some of the basic building blocks for a complete DR plan. DR training, validation & audit Oracle VM DR solution A Complete Business Continuity Solution Service level agreements Network DR solution Application DR solution Storage DR solution Backup DR solution Figure 1: Oracle VM is just one of many pieces needed to build a complete DR solution Oracle VM is flexible enough that it can be adapted to a wide variety of DR solutions. We provide an example throughout this document using a simple two site, active/standby solution with a server pool at each site, but there are many possible solutions that go far beyond our example. Using the information provided in this paper, the same concepts for the active/standby solution can easily be adapted to create any number of active/active or active/standby scenarios imagination is the only limitation to potential solutions. Flexibility through independent Oracle VM Managers No matter which type of solution you choose, whether it is a variation of the active/standby presented in this paper or active/active coverage, the key to integrating Oracle VM 3 into a multisite disaster recovery plan is ensuring each site has a completely independent instance of Oracle VM Manager. Maintaining independent Oracle VM Managers is a key point since it means everything about the two sites can be completely different; different number of server pools, different number of servers, different server models, different network infrastructure, different storage infrastructure, etc. Independent Oracle VM Managers allow maximum flexibility in the way each site is configured. Having separate Oracle VM Managers means that the database maintained by Oracle VM Manager at Site A is never replicated to Site B or vise versa. Flexibility through independent server pools Since the Oracle VM Managers at each site are independent of each other, it follows that the pool file systems for each server pool are also independent. Pool file systems are not replicated for any of the 3

8 DR solutions discussed in this paper since the data contained in a pool file system are relevant only to the servers and storage belonging to a particular server pool. Not replicating server pool file systems gives you the flexibility to make each server pool at each site completely different. This opens many possibilities for more cost effective hardware solutions between sites and gives you the flexibility to design each site any way that fits your requirements. Flexible active/standby solutions with a single server pool at one site We briefly touch on a few different conceptual DR solutions in this section of the paper to provide a glimpse of the many possible ways the Oracle VM Manager can be used at each site. The active/standby solution shown in Figure 2 below is one of many possible solutions, but it is the only scenario we explain in detail and use as an example throughout this paper. Figure 2: Overview of an active/standby disaster recovery scenario Figure 2 shows a primary (active) server pool at Site A and a recovery (standby) server pool at Site B. The Site B server pool has running Oracle VM Servers, but no running Oracle VM Guests the servers are standing by waiting for a site failover before any Oracle VM Guests from Site A are imported and started at Site B. You need to devise a periodic disk/data replication process to replicate only the Site A storage repositories and physical SAN disks containing applications and application data from Site A to Site B. The replicated storage at Site B is not made available to the Site B server pool until after a failure of Site A. Flexible active/active solutions with a single server pool at each site Although we do not specifically explain an active/active multi-site solution in this document, one can be achieved by simply applying the same process shown in this white paper in reverse. Figure 3 describes an active/active solution with a single server pool, yet four server pools can be seen which may be a little confusing. To be clear, this is because each site has a single active server pool with a corresponding standby server pool at each of the sites. In this scenario, Site B acts as a recovery site for Site A and Site A acts as a recovery site for Site B. Note that the only difference between the 4

9 active/standby and active/active illustrations in Figure 2 and Figure 3 below is the additional arrow showing the storage replication from Site A to Site B as well as from Site B to Site A. Figure 3: Overview of an active/active disaster recovery scenario In this scenario, both Site A and B have primary server pools with actively running Oracle VM Guests. Each site also has a matching standby server pool waiting to take over running guests from its sister site. For example, Figure 3 shows that the primary Server Pool 1 is active at Site A with a standby Server Pool 1 at Site B. It also shows a primary Server Pool 2 is active as Site B with a corresponding standby Server Pool 2 at Site A. This solution also requires you to devise a periodic disk/data replication process to replicate the Site A storage repositories and physical SAN disks containing applications and application data from Site A to Site B. But you also need to devise a periodic disk/data replication process to replicate the storage repositories and data disks from Site B to Site A. The replicated storage from either site is not made available to the matching standby server pool until after a failure has occurred at one site or the other. Flexible active/active solutions with multiple server pools at each site Oracle VM is flexible enough to implement an active/active disaster recovery solution with multiple server pools at each site as illustrated in Figure 4 below. You can also choose a subset of server pools from either site. For example, perhaps only two server pools at Site A are replicated to Site B and only one server pool from Site B is replicated to Site A. Imagination is the only real limitation to the possible combinations of solutions when integrating Oracle VM into a disaster recovery solution. 5

10 Figure 4: Overview of an active/active scenario with multiple server pools Flexible hybrid solutions including non-dr server pools In the final example shown in Figure 5, we illustrate an active/active scenario that incorporates server pools that are part of a DR solution and two server pools that are not part of a DR solution at all. The illustration is identical to the one shown in Figure 4, except in this case we show Server Pool 4 and Server Pool 5 are not part of the DR solution. The Oracle VM Manager at each site can manage active server pools that are part of a DR solution as well as server pools that are not part of a DR solution Figure 5: Overview of an active/active scenario with multiple server pools including some that are not part of a DR solution 6

11 Key concepts Now that we ve introduced a few different architectural possibilities for you to consider, it is time to introduce some key concepts that will require much deeper understanding to successfully integrate Oracle VM into a disaster recovery solution. Some of the listed concepts have already been mentioned briefly while others have not been discussed yet. In either case, each of the following concepts will be explained in much more detail and should become clearer as you continue to progress through the paper. Independent Oracle VM Managers. All Oracle VM Managers should be completely independent of each other. Therefore the databases for each manager are not replicated. Oracle VM Manager UUID. The Site B Oracle VM Manager must be installed using the same UUID as the Site A Oracle VM Manager. This allows the Site B Manager to use the replicated storage repository that is owned by the Site A Manager. Independent server pools. All server pools should be completely independent of each other. Therefore the pool file systems for each site are not replicated. Storage replication. Only the storage repositories and disks containing applications and application data are replicated between sites. Disk World Wide IDs. World Wide IDs for disks must be the same at both sites. Not to be confused with Fibre Channel World Wide Node or Port Names, the physical disks at Site B must have the same exact WWID as they do on Site A so the Site A guests can find the same disks when they start on Site B. Global network name space. The solution assumes that both Site A and Site B have exactly the same network infrastructures for the virtual machine network channels. Virtual machine network IDs. Virtual machine network IDs must be exactly the same at both sites. Overview of the Active/Standby Example Used in this Paper We now introduce you to the fictional active/standby example used throughout the remainder of this document which we use to help illustrate concepts and processes in the remainder of this white paper. Figure 6 below shows a graphic depiction of our example scenario. Let s use the diagram shown in Figure 6 help explain a few of the key concepts from the previous subsection. Independent Oracle VM Managers. The diagram is divided into a primary site (Site A) and a standby site (Site B). Each site has a completely independent Oracle VM Manager with a completely independent database. You will want to create a daily backup of each database, but do not include either of the databases into the DR data/disk replication scheme. This will be explained in more detail as you progress through the remainder of this document. Oracle VM Manager UUID. Notice that the top of the diagram indicates that the Managers at both are installed using the same UUID. As indicated in the previous subsection, the duplicate UUIDs allow the Site B Manager to use the replicated storage repository that is owned by the Site A 7

12 Manager. We explain how to install the Site B Manager so it has the same UUID as the Oracle VM Manager at Site A in part 2 of this document. Figure 6: Example scenario used in this white paper Independent server pools. The diagram also shows a server pool at the primary site called Site A mypool1. This server pool contains two Oracle VM Servers named myserver1 and myserver2; the servers are actively running Oracle VM Guests. The standby site has a server pool called Site B mypool1. This server pool contains two completely different Oracle VM Servers called myserver3 and myserver4. Unlike the Site A server pool, Site B mypool1 is not running any Oracle VM Guests at all. As described previously, all server pools should be completely independent of each other. Therefore the pool file systems for each site are not part of any data/disk replication scheme. This concept is further explained in Storage Considerations in Part 1 of this document. Storage replication. It s recommended to use storage replication solutions from your storage vendor or Oracle products such as Data Guard. Both of these solutions are beyond the scope of this document. Notice the lower part of the diagram in Figure 6 shows a representation of a storage array and the LUNs/disks contained on each storage array that need to be part of the data/disk replication scheme. One array is located at the primary Site A and another storage array is located at the standby Site B; the arrays in our example are physically independent of each other. The storage arrays depicted in the diagram can be Oracle ZFS appliances or a vendor platform of your choice. 8

13 The green arrows indicate the disks/luns that should be part of a data/disk replication scheme for your DR solution. Only the storage repositories and disks containing applications and application data are replicated between sites. The pool file system should not be replicated to the standby site. This concept is further explained in Storage Considerations in Part 1 of this document. Disk World Wide IDs. World Wide IDs for disks must be the same at both sites. This is not specifically shown in the diagram but is further explained in Storage Considerations in Part 1 of this document. Global network name space. The yellow box at the bottom of the diagram is a very important concept, but is only applicable to Oracle VM networks that have been assigned a Virtual Machine channel role; the solution assumes that both Site A and Site B have exactly the same network infrastructures for the virtual machine network channels. This is also explained in much greater detail in Network Considerations. Virtual machine network IDs. Virtual machine network IDs must be exactly the same at both sites. This concept is closely related to the requirement for a global network name space. We devote quite a lot of attention to explaining both the global network namespace concept and network IDs in Network Considerations in Part 1 of this document. Site Considerations We begin our more in-depth explanation of key concepts with this section by discussing a few critical concepts that need to be understood about the disaster recovery sites, assuming a primary site (Site A) and a recovery site (Site B). The ability to extend the DR integration to more than two sites is entirely possible, but a two site solution is a very common solution so the discussion in this document is limited to a two sites. Required Hardware The solution presented in this white paper does not depend on any particular hardware solution. Oracle customers can use any supported hardware platform and storage vendor that fulfills the requirements for their unique business system. Homogeneous Vs. heterogeneous hardware between sites For best performance and higher availability, Oracle recommends that the hardware platform be identical at both sites for an Active/Standby or Active/Active business continuity model. This ensures that the disaster recovery site will be sized correctly to avoid unexpected results from Oracle VM Guests not being able to start at Site B due to fewer compute resources being available such as available memory, CPUs, differing network capacities, etc. Figure 7 below shows a homogeneous solution where the server models, memory, CPUs, storage, storage capacity are identical at both sites. The advantage is that the recovery site can be counted on to handle whatever the primary site is managing in the event of a failover. The disadvantage is that it is harder to maintain consistency between sites over time. 9

14 Figure 7: Homogeneous solution with storage and servers being the same at each site On the other hand, Figure 8 below shows a more heterogeneous solution where one site can have more or less hardware resources with completely different server models, different memory, CPU, network and storage capacity. The advantage to this paradigm is that data centers can utilize existing equipment or find less costly solutions for either the primary or recovery site. In fact, one of the primary benefits of virtualized guest operating system is hardware independence. Basically, each site can be built using servers and storage from entirely different manufacturers. The obvious disadvantage to using a heterogeneous solution is that there is a distinct possibility that the recovery site may not have the resources of the primary site and some Oracle VM Guests fail to start on the recovery site due to computing resources being wholly utilized by the first few Oracle VM Guests that are able to start after a failover. In addition, particular attention must be paid to the storage replication software when using different storage vendors at each site. Data replication requirements are examined in more detail in the Storage Considerations section of this white paper. 10

15 Figure 8: Heterogeneous solution with storage and servers being different at each site Oracle VM 3 allows quite a bit of latitude when it comes to architecting complex solutions that fit the requirements and capabilities of global or independent data center operations. Network Considerations Networking and storage are the most important pieces of the integration process as well as the most challenging to understand. So, this white paper devotes a little more space explaining networking and storage concepts in the hope that some of the tasks in Part 2 make a little more sense. Global Network Name Space The single most problematic issue in any disaster recovery solution is the network configuration for the Oracle VM guest operating systems as well as the applications and databases that run within the guest operating systems. Oracle VM can certainly be configured to start all the Oracle VM Guests at the recovery site, but this is completely useless if the guest operating systems and applications cannot be reached when everything is up and running at the recovery site due to fact that everything is configured for the Site A broadcast domain if the Site B broadcast domain is completely different. Therefore, the solution presented in this paper assumes a global network name space across both data centers as a requirement. The same exact IP addresses, subnet, gateways and host names for the guest operating systems need to be available in both the primary and recovery sites. Such global software designed networking solutions are becoming more and more prevalent since they greatly simplify the! Critical Concept! Understanding constraints imposed by Oracle VM network IDs is critical to the success of the integration effort do not skip this section to save time. 11

16 deployment of applications on a worldwide scale for organizations using a unification or coordination type operational model. If the overall disaster recovery solution you have designed includes software products that obviate the need for a global network name space, then each site can have completely different broadcast domains. However, you will need to define and build a solution to change the network configuration for each guest operating system since the replicated guests will still have a network configuration relevant only to Site A; designing and automating such a solution is beyond the scope of this white paper. Oracle VM Network IDs One very important aspect of networking that needs to be accomplished for any DR solution with Oracle VM is ensuring that the Oracle VM network IDs for any networks with the Virtual Machine Channel assigned are exactly the same at both Site A and Site B. The method for making the network IDs match at both sites is discussed in detail in Part 2 of this paper. This section explains why the network IDs need to match. The user friendly simple names are inconsequential and can be quite different at both sites without any impact what-so-ever (note the different network names in boxes 1 & 4 in Figure 9 below). However, there is a direct relationship between the network ID shown in Oracle VM Manager and a Xen Bridge on the Oracle VM Servers. This is very important. Xen bridges are only created and associated with network devices on the Oracle VM Servers for Oracle VM networks with the Virtual Machine Channel selected in the Oracle VM Manager. It is imperative that the IP range, netmask, IP assignment and Oracle VM network ID be the same at both Site A and Site B for these virtual machine networks. Figure 9: Network IDs must be the same at both sites for virtual machine networks Using Figure 9 as an example, any Oracle VM networks enclosed in boxes 2 or 5 will have a corresponding Xen bridge on each Oracle VM Server while those Oracle VM networks enclosed in boxes 3 and 6 will not have a corresponding Xen bridge since bridges are only needed for network traffic between the network devices on Oracle VM Servers and Oracle VM Guests. However, the network IDs for the virtual machine networks must be exactly the same at both sites as shown in boxes 2 and 5. If the IDs are not the same, then the Site A Oracle VM Guests will not know which Oracle VM networks to use when they are running on Site B after a failover. 12

17 All other networks related to Oracle VM servers for the management of server pools can be completely different at both sites since each site has its own server pool with completely different servers and storage. Using network metafiles to sync Oracle VM network IDs The exact steps needed to make the network ID match at each site are articulated in Part 2 of this paper. However, the simple concept is to copy the Oracle VM network device metafiles from one of the Site A Oracle VM Servers to each of the Site B Oracle VM Servers before they are discovered by the Site B Oracle VM Manager. The diagram shown in Figure 10 on the next page illustrates the relationship between the network ID, the Xen bridges and the metafiles that define the network ID. The commands shown in Figure 10 are only examples to help illustrate general concepts and are not actually used at this point. You will need to understand the basis of this relationship to determine which network metafiles to copy from Site A to Site B during the initial installation and configuration of Site B the actual steps are written in Part 2 of this document. The metafiles reside in /etc/sysconfig/network-scripts on the Oracle VM Servers. The files contain the network ID for each Oracle VM network device which will be used during the server discovery process at Site B to create the matching Xen bridges using the same network IDs. These files must not be changed or edited by hand. The example shown on the next page assumes that VLAN segments are being used for your solution. You will need to adjust if your particular solution does not use VLAN segments, or the VLAN segments are assigned to some other bonds or ports the point is that you will need to determine the appropriate metafiles that fit your unique environment. Figure 10 is comprised of four different diagrams showing how to logically connect the network ID from the Oracle VM Manager to the appropriate metafile. The top diagram shows a logical representation of the network devices that exist for the example screen shots network traffic flows from the left of the diagram through a series of logical network devices to Oracle VM guests on the right side. This diagram shows the Xen bridges (1) that are created when the virtual machine channel is assigned to a network in Oracle VM Manager. The second diagram is a screen shot of the Oracle VM Manager networking tab showing the network ID associated with each virtual machine enabled network. There is a direct relationship between the network ID (2) in the Oracle VM Manager UI and the Xen bridge for guest traffic shown in (1) on the Oracle VM Servers. The third diagram in Figure 10 shows output from the brctl command. The brctl command displays the relationships between the Xen bridges and the logical network interface that connects the bridge to the rest of the world. There is a direct relationship between the network ID (2) and the name of each Xen bridge (3). The MAC address (4) is related to the bond or port that network traffic flows to and from the guests, while (5) shows which VLAN segment the Xen bridge is related. The forth diagram shows the final piece of the puzzle, which is the relationship between the name of the VLAN segment (5) and the actual metafile (6) that will need to be copied. 13

18 1 2 3 [root@myserver1 ~]# brctl show bridge name bridge id STP enabled interfaces 0004fb0010bc e87d40 no bond fb e87d40 no bond fb001052e3b e87d40 no bond1.102 [root@myserver1 ~]# 4 5 [root@myserver1 ~]# ls /etc/sysconfig/network-scripts/meta-bond1.* /etc/sysconfig/network-scripts/meta-bond1.100 /etc/sysconfig/network-scripts/meta-bond1.101 /etc/sysconfig/network-scripts/meta-bond1.102 [root@myserver1 ~]# 6 Figure 10: Diagram showing the relationship between the network ID in the Oracle VM Manager and the Xen bridges on the Oracle VM Servers 14

19 Revealing Oracle VM network IDs The network IDs are not shown by default in Oracle VM Manager, so they need to be revealed by selecting the View pull down menu from the toolbar and then choose ID from the menu. The only reason no real need during the integration process Figure 11: Screen shot showing the default view for the Networking tab with the ID column hidden Figure 12: Screen shot showing how to reveal the hidden ID column Figure 13: Screen shot showing Networking tab with the ID column revealed 15

20 Virtual NICs Do not use the same range of virtual MAC addresses on the Site B Oracle VM Manager. The virtual MAC addresses from Site A will be imported along with the Oracle VM Guests when the Site A repository is imported into the Site B server pool. Storage Considerations The solutions presented in this paper are centered on replicating and using the storage repositories as well as any physicals disks presented to Oracle VM Guests containing applications and application data associated with each Oracle VM Guest. It is important that you understand what does and does not need to be replicated to the recovery site (Site B). Pool file system Each server pool has a single file system that contains metadata about the server pool itself. Pool file systems should not be! Critical Concept! replicated to the recovery site since at the end of the integration Do not replicate the pool file system. process the recovery site (Site B) should have a server pool ready to manage and run the Oracle VM Guests from the primary site (Site A) during a failover. Pool file systems contain information that is specific to the particular Oracle VM Servers, storage arrays and other hardware at the primary site the servers and storage at the recovery site will be completely different so copying the pool file system is unnecessary and will make the server pool at the recovery site unusable. Physical disks Only physical disks presented to the Oracle VM Guests should be replicated to the recovery site. Physical disks for applications and data are not part of the storage repositories, so provisions for replicating these separately must be made; this must also be part of your overall disaster recovery solution. Ensure all physical disks containing the following types of data are replicated to the recovery site: Physical disks containing storage repositories for Oracle VM Guests, Oracle VM Templates, assemblies, clones of guests that used by the server pool being replicated to the recovery site Physical disks containing applications being used by the pool being replicated to the recovery site Physical disks containing application data being used by the applications being replicated to the recovery site Storage repositories Stop! It is very important each site maintain its own set of unique virtual MAC addresses. Storage repositories are assumed in this document to be physical disks presented to Oracle VM Servers using one of the SAN protocols such as FCP or iscsi. The storage repositories contain the configuration files, system boot images and any associated virtual disks for each Oracle VM Guest. The storage repositories are replicated from the primary site (Site A) to the recovery site (Site B), 16

21 imported into the Site B Oracle VM Manager, and then presented to the Site B failover server pool where the Site A Oracle VM Guests will be managed. It is important to note that the import process for all physical disks including storage repositories should only occur when a failover is triggered. The import process as described in this document is currently a manual process but can be automated using the Oracle VM CLI. Oracle suggests contracting custom automation solutions through Oracle Advanced Customer Support Services or one of our excellent systems integrators from the Oracle Partner Network. The storage repositories must be replicated from Site A to Site B on a periodic basis defined by the recovery point objective that is most acceptable for a given business computing environment. This decision should be made by the team responsible for the overall business continuity solution at your site. Virtual disks Virtual disks such as the system.img system disk for each Oracle VM Guest are simply files that reside in each storage repository. Virtual files are replicated by default when the storage repositories are replicated to the recovery site. Data Replication Software The actual replication of storage containing applications, application data and Oracle VM repositories between sites is completely outside the purview of Oracle VM 3 or this technical white paper. Your storage vendor should have data replication products as well as technical white paper and user guides that explain how to develop and implement data replication between sites for disaster recovery. The data replication solution you develop is the most critical piece of the overall a disaster recovery solution and the backbone of the process explained in this document. Oracle VM is storage vendor neutral, but it is extremely critical that the replication technology incorporated into your overall business continuity solution copy 100% of each physical disk (LUN) blockfor-block to ensure the metadata containing the WWID of each disk is exactly the same at both sites. This is critical since the physical disks containing applications and application data are presented to the Oracle VM Guests using a unique UUID and the guest operating systems will not be able to see the physical disks if the WWID is not the same on Site B as they were on Site A.! Critical Concept! It is extremely important that the software used to replicate the physical disks across data centers copy 100% of the disk metadata blockfor-block to ensure the WWID for each disk is exactly the same for both sites (VPD page83). Each disk (LUN) has two numbers associated with it: a UUID and WWID. Oracle VM creates devices special files using the WWID (worldwide ID); the device special file names can be seen in the Oracle VM Manager whenever you select and expand the view for a physical disk under the Storage tab. The device names are usually something like /dev/mapper/360a a6d56304b336b as the screen shot illustrates in Figure 14 below. 17

22 Figure 14: Showing the WWID and corresponding device special file for a physical disk Unlike the UUID for a disk, the WWID is exactly the same on any server it is presented even if the disk is presented to different servers as a block device in two different data centers in different parts of the world; the WWID never changes. This makes it very easy to identify the same disk presented to multiple servers. This is why it is so important that the WWID be copied along with the replicated disk. Simple names for physical disks The simple names (user friendly names) for physical disks are maintained in the Oracle VM database. Since the database is not being replicated between sites, the simple names are not persistent. These can be manually updated as needed or the Oracle VM Command Line Interface (CLI) can be used to automate scanning for simple names in the daily XML dump file and then adding the names to the Oracle VM database at the recovery site. Triggering a DR Failover Triggering a disaster recovery failover using Oracle VM should be a manual process. This ensures the operations team has verified that a failure of the primary site has actually occurred. The manual process for triggering and performing a site failover is articulated in Part 3 of this document. Develop Meaningful Conventions Please note that the object names, simple names (user friendly names), locations, configuration examples and screen shots shown in this document are simply used to convey concepts and are not meant to be taken literally. Use IP addresses and naming conventions that are appropriate for your environment. 18

23 Oracle VM Version Considerations This document is written specifically for Oracle VM Please do not use the document with earlier or later versions of Oracle VM. Product Licensing Please check with your account representative or Oracle product licensing before making decisions about licensing products for multisite disaster recovery solutions. Conclusion At this point you should have a very good grasp of the concepts surrounding the integration process needed to incorporate Oracle VM into a new or existing business continuity architecture. One of the primary concepts is all Oracle VM Managers at various sites should be completely independent of each other. It is also very important that the network name space be exactly the same for Oracle VM virtual machine networks, including the Oracle VM network ID associated with each virtual machine network. Part 2 of this document explains how to prepare and install Oracle VM Manager at the recovery site. It is assumed by this paper that the Site B Oracle VM Manager has not, and will not be installed until the primary site is completely validated and stable. Part 3 of this document explains how to perform a site failover. The site failover should be performed once the integration process has been completed in order to validate the viability of the solution as well as familiarize the operations staff with the process. As with any good disaster recovery solution, periodic site failovers should be scheduled to ensure the process still works over time. 19

24 Part 2: Integrating Oracle VM into a DR Environment Process Overview Oracle VM is simply a small piece of an overall business continuity solution. Part 2 of this document explains the steps needed to prepare, install and integrate Oracle VM into your multisite disaster recovery solution. All other aspects of a disaster recovery solution are assumed to be in place before integrating Oracle VM into the overall solution. The diagram shown in Figure 15 provides a brief look at all the steps involved with integrating Oracle VM into a multisite disaster recovery solution. Figure 15: Process flow for integrating Oracle VM into a disaster recovery plan Note that all of the steps in Figure 15 are accompanied by colored text used to denote the level of detail provided in this document. The key to the meaning of the text is as follows: Normal Oracle VM process: A step with this designation means there is nothing special about that particular step beyond the normal Oracle VM implementation process; you should already be very familiar with the process noted in a step with this designation. 20

25 DR specific Oracle VM process: A step with this designation means that the integration process deviates in some way from the normal implementation methodology for Oracle VM. You should pay particular attention to any step with this designation since it will impact the success of your project. Follow vendor specific solution: A step with this designation indicates that the step has a vendor specific solution and process. This paper articulates what Oracle VM will expect for these steps, but offers no particular way of accomplishing the task since that should have been devised by the team responsible for devising the overall DR solution at your site. Before Beginning the Installation and Configuration Process The following subsections discuss some important things to remember before beginning the process of installing and configuring Oracle VM at Site A and B. Servers for Site B All Oracle VM Servers should already be cabled and powered up at Site B. Network cabling should be verified to ensure each network device is connected to the correct subnets, VLANs or broadcast domains on the Ethernet switches. Service processors such ILOMs should be configured and tested for access before attempting to undertake any tasks related to Part 2 of this white paper. The Oracle VM Server product should be installed on each server, but the servers must not be part of any existing server pools. The servers must not belong to an existing server pool since it will prevent you from matching the Site A network IDs for the virtual machine networks at Site B. If you are using bonded ports without VLANs, then the Oracle VM Servers must not be part of any existing server pools. However, if you plan on using VLANs for 100% of the virtual machine networks at Site B, then the servers can already be part of an existing server pool. Storage for Site B All storage arrays for Site B should be up and running before installing and configuring Oracle VM. All storage that will be used to create the recovery server pool such as pool file system and a temporary storage repository should already exist and be presented to the Oracle VM Servers at Site B. Stop! The Site B Oracle VM Servers must not belong to an already existing server pool if you are only using bonded interfaces without VLANs. The pool file system and a temporary storage repository (discussed below) should already exist and be presented to the Oracle VM Servers. Fibre Channel disks will appear as soon as Oracle VM Server is installed if your Fibre Channel HBAs are configured correctly and the LUNs are presented to each of the severs; this will make it easier to validate that the physical disks can be seen by each of the servers. Physical disks presented via iscsi will not appear until the iscsi array is registered in the Oracle VM Manager. So, physical disks presented to Oracle VM Servers via iscsi cannot be verified until much You will not be able to match the network IDs from Site A if the servers already belong to a server. 21

26 later in the process. However, ensure that the physical disks are presented to the Oracle VM Servers before beginning the install and configuration of Site B even though the disks won t be seen by the server until later on. Pool file system As mentioned in Part 1, a pool file system that is completely independent of the pool file system at the primary site (Site A) must be presented to the Oracle VM Servers at the recovery site (Site B). Storage repository A small, temporary storage repository should be created and presented to all Site B Oracle VM Servers. This temporary repository will be used to store a few very simple Oracle VM Guests that will be used exclusively for validating network and storage access at Site B. A 36 Gigabyte storage repository is all that is needed. Just enough space to contain an Oracle VM Template for Oracle Linux and one or two Oracle VM Guests that will be created using the templates. Oracle VM Guests for validation Create one or two Oracle VM Guests based on an Oracle VM Template once the Site B server pool is created and the temporary storage repository is presented to all the Site B Oracle VM Servers. The Oracle VM Guests will only be used to validate networking to ensure everything is configured correctly at Site B. Do not add any applications or do anything with the guest operating systems other than configure networking. The goal is to keep the guest operating systems very simple with just enough changes to emulate the network configuration for Site A Oracle VM Guests. We do not want to introduce too many changes to the operating systems that cause more troubleshooting than necessary. These temporary Oracle VM Guests can be removed from the server pool once the Site B networking has been validated. Step 1: Install and Configure Oracle VM at Site A Ensure that Oracle VM has been installed, fully configured to fit the requirements of your computing environment. Also ensure the entire Site A solution has been validated and is completely stable before and during the entire Oracle VM integration process. Installing and configuring Oracle VM 3 is beyond the scope of this document. Please refer to the Oracle VM Documentation on Oracle Technology Network for release for detailed information regarding the installation and configuration of server pools and Oracle VM Guests. Stop! It is especially important that Site A remain completely stable before and during the entire integration process. There should be zero changes being made at the Site A environment. Step 2: Install Oracle VM Servers at Site B Note that this step does not include installing the Site B Oracle VM Manager it is important that you do not install the Oracle VM Manager until later steps. 22

27 Step 2.1: Install Oracle VM Server on Site B servers Install the Oracle VM Server product on the physical servers at Site B. Once this is completed, we will make some manual modifications to the installed operating system before the servers are discovered. The manual modifications are explained in step 3. Step 3: Replicate Site A Network IDs to Site B This step is very import and must be completed before discovering the Site B Oracle VM Servers with the Site B Oracle VM Manager. Please refer to the Network Considerations section in Part 1 of this paper if you are not familiar with the purpose of this step or are not familiar with the reason for ensuring virtual machine network IDs match at both sites. Step 3.1: Determine which metafiles to copy Stop! Don t attempt to install Oracle VM Manager or discover servers at Site B until step 3 is completed! Your Site A Oracle VM Guests will not be able to use the networking at Site B if this step is skipped. The network bridge names used for the virtual machine networks on the site B servers must match the bridge names on site A servers. This is a very important step to ensure the Oracle VM guests don t require manual editing to straighten out false networks during an actual failover attempt. The reason behind this particular step and the method for determining which network metafiles must be copied is explained in Part 1 of this paper. Please review the section on network considerations in Part 1 of this document if this step seems confusing or unclear. The following is an example of the process for copying the metafiles it is only an example copy only the actual metafiles that are appropriate for your environment. In the case of this example, only three network metafiles will need to be copied assuming the three fictional VLANs shown in Figure 16 below. Figure 16: From Site A, copy only the metadata files for Oracle VM networks with the Virtual Machine channel assigned Run the following commands shown in Figure 17 below on any Site A Oracle VM Sever that is part of the disaster recovery solution. 23