NetApp for Oracle Database

Transcription

1 NetApp Verified Architecture NetApp for Oracle Database Enterprise Ecosystem Team, NetApp November 2012 NVA-0002 Version 2.0 Status: Final

2 TABLE OF CONTENTS 1 NetApp Verified Architecture NetApp for Oracle Database NVA Overview Problem Statement Target Audience Technology Solution Use Case Summary Solution Use Cases Use Case 1: Data Replication in Database Environments Use Case 2: Conversion of Physical Servers to Virtual Machine Images (P2V) Use Case 3: Cloning of Oracle VM Virtual Machines Use Case 4: Cloning of Oracle Databases Between Physical Servers and Virtual Machines Data ONTAP Operating in Cluster-Mode Use Case 5: Storage Migration and Database Consolidation Use Case 6: Dynamic Scalability with Online Scale-Up and Scale-Out Use Case 7: Live Migration of Data Volumes Use Case 8: Operational Flexibility Use Case 9: Oracle Running on VMware with NFS and SAN Datastores Use Case 10: Application of Storage-Efficiency Features on VM Datastores and Oracle Databases Design Validation Solution Architecture Details Hardware Requirements Software Requirements Solution Sizing and Performance Considerations Network Infrastructure Details Storage Infrastructure Details Virtual Infrastructure Solution Manageability Success Stories Conclusion References Supporting Documents NetApp for Oracle Database

3 Version History LIST OF TABLES Table 1) NetApp for Oracle Database solution use cases Table 2) Production hardware resources Table 3) Dev/test hardware resources Table 4) Production software resources Table 5) Dev/test software resources Table 6) Networks Table 7) Storage Table 8) Virtual infrastructure LIST OF FIGURES Figure 1) Physical architecture of the NetApp for Oracle Database solution (with representative connectivity lines added for readability) Figure 2) Data replication in database environments using Oracle Data Guard Figure 3) Oracle RAC with Data Guard physical standby Figure 4) Oracle RAC database replication with Oracle Data Guard Figure 5) Rapid cloning and provisioning of VMs Figure 6) Oracle VM cloning using the NetApp file and volume FlexClone capability Figure 7) Cloning of databases (on NFS and FCP/ASM) between physical servers and virtual platforms Figure 8) Cloning of databases (on FCP/ASM) between physical servers and Oracle VM Figure 9) Migration diagram Figure 10) Database consolidation Figure 11) Scaling up and out Figure 12) Failover protection Figure 13) Data volume migration Figure 14) Logical architecture of the solution Figure 15) Oracle dev/test use cases Figure 16) Network configuration of VMware environment NetApp for Oracle Database

4 1 NetApp Verified Architecture The new NetApp Verified Architecture (NVA) program offers customers a validated architecture for NetApp solutions. The NVA provides customers with a NetApp solution architecture that: Is thoroughly tested Is prescriptive Minimizes customers deployment risk Accelerates their time to results The NVA program is governed by a group of NetApp technical experts who verify that each NVA: Consists of a pretested, preintegrated verified architecture that meets specific customer needs Is backed up by prescriptive deployment, testing, and operational procedures Has a proven design that is based on prescriptive best practices Reduces the complexity of architecting and implementing NetApp technologies 2 NetApp for Oracle Database NVA Overview The NetApp for Oracle Database NVA enables companies to optimize and protect their Oracle Database infrastructures by providing advanced storage and data management capabilities. The NetApp industryleading storage solutions provide the features, scalability, flexibility, and availability necessary for a dynamic organization. 2.1 Problem Statement NetApp customers encounter various primary business challenges when operating and protecting Oracle Databases: Operational backup and restore within shrinking operational windows and with shrinking budgets Efficient data replication from a primary to a secondary environment Provisioning of copies of the primary database environment for development, test, QA, and other purposes in physical and virtual environments Data protection for DR purposes Meeting service-level agreements within a constrained IT budget Customers can leverage NetApp technology to meet these needs while saving precious resources. This document focuses on the NetApp for Oracle Database application use case to demonstrate how NetApp products and solutions can help customers meet these business challenges. By deploying the architecture referenced in this document, customers can be positioned to effectively meet the business challenges of optimizing and protecting their Oracle Database infrastructure. 2.2 Target Audience This document is intended for the following groups: Development managers IT staff Technical directors, architects, and consultants Technical sales and presales engineers 4 NetApp for Oracle Database

5 2.3 Technology Solution The following challenges are typical of those faced by customers who deploy Oracle application development and test (dev/test) environments: Excessive time required to clone and provision a database environment Increased storage capacity consumption required to support multiple copies of databases Cost considerations of the dev/test environment For customers who manage Oracle application dev/test environment databases on third-party storage, the simple process of cloning, creating, and restoring an Oracle Database can take hours, if not days. This can lead to excessively long dev/test cycles. A large number of dataset copies are also usually required in dev/test environments, presenting customers with the problem of significant and constant storage growth caused by maintaining multiple environments. This growth places additional pressure on the power, space, and cooling capacity of data centers. Moreover, supporting Oracle application dev/test environments in a mixed infrastructure that has a combination of physical and virtual servers creates its own set of challenges. Cloning databases between physical servers and virtual machines (VMs) can be a labor-intensive and time-consuming task. Managing storage efficiency for virtualization platforms and the overall dev/test environments on traditional storage systems can be difficult, and the effort required can be significant. By leveraging NetApp storage systems featuring core technologies such as Snapshot technology, space-efficient FlexClone technology, and compression, customers can greatly reduce the time and storage capacity consumed by backup, replication, and provisioning operations. The architecture described in the application dev/test use case is designed to simulate a customer s infrastructure, which normally consists of a production environment and a corresponding dev/test environment. Production data is assumed to be replicated to the dev/test environments and is subsequently duplicated and cloned to create copies for further dev/test purposes. This use case also incorporates a disaster recovery (DR) component, which is the replicated secondary site for production. This in itself delivers additional value because customers can use the dev/test infrastructure for its core purpose while leveraging it to create a DR framework for production, further improving data protection for the production environment. Customers who have existing DR infrastructures can use them as the source to be duplicated for dev/test environments. The architecture depicted in Figure 1 illustrates a production environment and its corresponding dev/test environment. 5 NetApp for Oracle Database

6 Figure 1) Physical architecture of the NetApp for Oracle Database solution (with representative connectivity lines added for readability). In summary, the components of the NetApp for Oracle Database solution combine to accomplish the following tasks: Replication of production databases running on NetApp or third-party storage to dev/test running on NetApp storage Cloning of databases to and from varying environments: Physical to physical Physical to virtual Virtual to virtual Conversion of physical server images to VM images Rapid cloning and provisioning of VMs Application of thin provisioning, space-efficient cloning, deduplication, and compression 2.4 Use Case Summary The use cases that compose application dev/test environments demonstrate the following features and functions: Replication of the production database to the dev/test environment using Oracle Data Guard in a heterogeneous environment (in which production might be running on a different OS platform from that of the dev/test environment, and production might be residing on third-party storage) Replication of production database volumes to the dev/test environment by using NetApp SnapMirror, with the production data residing on either NetApp FAS series or V-Series storage Cross-protocol or same-protocol replication of the production database by using Oracle Data Guard Note: The production database is on Fibre Channel Protocol (FCP)/Automatic Storage Management (ASM), and the Data Guard physical standby database is on Network File System (NFS) or FCP/ASM. Configuration of Oracle Data Guard, consisting of a physical standby database and a cascading physical standby database 6 NetApp for Oracle Database

7 Mixed configuration of databases, in which production is a multinode Oracle Real Application Clusters (RAC) and the Oracle Data Guard physical standby database is a single-instance RAC Conversion of physical server images to VMware VM images and Oracle VM (OVM) images (known as P2V) Cloning of Oracle Databases between physical and virtual platforms by using SnapManager for Oracle (physical to virtual, virtual to virtual, with VMware and Oracle VM) Cloning of Oracle Database volumes between physical and Oracle VM platforms by using Snap Creator Cloning of Oracle Databases between physical and Oracle VM platforms by using the NetApp Manageability Software Development Kit (NMSDK) and Perl scripting Thin-provisioning of datastores in a VMware environment by using the NetApp Virtual Storage Console (VSC) plug-in for VMware Mass cloning and deployment of VMware VMs by using the NetApp VSC plug-in for VMware Mass cloning of Oracle VMs by using NetApp file-level FlexClone technology Leveraging of NetApp deduplication and compression technologies to reduce the storage footprint involved in maintaining clones, or duplicates, of databases and VMs 3 Solution Use Cases Table 1 lists the use cases that are integrated into the NetApp for Oracle Database solution. Table 1) NetApp for Oracle Database solution use cases. Use Case 1: Data Replication in Database Environments Use Oracle Data Guard physical standby and cascading physical standby to replicate production database (cross-platform, cross-protocol). Use NetApp volume SnapMirror to replicate production database volumes. Use Case 2: Conversion of Physical Servers to Virtual Machine Images (P2V) Convert physical server images to VMware VM images. Convert physical server images to Oracle VM images. Use Case 3: Cloning of Oracle VM Virtual Machines Use NetApp file-level FlexClone technology to clone Oracle VM images. Use Case 4: Cloning of Oracle Databases Between Physical Servers and Virtual Machines Use SnapManager for Oracle to clone databases. Use Snap Creator to clone database volumes. Use Perl and NMSDK to clone databases. Use Case 5: Storage Migration and Database Consolidation Use Oracle Data Guard to migrate Oracle Database storage from Data ONTAP operating in 7-Mode to Data ONTAP operating in Cluster-Mode. Use Data ONTAP Cluster-Mode storage to consolidate databases with Oracle RAC and/or Oracle RAC One Node. 7 NetApp for Oracle Database

8 Use Case 6: Dynamic Scalability with Online Scale-Up and Scale-Out Add more storage disk shelves (scale up) or add additional storage controllers (scale out). Achieve transparent controller clustering and failover capability to provide continuous operations. Use Case 7: Live Migration of Data Volumes Use DataMotion for Volumes to move database volumes across storage controllers. Move logical interfaces (LIFs) along with volumes for continued access to data, load balancing, and workload balancing. Use Case 8: Operational Flexibility Use DataMotion for Volumes to move data between different storage aggregates and controllers within the cluster. Use DataMotion for Volumes to move data between aggregates made up of a higher number of disks or faster drive types. Use Case 9: Oracle Running on VMware with NFS and SAN Datastores Provision Data ONTAP Cluster-Mode NFS and Fibre Channel (FC) (FC raw device mapping [RDM] logical unit numbers [LUNs]) storage for VMware environments. Clone and provision VMware VMs using the NetApp VSC plug-in. Use Case 10: Application of Storage-Efficiency Features on VM Datastores and Oracle Databases Use SnapManager for Oracle on NFS-based Cluster-Mode storage to clone databases. Use Snap Creator on NFS-based and FC-based Cluster-Mode storage to clone database volumes. Use the NetApp VSC plug-in to clone and provision VMware VMs. 3.1 Use Case 1: Data Replication in Database Environments This use case assumes that the environment is heterogeneous. Either the database server platform in production is different from the database server platform in dev/test, or the storage platform in production is different from the storage platform in dev/test. Therefore, Oracle Data Guard is used to provide database replication for heterogeneous platforms, as shown in Figure 2. This use case addresses two scenarios: A replication of the production database to the dev/test environment in which the production database is on FCP/ASM-based storage while the replicated (standby) database in dev/test is on NFS-based storage. This scenario is configured with the Oracle Data Guard physical standby database. A replication of the production database to the dev/test environment in which the production database is on FCP/ASM-based storage and the replicated (standby) database in dev/test is also on FCP/ASMbased storage. This scenario is configured with Oracle Data Guard cascading a physical standby database. 8 NetApp for Oracle Database

9 Figure 2) Data replication in database environments using Oracle Data Guard. Figure 3 shows the high-level architecture of the Oracle Database replication from primary database storage to dev/test storage using Oracle Data Guard. Figure 3) Oracle RAC with Data Guard physical standby. This process includes the following steps: 1. Prepare the primary database for DR. 2. Configure the standby environment. 3. Instantiate the standby copy. 4. Validate the redo transport and the standby database. Architecture Figure 4 shows an example of a typical architecture of production and dev/test environments. 9 NetApp for Oracle Database

10 In this example, the production site is on third-party storage, where Data Guard is used to replicate data to the dev/test site, which resides on the physical server infrastructure. The production database is RAC, and the standby database is a single instance. Figure 4) Oracle RAC database replication with Oracle Data Guard. Homogeneous implies that the production site uses the same database server platform as the dev/test environment and that the production database resides on NetApp storage systems. In this case, NetApp SnapMirror can be used to replicate the production database volumes to the dev/test site. This use case demonstrates NetApp volume SnapMirror Async with network compression. In this mode, SnapMirror performs incremental, block-based replication as frequently as once per minute. The first and most important step in this mode involves the creation of a one-time baseline transfer of the entire dataset. This step is required before incremental updates can be performed. This operation proceeds as follows: 1. The source storage system creates a Snapshot copy (a read-only, point-in-time image of the file system). This copy is called the baseline copy. 2. All data blocks referenced by this Snapshot copy and any previous copies are transferred by using volume SnapMirror and are written to the destination file system. Qtree SnapMirror copies only the latest Snapshot copy. 3. After the initialization is complete, the source and destination file systems have at least one Snapshot copy in common. Scheduled or manually triggered updates can occur after the initialization is complete. Each update transfers only the new and changed blocks from the source to the destination file system. This operation proceeds as follows: 10 NetApp for Oracle Database

11 1. The source storage system creates a Snapshot copy. 2. The new copy is compared to the baseline copy to determine which blocks have changed. 3. The changed blocks are sent to the destination and written to the file system. 4. After the update is complete, both file systems have the new Snapshot copy, which becomes the baseline copy for the next update. The Snapshot copy that will be transferred must be a database-consistent image; therefore, the database should be placed in hot-backup mode when creating a Snapshot copy of the source volume. Asynchronous replication is periodic. Therefore, SnapMirror Async is able to consolidate the changed blocks and conserve network bandwidth with minimal impact on write throughput and write latency. SnapMirror Async can also take advantage of a feature known as compression. SnapMirror network compression enables data compression over the network for SnapMirror transfers. This native feature is built into SnapMirror software. SnapMirror network compression is not the same as WAFL (Write Anywhere File Layout) compression. SnapMirror network compression does not compress data at rest. However, it is very beneficial when used with database replication based on initial testing. 3.2 Use Case 2: Conversion of Physical Servers to Virtual Machine Images (P2V) To expedite the process of creating virtual environments in which the VMs are provisioned with the desired software configuration, existing (Linux and Windows ) physical servers can be duplicated and converted into VM images. For example, in the case of this dev/test solution, a physical Linux server was configured with a required set of software applications (such as Oracle Relational Database Management System [RDBMS] with ASMLIib, NetApp SnapDrive for UNIX, SnapManager for Oracle, Snap Creator Agent, Host Utilities Kit, and so on); it was then used as the source server in the physical-to-virtual conversion. Using this method, the resulting VM images (which are conversions of physical server images) contain the required applications and software. Therefore, this method eliminates the need to install individual applications or software on the VM guest operating system (OS) after the VM has been created. This use case applies to the physical-to-virtual conversion process for both VMware and Oracle VM environments. The P2V conversion for VMware can be an online process, whereas the P2V conversion for Oracle VM is a cold, or offline, conversion. 3.3 Use Case 3: Cloning of Oracle VM Virtual Machines The cloning of Oracle VM images can be simplified and streamlined by using NetApp file-level FlexClone technology. With the file-level FlexClone capability, the gold master VM image or template can quickly be cloned and imported into Oracle VM Manager to create new VMs. Figure 5 outlines this process. 11 NetApp for Oracle Database

12 Figure 5) Rapid cloning and provisioning of VMs. In November 2007, Oracle introduced Oracle VM, Xen architecture-based server virtualization software that fully supports both Oracle applications and applications other than Oracle. Oracle VM offers scalable, highly efficient, and low-cost server virtualization. Consisting of open source server software and an integrated browser-based management console, Oracle VM provides an easy-to-use GUI for creating and managing virtual server pools running on x86- and x86 64-based systems across an enterprise. Oracle VM Server for x86 is a free, next-generation server virtualization and management solution from Oracle that makes enterprise applications easier to deploy, manage, and support. Users can create and manage VMs that exist on the same physical server but behave like independent physical servers. Each VM created with Oracle VM has its own virtual CPUs, network interfaces, storage, and OS. With Oracle VM, users have an easy-to-use, browser-based tool for creating, cloning, sharing, configuring, booting, and migrating VMs. The Oracle VM platform includes three components: Oracle VM Manager. A Web-based application that enables users to manage and administer VMs and Oracle VM environments. Oracle VM Server. A self-contained virtualization environment that is based on an updated version of the underlying Xen hypervisor technology. Oracle VM Agent. A component of Oracle VM Server that provides a set of Web services application programming interfaces (APIs) and communicates with Oracle VM Manager for management of VMs, server pools, and resources. For more information, refer to the Oracle VM Documentation Web page. Oracle VM supports two types of VMs: Hardware virtualized. The guest OS does not require modification. It is available only on Intel virtualization technology and AMD secure virtualization machine CPUs. 12 NetApp for Oracle Database

13 Paravirtualized. The guest OS is recompiled for optimal performance in the virtual environment. A number of versions of Linux and Windows are supported as guest OSs on either 32-bit or 64-bit server platforms. Oracle VM uses native Linux device drivers so that users do not have to wait for the latest hardware to be supported by the virtualization solution layer. For more information on Oracle VM Server technology, refer to the Oracle VM Documentation Web page. When VMs are deployed on Oracle VM repositories that reside on NetApp NFS volumes, individual VM system images can be quickly cloned by using NetApp file-level FlexClone technology. The resulting cloned system images can then be registered and managed with Oracle VM Manager as new VMs. The additional benefit of using the NetApp file-level FlexClone capability to clone Oracle VM images is the immediate realization of storage savings. This technology avoids duplication of blocks between the source file and the clone file, thus minimizing the storage consumption required to create the clone file. Mass cloning of VMs can be accomplished rapidly with NetApp volume FlexClone technology. It can be used to clone an Oracle VM repository, which might contain many VMs. For example, an Oracle VM repository might have 20 VMs; a single clone of the volume that hosts the Oracle VM repository instantly creates 20 more VMs. As Figure 6 illustrates, large numbers of VM clones can be created by simply performing volume clones of the repository. Figure 6) Oracle VM cloning using the NetApp file and volume FlexClone capability. Guidelines for Using Oracle VM Follow these guidelines for using Oracle VM: Create a VM template for each major version or configuration of the VMs (for example, templates for database VMs and for application VMs). Create and maintain golden images. Similar to VM templates, golden images can represent major versions or configurations of VMs. However, golden images can easily be started, and their contents (such as OS drivers, configurations, applications, and so on) can be updated, whereas templates are read-only copies of VM images. 13 NetApp for Oracle Database

14 3.4 Use Case 4: Cloning of Oracle Databases Between Physical Servers and Virtual Machines This use case assumes a mixed infrastructure with the following requirements: Cloning of databases between physical servers and virtual environments Cloning of databases between different virtualization platforms Figure 7 shows the cloning of databases between physical servers, VMware VMs, and Oracle VMs, for both FCP/ASM-based storage and NFS-based storage, using space-efficient NetApp FlexClone technology. Figure 7) Cloning of databases (on NFS and FCP/ASM) between physical servers and virtual platforms. For databases residing on NFS-based storage, SnapManager for Oracle can be used to clone databases between physical servers, VMware VMs, and Oracle VMs, as long as the end OSs are the same. For databases residing on FCP/ASM-based storage, SnapManager for Oracle supports the cloning of databases from physical servers to VMware VMs (using FC RDM LUNs). In the case of Oracle VM, databases can be cloned from physical servers by using either Snap Creator or Perl with NMSDK. Figure 8 shows the cloning of databases between physical servers and Oracle VM for FCP/ASM-based storage. 14 NetApp for Oracle Database

15 Figure 8) Cloning of databases (on FCP/ASM) between physical servers and Oracle VM. 3.5 Data ONTAP Operating in Cluster-Mode The following use cases illustrate key ways to leverage the primary benefits of using Oracle Database products in an environment running NetApp Data ONTAP Cluster-Mode. Database and storage consolidation can be achieved by integrating Oracle RAC or RAC One Node with NetApp Data ONTAP Cluster-Mode, a scale-out and dynamic storage platform. The Oracle Data Guard solution enables organizations to migrate their data nondisruptively: Note: From Data ONTAP 7G From Data ONTAP Mode Even from another vendor s storage to NetApp Data ONTAP Cluster-Mode Data ONTAP Cluster-Mode does not support consistency group Snapshot copies; therefore, ASM and Logical Volume Manager (LVM) design considerations must be taken into account. There should be no disk groups that span multiple FlexVol volumes. All NetApp LUNs that make up a given LVM/ASM disk group must be placed in a single FlexVol volume. If a database is distributed across nodes, each node must have independent LVM/ASM disk groups in dedicated FlexVol volumes. Placement of LUNs in this manner requires only a single Snapshot copy of a single FlexVol volume to protect that disk group, thus avoiding the need for consistency group Snapshot copies. The ability of SnapManager for Oracle and SnapDrive for UNIX to work with this configuration is still under analysis, but following these rules offers a potential path to supportability even before consistency groups become available in Data ONTAP. With Cluster-Mode, scaling up is achieved by adding disk arrays to existing controllers to increase input/output operations per second (IOPS) and capacity. Additional and larger storage systems can be added to the existing storage cluster to meet scaling-out demand. Cluster-Mode s online nondisruptive storage expansion and reconfiguration capability make it the industry-leading scale-out storage platform. The features described in the following use cases cover the final business challenge identified in section 2.1, the challenge of meeting service-level agreements within a constrained IT budget. 15 NetApp for Oracle Database

16 3.6 Use Case 5: Storage Migration and Database Consolidation As companies grow, multiple Oracle Database and application environments can develop within the organization, each with its own management, availability, and growth needs. With the advent of virtualization, many organizations are undertaking consolidation projects to reduce the cost and complexity and increase the efficiency of the IT infrastructure. This consolidation has resulted in increased service-level and growth demands. Clustering, including Oracle RACs, has addressed the availability and scalability needs at the server and application levels. The same consolidation must take place at the storage layer for the same reasons as those previously mentioned. NetApp clustered storage systems provide the flexibility, efficiency, and scalability to meet and exceed those needs. These consolidation projects have stimulated interest in NetApp storage clusters that use Data ONTAP Cluster-Mode. These projects include organizations that have storage from vendors other than NetApp or NetApp Data ONTAP 7G or Mode. The organizations need a way to safely migrate their Oracle data in a nondisruptive fashion. This solution, shown in Figure 9, meets all of those challenges by consolidating the disparate Oracle data storage environments into a single, scalable, resilient storage cluster. Figure 9) Migration diagram. As Figure 10 shows, NetApp Cluster-Mode technologies can combine with Oracle RAC and/or Oracle RAC One Node to provide a scalable, resilient environment into which databases can be consolidated. 16 NetApp for Oracle Database

17 Figure 10) Database consolidation. Instance caging and node affinity can be used to control resource boundaries at the server and database levels for different databases as they are consolidated into a single Oracle grid cluster. Storage resource isolation can also be accomplished by defining different Vservers to store data files for different databases. 3.7 Use Case 6: Dynamic Scalability with Online Scale-Up and Scale-Out A result of storage consolidation is that the availability and resiliency of the central cluster become even more vital. Many departments and applications are relying on the availability of their resources; therefore, the ability to scale storage up and out is not only desirable but also required. With Data ONTAP Cluster- Mode, scaling up is achieved by adding disk arrays to existing controllers to increase IOPS and capacity. Additional and larger storage systems can be added to the existing storage cluster to meet the scaling-out demand. The online nondisruptive storage expansion and reconfiguration capability of Cluster-Mode makes it the industry-leading scale-out storage platform. Although they are not part of this use case, Flash Cache cards can also be applied for better read performance with most random-access workloads, a large percentage of which are read operations. Figure 11 shows the scale-up and scale-out process. 17 NetApp for Oracle Database

18 Figure 11) Scaling up and out. Storage failover protection, shown in Figure 12, is a core attribute of Data ONTAP Cluster-Mode. Highavailability (HA) pairs of controllers are the building blocks that together form the storage cluster. This architecture enables transparent controller clustering and failover capability, in which a failed storage controller causes its partner node to take over its disk arrays, volumes, and running services to provide continuous operation. 18 NetApp for Oracle Database

19 Figure 12) Failover protection. 3.8 Use Case 7: Live Migration of Data Volumes Data ONTAP Cluster-Mode is equipped with NetApp DataMotion for Volumes software, which is a nondisruptive transparent volume move feature. Volumes can be moved between storage systems within the storage pool transparently, without requiring coordination between applications or application downtime. This single feature has many benefits, including the following: Eliminates application outages Provides always-on data access Enables load and resource tiering Improves resource utilization Facilitates dynamic scaling Provides phased upgrades (tech refresh) The DataMotion for Volumes feature of Cluster-Mode can also be used to facilitate online load balancing, adjust to the service-level demand, and manipulate resource utilization. The LIF migration capability of Data ONTAP allows the kernel NFS (knfs)-mounted and Oracle Direct NFS (dnfs)-mounted LIFs to be moved with the volume, enabling continued primary local access of the data. As a best practice, move the volume and the corresponding LIF together in NFS environments. If the volume is moved alone, the data traffic takes remote access through cluster interconnect, which might degrade performance if there are many of these activities. In the case of FC environments, there is no need to carry out LIF migration (unlike with NFS) because it is taken care of through the host-side multipathing I/O (MPIO) settings. Figure 13 shows the data volume migration process. 19 NetApp for Oracle Database

20 Figure 13) Data volume migration. 3.9 Use Case 8: Operational Flexibility DataMotion for Volumes in Cluster-Mode provides the following benefits: Volumes can be selectively moved and redesignated to different aggregates and controllers within the cluster to drive a better average utilization across the storage pool. Increased demand in input/output (I/O) can be met by relocating volumes to aggregates made up of a higher number of disks or faster drive types. Performance and capacity gains to meet new service levels can be achieved by relocating volumes to higher performance aggregates or controllers. The controllers can be of a different class, have higher specifications, or be equipped with solid-state drive and Flash Cache. Some storage systems might suffer from sustained high load, or certain volumes might be particularly hard hit, which could result in significant hot spots and unsatisfactory levels of service to applications. Moving the subject volumes to different storage nodes might alleviate the problems. If Oracle Clusterware and RAC must be migrated to a new storage system, DataMotion for Volumes can be used to move not only the database volumes but also the crucial Clusterware volumes that store Oracle Cluster Registry and voting disks, with no disruption to the operation of Oracle Clusterware and RAC databases Use Case 9: Oracle Running on VMware with NFS and SAN Datastores NetApp Data ONTAP clustering provides a storage virtualization solution that can dramatically reduce the time, resources, and effort required for rapid and cost-effective database deployment. The NetApp unified Cluster-Mode storage architecture, coupled with VMware virtualization technologies, provides simplified management and support across storage protocols, including NFS and FC storage area network (SAN): Server virtualization technologies provided by VMware vsphere allows creation of base VMs that run an Oracle RAC database with NFS, dnfs, and FC SAN. The failover, load balancing, performance, and capacity management features provided by Cluster-Mode storage go well hand in hand. 20 NetApp for Oracle Database

21 Oracle Database running on guest VMs has the database files on physical disks hosting flexible volumes in the namespace on NetApp Cluster-Mode mounted over NFS, dnfs, and FC SAN storage. NetApp offers unique advantages related to hardware-assisted Snapshot copies and space reservation for Oracle VMs, making it possible to create multiple space-efficient clones for other purposes, such as reporting or training. NetApp VSC for VMware vsphere can be used to manage VMware ESX and VMware ESXi servers connected to NetApp Cluster-Mode storage systems. This plug-in is available to all VMware vsphere Clients that connect to the VMware vcenter server. It provides features to configure, monitor, provision, clone, back up, and recover VMs. NetApp Rapid Cloning Utility provides a quick and easy way to create database and VM clones. It uses instant cloning of Oracle guest VMs to a different node in the storage cluster for development and test environments, which can also be offloaded to storage using VMware vstorage APIs for Array Integration (VAAI) Use Case 10: Application of Storage-Efficiency Features on VM Datastores and Oracle Databases Traditional database application cloning is very time consuming. More storage capacity overhead is required for creating multiple clones for development and test environments. When multiple copies of the production Oracle Database are needed, there are challenges of space efficiency, cost effectiveness, and time management. Implementing the Oracle application dev/test environment with the powerful capabilities of storage efficiency technologies from NetApp Data ONTAP Cluster-Mode and VMware server virtualization technologies provides the following advantages: Efficient Oracle Database cloning with NetApp and VMware improves database administrator productivity by providing space-efficient, on-demand database cloning. For database cloning, NetApp FlexClone technology can be used to replicate data volumes and datasets as transparent virtual copies instantaneously without affecting the production or master database copy. FlexVol volumes allow the database administrator to create nearly instantaneous, space-efficient, writable clones of a master or production database. VMware VAAI enables rapid provisioning of server and storage resources and speeds up the dev/test lifecycle. This allows customers to move to an environment in which commodity clones are widely deployed to more users to speed development and testing tasks. The combination of storage virtualization with NetApp Data ONTAP Cluster-Mode and server virtualization with VMware addresses the challenges of provisioning adequate server and storage resources. The VMware ESX server HA feature fits very well with NetApp Cluster-Mode features to help database environments meet HA requirements across the stack. For databases hosted on NFS-based storage on VMware, SnapManager for Oracle can be used to clone databases along with VMware VMs. For databases hosted on FCP/ASM-based storage on VMware, cloning can be achieved by using Snap Creator. Snap Creator can be used to facilitate the Snapshot copies and cloning with NFS-based as well as FCP-based Cluster-Mode storage, but the same storage design considerations just described must be taken into account. NetApp also offers database and VMware administrators automated and powerful tools, such as SnapManager for Oracle and Rapid Cloning Utility, to easily create nearly instantaneous database clones on demand, using negligible additional storage with NetApp Data ONTAP Cluster-Mode storage, without the need for extensive storage expertise. 4 Design Validation The design used for the dev/test use case has been tested and validated by the NetApp Enterprise Applications Solution Team. Several NetApp customers are currently running this configuration with great success. 21 NetApp for Oracle Database

22 The NetApp Snapshot and SnapRestore technologies are the choice of almost all NetApp customers who use Oracle Database. NetApp considers these technologies to be the core on which all other use cases are based. Although the solution concentrates on the cloning and provisioning of databases and virtual infrastructure to enable rapid deployment of dev/test environments, to some extent the solution does embed components of DR and of backup and restore: Disaster recovery. This is demonstrated through the methods of database replication and storage volume replication. Database replication is the key function behind the use case of heterogeneous environment data replication, which relies on Oracle Data Guard to provide the replication mechanism. Storage volume replication is achieved by using NetApp volume SnapMirror, which is associated with the use case of homogeneous environment data replication. Backup and restore. The database cloning process essentially encapsulates the backup and recovery components within itself. The database image is first backed up (using Snapshot technology), and the backup image (the Snapshot copy) is subsequently cloned and used to create a new, cloned database. When the benefits described in use cases 1 through 10 are combined, the net result is a consolidated, highly flexible, vastly scalable, nonstop operational environment that helps NetApp customers meet evermore-demanding service-level agreements. 4.1 Solution Architecture Details The NetApp for Oracle Database solution is designed to support real-world environments in which an IT infrastructure might consist of a mix of physical servers and different virtualization platforms. The solution architecture represents a typical customer scenario that has Oracle Databases running in a production environment in which a DR site exists that can be leveraged to facilitate the creation of dev/test environments. In the absence of a DR site, the dev/test environment can be configured to replicate data from production, which then can act as a temporary DR site and at the same time serve as the master data source for dev/test clonings. At a high level, the solution is about creating and deploying an Oracle application dev/test environment in a mixed infrastructure by using the production environment as the data source, as shown in Figure 14. Within the solution itself, this can be viewed as integrating multiple use cases. Each use case delineated in Figure 15 represents a different functional area. 22 NetApp for Oracle Database

23 Figure 14) Logical architecture of the solution. 23 NetApp for Oracle Database

24 Figure 15) Oracle dev/test use cases. Characteristics of the Production Environment The Oracle production environment shown in Figure 15 includes the following characteristics: A two-node Oracle Clusterware configuration hosting a two-instance Oracle RAC production database Oracle Clusterware nodes using gigabit network and interfaces for the cluster interconnect In the heterogeneous use case, use of Oracle Data Guard to replicate the production database to the dev/test environment Note: In this case, the production servers can be either Solaris x64 or Linux, and the production storage can be either third party or NetApp. In the homogeneous use case, use of NetApp SnapMirror to replicate production data to the dev/test environment Note: In this case, the production servers must be on the same platform as the dev/test servers, and the production storage must be NetApp storage. NFS and FC storage connectivity between the servers and the storage systems A pair of FC switches for FCP storage connectivity Gigabit switches for NFS storage connectivity Storage systems in an HA configuration 24 NetApp for Oracle Database

25 Characteristics of the Development and Test Environment The Oracle application dev/test environment shown in Figure 15 includes the following characteristics: A two-node Oracle Clusterware configuration hosting either Oracle Data Guard physical standby databases for production or SnapMirror replicated database images of the production database Oracle Clusterware nodes using gigabit network and interfaces for the cluster interconnect A physical server (Linux) that is a standalone database host Multiple VMware VMs running Linux guest OSs that are standalone database hosts Multiple Oracle VMs running Linux guest OSs that are standalone database hosts A VM with a Linux guest OS that hosts Oracle Enterprise Manager Grid Control and Oracle VM Manager A VM with a Windows guest OS that hosts VMware vsphere, vcenter, and vclient, as well as the NetApp VSC plug-in A VM with a Linux guest OS that hosts applications and client software such as Snap Creator server and Perl with NMSDK NFS and FC storage connectivity between database servers and storage systems Gigabit switches for NFS storage connectivity on the physical database servers and a virtual infrastructure (VMware and Oracle VM servers) equipped with 10 Gigabit Ethernet (10GbE) for NFS storage connectivity HA configuration for storage systems 4.2 Hardware Requirements The physical infrastructure on which the solution was implemented consists of a production environment and a dev/test environment. Production Environment Table 2 lists the hardware resources required for the production environment. Table 2) Production hardware resources. Resource Specification Description Servers 2 x x64 servers Intel or AMD server with 8 cores, 32GB, 2 internal drives, 4GbE ports, dual-port HBA Oracle Clusterware and RAC nodes Storage 2 x NetApp storage systems Network NetApp FAS32XX storage systems in HA configuration with disk shelves, Gigabit Ethernet (GbE) adapter cards, and FC HBAs Storage systems Note: Although NetApp storage systems are used to support the production environment in this architecture, the storage systems can be third party in the use case of a heterogeneous environment in which Oracle Data Guard is the method of production replication. 25 NetApp for Oracle Database

26 Resource Specification Description 2 x GbE switches GbE network switches: One for public/management One for cluster interconnect Network descriptions: Public/management network Oracle Clusterware cluster interconnect network Storage Network 2 x FC switches 4Gb FC switches FCP storage connectivity network with dual paths to servers and storage systems 1 x GbE switch GbE network switch NFS storage connectivity network Development and Test Environment Table 3 lists the hardware resources required for the dev/test environment. Table 3) Dev/test hardware resources. Resource Specification Description Servers 2 x x64 servers Intel or AMD server with 8 cores, 32GB, 2 internal drives, 4GbE ports, dual-port HBA 1 x x64 server Intel or AMD server with 8 cores, 24GB, 2 internal drives, 4GbE ports, dual-port HBA 1 x x64 server Intel or AMD server with 16 cores, 48GB, 2 internal drives, 4GbE ports, dual-port HBA, dual-port 10GbE adapter card 1 x x64 server Intel or AMD server with 16 cores, 48GB, 2 internal drives, 4GbE ports, dual-port HBA, dual-port 10GbE adapter card 1 x x64 server Intel or AMD server with 12 cores, 48GB, 2 internal drives, 2GbE ports Oracle Clusterware and RDBMS RAC nodes Oracle RDBMS standalone database host VMware ESX Server Oracle VM Server VMware VMs for: Oracle Enterprise Manager Grid Control and Oracle VM Manager VMware vsphere, vcenter, and vclient and NetApp VSC Application and client software (Snap Creator server, NMSDK, Swingbench) Storage 2 x NetApp storage systems Network NetApp FAS32XX storage systems in HA configuration with disk shelves, GbE adapter cards, 10GbE adapter cards, and FC HBAs Storage systems 26 NetApp for Oracle Database

27 Resource Specification Description 2 x GbE switches GbE network switches: One for public/management One for cluster interconnect Network descriptions: Public/management network Oracle Clusterware cluster interconnect network Storage Network 2 x FC switches 4Gb FC switches FCP storage connectivity network with dual paths to servers and storage systems 2 x 10GbE switches 10GbE network switches 10GbE NFS storage connectivity network 1 x GbE switch GbE network switch GbE NFS storage connectivity network 4.3 Software Requirements Operating System Red Hat Enterprise Linux is the primary OS used in the implementation and validation of the target solution. The physical and virtual hosts that are designated as database servers are deployed with Red Hat Enterprise Linux. Within the use case of heterogeneous environment data replication, Solaris x64 is deployed in the simulated production environment to demonstrate Oracle Data Guard supporting heterogeneous platform replication. The Windows OS is used on the management host that supports vcenter Server, VMware vsphere Client, and the NetApp VSC plug-in. Application The primary application addressed by this solution is Oracle 11g RDBMS, also known simply as Oracle Database. This solution supports two types of databases: Oracle RAC, standalone (single instance) Data Guard physical standby In addition to the primary Oracle application, the solution is also composed of infrastructure software, management products, and manageability solutions from key vendors. Some of the supporting applications are fundamental to enabling the core values of the solution. Production Environment Table 4 lists the resources required for the production software. 27 NetApp for Oracle Database

28 Table 4) Production software resources. Resource Specification Description Oracle Oracle or software stack: Grid infrastructure RDBMS ASM Data Guard Recovery Manager (RMAN) or Oracle Clusterware and database software (which includes ASM, Data Guard, and RMAN) Solaris 20U9 x64 Database server OS Red Hat Red Hat Enterprise Linux bit Database server OS NetApp NetApp Data ONTAP operating in 7- Mode with FCP, NFS, Snapshot, SnapRestore, FlexClone, and SnapMirror capabilities Storage system software Development and Test Environment Table 5 contains a list of the resources for the dev/test software. Table 5) Dev/test software resources. Resource Specification Description Oracle Oracle or software stack: Grid infrastructure RDBMS ASM Data Guard RMAN Oracle VM Server Oracle VM Manager Oracle Enterprise Manager Grid Control or Oracle Clusterware and database software (which includes ASM, Data Guard, and RMAN) Oracle VM hypervisor Oracle VM hypervisor manager Oracle Enterprise Manager VMware VMware vsphere ESX 5.1 VMware ESX hypervisor vcenter Server and vclient 5.1 VMware Converter vcenter and vclient P2V converter Red Hat Red Hat Enterprise Linux bit Database servers and VM guests OS Microsoft Windows Server 2008 VM guests OS 28 NetApp for Oracle Database

29 Resource Specification Description NetApp Data ONTAP operating in 7-Mode with FCP, NFS, Snapshot, SnapRestore, FlexClone technology, SnapMirror, deduplication, and compression capabilities Data ONTAP operating in Cluster- Mode with cluster, FCP, NFS, Snapshot, SnapRestore, FlexClone technology, SnapMirror, deduplication, and compression capabilities Host Utilities Kit for Linux 6.0 Host Utilities Kit for ESX VSC 2.1 with Provisioning and Cloning module and Backup and Recovery module SnapDrive for UNIX SnapManager for Oracle 3.2 Snap Creator 3.5 NMSDK 4.0 Storage system and manageability software 4.4 Solution Sizing and Performance Considerations As with all Oracle Database deployments, NetApp recommends that customers work with their NetApp account teams to gather appropriate data to conduct a thorough sizing exercise. NetApp maintains an Oracle Database sizing tool to help the account team determine a recommended hardware configuration to meet the customer s current and future needs (based on data currently available). Performance is a topic that stimulates discussion. NetApp supports all protocols equally well. Performance and protocol decisions typically come down to the customer s needs and, on occasion, preferences. To help decide which protocol might best meet business needs, refer to the following technical reports: TR-3932: Red Hat Enterprise Linux Protocol Performance Comparison with Oracle Database 11g Release 2 TR-3961: Oracle Database 11g Release 2 Performance Using Data ONTAP 8.1 Operating in Cluster- Mode 4.5 Network Infrastructure Details The solution infrastructure employs multiple types of networks, as listed in Table 6. Most of these networks are based on industry-standard GbE. The storage network has both NFS-based and FC-based connectivity. The physical networks that support the NFS storage connectivity for the virtual environments (VMware and Oracle VM) are built on 10GbE components. Figure 16 shows the network configuration of the VMware environment. Table 6) Networks. Type of Network Public and management Oracle Clusterware: SCAN/VIP Oracle Clusterware: cluster interconnect Specification Gigabit Gigabit Gigabit 29 NetApp for Oracle Database

30 Type of Network NFS storage connectivity FC storage connectivity NFS storage connectivity for virtual environments Specification Gigabit Fibre 10GbE Figure 16) Network configuration of VMware environment. 4.6 Storage Infrastructure Details In the use case of heterogeneous environment data replication support, the production environment can run on either third-party or NetApp storage. If NetApp storage is used, the storage infrastructure is usually configured as an HA pair (also referred to as cluster failover) for extra protection. In the use case of homogeneous environment data replication support, production storage must be NetApp storage, and the database server s OS platform must be the same for both the production and the dev/test environments. The storage for the Oracle Database environment is NetApp storage, which can be in an HA or non-ha configuration. NetApp recommends using the HA configuration if the dev/test environment is also serving as the DR infrastructure for protecting the production environment. Table 7 lists the production and the dev/test storage required for data replication in both heterogeneous and homogeneous environments. Table 7) Storage. Production Storage Development and Test Storage Heterogeneous environment data replication Third-party or NetApp NetApp with deduplication and compression; HA configuration is optional 30 NetApp for Oracle Database

31 Production Storage Development and Test Storage Homogeneous environment data replication NetApp in HA configuration with SnapMirror NetApp in HA configuration with SnapMirror, deduplication, and compression 4.7 Virtual Infrastructure The infrastructure used to implement the solution consists of physical and virtual platforms based on both VMware and Oracle VM virtualization technologies. VM guests that are dedicated as database hosts have multiple database instances per guest. The database instances on these guests are primarily the clones of the DR site s database. Each guest supports databases residing on NFS-based storage and block-based storage (FC RDM LUNs). Table 8 lists the components of the virtual infrastructure. Table 8) Virtual infrastructure. Virtualization Platform Components in Virtual Environment Components in VM Guest VMware Oracle VM VMware vsphere ESX Server vcenter Server VMware vsphere Client NetApp VSC NetApp Host Utilities Kit for ESX Oracle VM Server Oracle VM Manager NetApp Host Utilities Kit for Linux Oracle: RDBMS ASM NetApp: SnapDrive for UNIX SnapManager for Oracle Snap Creator Host Utilities Kit for Linux Red Hat Enterprise Linux OS Oracle: RDBMS ASM NetApp: SnapDrive for UNIX SnapManager for Oracle Snap Creator Host Utilities Kit for Linux Red Hat Enterprise Linux OS 4.8 Solution Manageability The management framework in the solution is made up of management components from Oracle and VMware: Oracle Enterprise Manager Grid Control. An integrated solution that enables customers to manage Oracle technology components as well as some third-party products. Within the solution, Oracle Enterprise Manager Grid Control is used to manage and monitor Oracle Clusterware, ASM, databases, and OSs. Oracle VM Manager. Dedicated to managing and provisioning the Oracle VM environment. VMware vcenter and vclient. Constitute a comprehensive management solution for managing the entire VMware virtualization infrastructure. 31 NetApp for Oracle Database

32 4.9 Success Stories NetApp Snapshot Backups and SnapRestore Siemens IT Solutions and Services is a European-based premium player providing state-of-the-art IT solutions and delivering outsourcing services with global reach. Within the IT service chain, the portfolio of Siemens IT Solutions and Services ranges from consulting, software deployment, and system integration to the comprehensive management of IT infrastructures. Through profound industry know-how and beneficial innovations, the business technologists at Siemens IT Solutions and Services enable customers to transform their business processes on an ongoing basis. With a workforce of more than 32,000 employees, Siemens IT Solutions and Services achieved revenues of some 4.2 billion in fiscal 2010, which ended on September 30, Over 75% of these revenues came from outside of Siemens. The combination of granular Snapshot backup and shared volumes has its benefits. With the NetApp Snapshot copies, we can back up and restore quickly and reliably, save resources, and offer our customers optimum conditions. Dr. Thomas Hartl Manager Database Non-SAP, Siemens IT Solutions and Services 1 NetApp for Oracle Database Application Development and Test Vantage Media is one of the largest performance search marketing firms in the United States. Its online marketing offerings deliver qualified customer prospects on a performance basis, helping clients drive revenues and profits by identifying motivated consumers and increasing the rate at which these browsers become buyers. Constraints on our ability to create dev/test environments meant we had to reject some promising development ideas early on because we just didn t have the resources to explore them. With NetApp FlexClone, we don t have to make as many hard decisions: We can work on about six projects at once, twice as many as before. Sergey Sundukovskiy VP of Engineering, Vantage Media, LLC 2 5 Conclusion The NetApp for Oracle Database solution provides simple, fast, space-efficient backups and restores; dev/test processes; data protection and DR; and always-on operations. This solution accelerates deployment time at drastically lower costs. With access to database clones in a mixed infrastructure environment, development teams can engage in parallel processes and reduce dev/test cycle durations. This solution offers clear advantages in the areas of VM cloning and mass deployment, database cloning and data replication, and storage efficiency. It significantly improves the savings in time and cycles associated with the process of cloning VMs and databases and delivers significant savings in storage consumption when compared to traditional storage and cloning methods. The benefits of using the NetApp Oracle application dev/test solution are faster application deployments, increased revenues, and a competitive advantage over industry. 1 For more information on this success story, refer to to Siemens IT Solutions and Services Optimizes Backup for More Than 1,000 Databases with NetApp Snap Creator. 2 For more information on this success story, refer to Vantage Media Speeds Innovation While Keeping Systems Safe Using NetApp FlexClone. 32 NetApp for Oracle Database

33 The ability to manage multiple physical storage systems as a single storage entity is a true benefit of Data ONTAP Cluster-Mode. Any object or resource (such as disks, aggregates, volumes, networks, I/O ports, and services) within the storage cluster can be managed, administered, and monitored from a single point of management. Resources can be accessed and shared clusterwide. Secure virtual containers called Vservers consisting of storage resources can be created for each application, with volumes and data access isolated and restricted to the virtual containers. NetApp support for multiple protocols provides customers with the flexibility necessary for a dynamic environment. In the implementation planning stages, customers rely on NetApp to support multiple protocols and to provide data on how these protocols might perform in their environment in various situations (for example, failover or moving a volume). The NetApp for Oracle Database solution delivers true business value to organizations, regardless of their differing storage requirements. References Supporting Documents The following supporting documents were used for this document: NetApp Technical Report 3979, Oracle on NetApp Cluster-Mode Solution Guide : pdf&cc=us&wid= &mid= NetApp Technical Report 3932, Red Hat Enterprise Linux Protocol Performance Comparison with Oracle Database 11g Release 2 : NetApp Technical Report 3961, Oracle Database 11g Release 2 Performance Using Data ONTAP 8.1 Operating in Cluster-Mode : NetApp Technical Report 3749, NetApp Storage Best Practices for VMware vsphere : NetApp Data ONTAP 8.1 Software Setup Guide NetApp Data ONTAP 8.1 System Administration Guide NetApp Data ONTAP 8.1 Storage and Data Protection Guide NetApp Data ONTAP 8.1 Network and File Access Management Guide NetApp Snap Creator Installation and Administration Guide Version History Version Date Document Version History Version 1.0 November 2011 Initial release. Version 2.0 November 2012 Updated with new and advanced use cases. 33 NetApp for Oracle Database

34 Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exact product and feature versions described in this document are supported for your specific environment. The NetApp IMT defines the product components and versions that can be used to construct configurations that are supported by NetApp. Specific results depend on each customer's installation in accordance with published specifications. NetApp provides no representations or warranties regarding the accuracy, reliability, or serviceability of any information or recommendations provided in this publication, or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS, and the use of this information or the implementation of any recommendations or techniques herein is a customer s responsibility and depends on the customer s ability to evaluate and integrate them into the customer s operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consent of NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo, Go further, faster, Data ONTAP, DataMotion, FlexClone, FlexVol, Snap Creator, SnapDrive, SnapManager, SnapMirror, SnapRestore, Snapshot, and WAFL are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. Linux is a registered trademark of Linus Torvalds. 34 NetApp for Oracle Oracle Database is a registered trademark of Oracle Corporation. ESX, VMware, and VMware vsphere are registered trademarks and ESXi, vcenter, and vclient are trademarks of VMware, Inc. Windows and Windows Server are registered trademarks of Microsoft Corporation. Intel is a registered trademark of Intel Corporation. UNIX is a registered trademark of The Open Group. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such. NVA