NEC Storage NV Series

Transcription

1 NEC Storage NV Series Guide to Oracle Storage Compatibility Program Network Attached File Server Snapshot Technologies is-wp Rev-1.00 February, 2005 NEC Corporation

2 Copyright 2005 NEC Corporation. All rights reserved. DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT ARE DISCLAIMED. NEC Storage NV Series and NEC Storage Manager/IP are trademarks or registered trademarks of NEC Corporation in Japan. Oracle, Oracle9i and other product names of Oracle are trademarks, registered trademarks of Oracle Corporation in the U.S. and other countries. Sun, Sun Microsystems, Sun logos, and Solaris are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. Microsoft and Windows are registered trademarks of Microsoft Corporation in the U.S. and other countries. All other brands or products are trademarks or registered trademarks of their respective holders are should be treated as such. Rev Feb

3 Contents 1. Introduction Glossary Overview of NAS What is NAS? NEC Storage NV Series Snapshots on NEC Storage NV Series What is Snapshot Function? Mechanism of the Snapshot Function Snapshot Area Management Snapshot Operation System configuration for OSCP Hardware Configuration Software Configuration Summary of OSCP Test Backup and Restore Startup of a replicated database Details of the verification tests based on the OSCP Test Database Construction for Verification and Snapshot Setting Database Construction for Verification Snapshot setting Backup and Restore Cold backup Hot backup Startup of a replicated database Starting up a replicated database using cold backup processing Starting up a replicated database using hot backup processing Reference...28 Acknowledgement

4 1. Introduction Recently, as the demand for such tasks as data sharing in multi-os environments and cost cuts in storage management has risen, the use of NAS (Network Attached Storage), the storage appliance product, has increased. Moreover, as the Internet moves to broadband and the number of services delivering high-volume content grows, the demand for a large-capacity, efficient, high-end NAS product is also expanding. NEC Storage* NV Series is a high-end NAS product providing high performance, high availability and high reliability and adopts the latest architecture features, such as a high performance TOE (TCP/IP Offload Engine), an OS specifically optimized for NAS, large capacity non-volatile memory (NVRAM), a remote replication capability, Snapshot function and more. Based on the Oracle Storage Compatibility Program (OSCP), Oracle provides snapshot tests guideline to verity a file server to be used for Oracle database. This document describes how to create snapshots for backup and how to use the snapshots for recovery for the Oracle database by using the OSCP test kit provided Oracle Corporation and the snapshot function of the NEC Storage NV Series, and also shows the verification test results. The verification was performed at NEC Corporation by applying the verification kit Snapshot test kit(stk) provided in the White Paper, 'Guidelines for Using Snapshot Storage System for Oracle Database' published by Oracle Corporation and Oracle Storage Compatibility Program (OSCP). * NEC Storage is a series name of disk arrays from NEC Corporation for a world-wide market. For a Japanese market, istorage is applied for a series name such as istorage NV Series. 2. Glossary CIFS A file-sharing protocol on Microsoft Windows It is used primarily in Windows environments. However, its use on various UNIX OSs is increasing with the spread in the use of the Windows platform in recent years. Cold Backup Creating a backup copy of the production data while the production system is suspended state Hot Backup. File Lock A mechanism that shows that a file is used by a specific process to avoid accidents such as double update. A NAS Device, in other words the NFS server, should maintain file lock information regardless of the - 4 -

5 state of the client. For example, if an NFS server reboots when a client locks a certain file, the state of the lock must be maintained after the reboot. The Oracle RDBMS uses the file lock mechanism to avoid, for example, starting the same instance twice. The NEC Storage NV Series is compliant with the RPC lockd mechanism commonly used on NFS. Snapshot Function to save the image of a file system at arbitrary time Consistent status of a file system can be saved per file system. LVM Logical volume Manager Logical Disk Manager controls multiple logical disks made by the Disk Management function of the NEC Storage NV series as logical volumes. Hard Mount (see also Soft Mount) One of the mount options for NFS. A mode in which the client keeps I/O processing without breaking off, and waits for the file access system to recover, even if an error is caused during a file access. Hot Backup Creating a backup copy of the production data while the production system is operating iscsi (see also NAS, SAN) One of the methods for connecting to a storage device over a commonly used network, such as LAN or Ethernet. FCIP and ifcp, and other methods which are similar to this, also exist. Unlike NAS, iscsi is a technology that passes the device level SCSI protocol through the TCP/IP layer, and makes it possible to avoid the cost and management load for a special communication mechanism required by usual Fibre-Channel SAN. Link Aggregation Technology that improves bandwidth and fault tolerance by bundling two or more network lines (trunks) together and treating them just like one line. There is a standard based on IEEE803.2ad, and other individual standards used by network equipment manufacturers. The NEC Storage NV Series is compliant with IEEE802.3ad (LACP: Link Aggregation Control Protocol). NAS (see also iscsi, SAN) Network Attached Storage A storage device connected over a commonly used network such as LAN or Ethernet that uses NFS and/or CIFS as the common file protocol on the application layer. The necessary client program is mostly embedded in the OS by default, and its most significant feature is that no specialized program or driver is required to use the storage device. To accommodate demands for recent highly - 5 -

6 sophisticated applications, some devices are also capable of handling other protocols such as WebDAV, HTTP or FTP. Note that sometimes even outside the field of data storage, in discussions of networks, Network Access Server is known by the abbreviation NAS. NFS A common file protocol using TCP/IP defined by Sun Microsystems As the specification is openly available, it is widely used on anything from the various UNIX platforms to the Windows system. NTP Network Time Protocol The time synchronization protocol used on LANs or the Internet NVRAM Non-Volatile Random Access Memory A generic name of memory that does not require an external power supply to retain information. Usually it is designed by using Static RAM or Flash ROM, etc. However, there are some defects with this approach such as difficulties in design, a high cost for large capacity (in the case of S-RAM) or low access speed (F-ROM). SAN (see also iscsi, NAS) Storage Area Network A technology that enables sharing from two or more host computers by treating storage as independent from a host computer when the storage capacity increases. Fibre-Channel is used for the connection. The host computer needs a connection and management mechanism to use Fibre-Channel. Soft Mount (see also Hard Mount) One of the mount options for NFS. A mode in which the client stops I/O processing if an error is caused during a file access. Stale Cache A state where data registered in a cache is older than what it replaces on the hard disk in the storage device. As all of NEC Storage Series have advanced cache management mechanisms, a stale cache state is usually impossible. Synchronous Writes A writing mode that guarantees a write has been completed when the storage device reports that the - 6 -

7 write is complete. Conventionally, a storage device may execute a write to a disk with a time delay (Asynchronous Write) after reporting to a client that the write is complete, and receiving data into the main memory of the storage device (cache memory), in order to improve the apparent response performance. In such a procedure, if any trouble occurs after the recognition of write on the client side and before an actual write of data on the disk as safely on the storage device, a data loss which the client cannot be aware of would be generated. In contrast, Synchronous Write enables the host to detect writes or losses of data at the time they happen by waiting to announce the completion of the write operation to a host side until the data is actually safely on the disk. The result is that reliability is improved in exchange for performance. TOE TCP/IP Offload Engine A technology that provides high-speed TCP/IP processing and reduces the load on the OS by shifting the TCP/IP process out of the network processes of each layer, which the OS is usually responsible for, to the network interface and processing it with a hardware engine. Some TOEs can support a higher-level application protocol. This type of technology specialized for a specific protocol (for example, iscsi) is sometimes called a Network Accelerator

8 3. Overview of NAS 3.1 What is NAS? NAS (Network Attached Storage) is one form of a storage appliance that specializes in file server functions. Its features are described below Ease of Management Conventionally, a file server is provided by configuring an external or internal storage such as a disk array coupled with a general-purpose host computer or server. Along with increasing storage capacity, networks known as SAN (Storage Area Networks) have become very popular. However, most of these systems need a special data transfer network using fiber channel or an independent management system, which causes an increase in management load. As NAS utilizes the commonly used Ethernet and common file protocols based on TCP/IP, which operates on the Ethernet, the management load can be greatly decreased. What is more, many engineers are familiar with Ethernet because it is a popular technology Storage Consolidation The file server described above naturally consolidates storage. That is, rather than each client having its own independent storage, many clients use the storage space concentrated on one site. The NAS system, which is specifically designed for this, has the following advantages compared with a file server configured with general-purpose servers and storage: - Disks can be added while the system is on line and in use - The ability to provide file services to many clients Multi-OS Utilization When using SAN for storage consolidation, common file processing and data sharing is difficult if the client OS differs, as the file access process is executed on the client. With the NAS system, the NAS device controls the file system and data is offered to the client in the form of the file system. What is more, a common file process is available with a high degree of compatibility even if the OSs on clients are different, as de-facto standards such as NFS and CIFS are used as the access protocol to files. 3.2 NEC Storage NV Series The NEC Storage NV Series has the following features in addition to the usual features of NAS High Performance TOE (TCP/IP Offload Engine) Conventionally, the OS deals with TCP/IP and higher-level processes using resources such as CPU and memory. However, due to improvements in the transmission rate brought about by the development of the Ethernet technology, the situation now is that the proportion of communication processing required of the CPU has increased and application processes have become unavailable because of the heavy communication processing load. The TOE technology is used to decrease the CPU load of the server and to achieve high performance and wide scalability by shifting the heavy load of TCP/IP processing from the software to the hardware. A technology specially designed for a specific application protocol (for example, NIC for iscsi) is - 8 -

9 sometimes called a Network Accelerator. The NEC Storage NV Series is available to use two or more network interface cards each with a TOE chip on board and two Gigabit Ethernet ports. When a network interface card is added, the load for the increased network interface is managed by the TOE chip on the network interface card, which helps the network interface to reduce the load on the OS operation of the NAS device itself and brings about a high degree of scalability An Operating System highly optimized for NAS To achieve a high-functionality, high-performance, high-scalability and high-availability NAS solution, the NEC Storage NV Series has highly optimized underlying software based on Linux High performance RAID Control LSI The NEC Storage NV Series has achieved a high speed data transfer by applying high performance RAID processing LSI already proven in use in NEC Storage S-series disk arrays and Fibre-Channel HDD communicating at 2G bps on board Large capacity Non-Volatile RAM (NVRAM) The NEC Storage NV Series makes use of large capacity non-volatile random access memory for its main memory, which greatly improves the performance compared with the standard speed of I/O response for writes to a disk. In addition, by configuring the batteries to use redundancy, a data loss in the memory can be prevented even if trouble occurs in one place Clusters The NEC Storage NV Series has a model which can be used with a two-node cluster configuration that operates in Active-Active mode. (For these tests a single configuration model was used.) Redundancy The NEC Storage NV Series has redundancy built into its main parts such as the power supply, battery, fan and internal disk access paths. In addition, clustering models in the NEC Storage NV Series also have server-level redundancy Data Recovery With the NEC Storage NV Series it is possible to make a snapshot for every file system available. A file that a user overwrote or deleted by mistake can be recovered by a simple operation from the client Optional Functions The NEC Storage NV Series can provide the following functions in addition to the above-mentioned basic functions: - Remote replication - Backup - Anti-virus functionality - 9 -

10 4. Snapshots on NEC Storage NV Series An overview of the Snapshot functions provided on the NEC Storage NV Series NAS Device is described. 4.1 What is Snapshot Function? The Snapshot function saves the image of a file system at arbitrary time. When an administrator creates a snapshot generation, the image of the file system at creation of the generation is saved. This feature enables reference to data before update processing, backup operation in online state, and data recovery by general users themselves. 4.2 Mechanism of the Snapshot Function After creating a generation, the snapshot function works for the following when the file is updated: A new area is allocated for the data to be updated, and the updated data is written into the area. The data area before update processing in not updated, and the data is retained as a snapshot for reference. The data area is called Snapshot area. The newly allocated area is updated until the next generation is created. As explained above, old data and new data are saved in different areas. The old data area is shared until being updated, and the area occupied by the image of a generation is saved as a snapshot area and not released until the generation is deleted. Therefore, the file system area is incremented by the amount of updated data from the point of creating the generation. 4.3 Snapshot Area Management Immediately after a generation is created, there is no area occupied by only snapshot generations. area, which is occupied by only snapshot generation, is increased by only the size of an updated area each time data is updated from the point of creating a generation. An If snapshot generations are created while frequently updated data is stored in a file system, the difference between the generation data and the latest data becomes large and the number of areas occupied by snapshot generations increases. The following functions are provided as snapshot area management functions for overall file system via NEC Storage Manager IP: Function to set a limit for areas occupied by only snapshot generations Function to refer to the limit and utilization When the number of areas occupied by only generations has reached the limit, a new generation cannot be created any more. The number of areas occupied by only snapshot generations keeps increasing each time data is updated. Delete generations created by the snapshot function when they are no longer necessary

11 4.4 Snapshot Operation Snapshot commands used via ssh/rsh are as follows in briefly. Regarding setting information and detailed usage for each command, please refer to the related manuals. snap Operate snapshot generations of logical volumes snap create Create snapshot generation snap copyback Recovery data by copying from snapshot generation snap delete Delete snapshot generation snap list Display snapshot generation listing 5. System configuration for OSCP Hardware and software configurations applied for the verification tests on snapshot processing are shown here. 5.1 Hardware Configuration Primary Oracle host/secondary Oracle host Host : Sun Fire 280R(1CPU,1024MB memory,36.4gb x 2 local disk) Storage : NEC Storage NV8210 NAS management host Host : NEC Express5800/100(IA server) 5.2 Software Configuration Primary Oracle host/secondary Oracle host OS : Solaris 9 04/02 Database : Oracle9iR2( ) Storage : SC-LX OS Version NAS management host OS : Microsoft Windows 2000 server SP4-11 -

12 6. Summary of OSCP Test Summaries of several verification tests results to utilize the backup data and to startup the replicated database on the OSCP test are shown here. 6.1 Backup and Restore At first, create the snapshot generation of data file/table space, then copy back the backup data and finally check the Oracle database operates normally. Remote snapshot function is not subject to the test this time. Location Backup Method Result local cold hot remote cold out of validation hot out of validation Table 6-1 : Backup and Restore 6.2 Startup of a replicated database At first, create the backup of necessary files on snapshot generation, then copy the backup data from snapshot generation to logical volumes under the other server on which the replicated database runs. Startup Oracle database from the server and finally check the Oracle database operates normally. Backup Method cold backup hot backup Result Table 6-2 : Startup of a replicated database

13 7. Details of the verification tests based on the OSCP Test In this chapter, details of the verification tests, its system configuration, the operation and some notes are described for each case of the table shown in the previous chapter with picking up an effective and typical combination of variations in the practical use. Also some extensions for each case are shown. 7.1 Database Construction for Verification and Snapshot Setting Database Construction for Verification Installation of Oracle Database We installed the Oracle database according to the Oracle installation guide. In general, the values of parameters are as recommended, except that we modified some parameters. Note that the same SID is used for verification of replicated database creation by dividing the execution server of the replicated database. (That is, we have not verified for the replicated database with another SID of the same host.) Snapshot setting In order to use the snapshot function of NEC Storage NV Series NAS Device, a user name on the NAS Device for rsh command execution from an operational server should be specified in advance. There are no settings for the operational server to execute rsh commands to the NAS Device. 7.2 Backup and Restore The procedures for using the snapshot function on the NEC Storage NV Series NAS Device to back up the Oracle database are described. Additional enhancement method such that a snapshot generation is saved on a tape should be taken in actual operation. Of course, there is no problem to use the snapshot generation as a backup media. In this validation test case, only one snapshot generation is used because the purpose is to check Oracle Database runs normally using the snapshot function

14 System Configuration sample Access from cx206 server to the NAS Device via NFS mount cx206 Primary Oracle host (SunFire 280R) Access via NFS NV LVM01 LVM02 LVM03 LVM04 cx206 can use these volumes as normal operation or backup/recovery operation Snapshot function At the snapshot creation, the difference data between the generation data and the latest data are saved in the same volume. The following is LVM01 case. Normal operation files LVM01 datafile rbs datafile datafile datafile rbs datafile datafile Create a generation as file images in the same volume by using snapshot function

15 Detailed volume usage LD name usage mount point LVM01 datafile/rbs/temp/system /export/oradata10 LVM02 control 1/Online REDO1 /export/oradata11 LVM03 control 2/Online REDO2 /export/oradata12 LVM04 init<sid>.ora/archived log /export/oradata Cold backup This chapter explains the procedures for applying the snapshot function to perform cold backup processing. The following descriptions are for the cases assuming to apply snapshot function to backup data file located in a NFS mount volume. Other types of operations are also described as an extension. 1) Command operation The following shows operation commands (create, delete, copy) to logical volumes in which data files exist for cold backup and their results. 1-1) Creating generation Create a generation against a logical volume subject to snapshot cx206:oracle > rsh -l cli nv snap create /export/vol_euc/lvm01 snap snap 2004/08/12 17:39:54 created ok. 1-2) Deleting generation Delete an unnecessary generation cx206:oracle > rsh -l cli nv snap delete /export/vol_euc/lvm01 snap sync snap 2004/08/12 16:44:54 removed ok. 1-3) Recovering data from generation Recover data from a generation which is saved in the same logical volume cx206:oracle > rsh -l cli nv snap copyback /export/vol_euc/lvm01 snap before_copyback 2004/08/12 17:43:51 created ok. 2) Creation of cold backup data

16 2-1) Stopping the Oracle database Cold backup is a backup solution while the database is not operating. Thus, stop the Oracle database. Issue the command from SQL*Plus. cx206:oracle > sqlplus /as sysdba SQL> shutdown immediate; SQL> startup restrict pfile=/export/oradata13/initsnap1.ora ; SQL> shutdown normal; SQL> exit Command output image is omitted here. not shown.) (Similarly, it is omitted in the subsequent explanation if a note is The Oracle database has stopped when the SQL*Plus prompt is displayed after issuing the command. 2-2) Creating generation For cold backup data, create a generation of a logical volume in which a data file exists. At this time, backup data can be saved to a tape device. a) Delete an unnecessary generation if any cx206:oracle > rsh -l cli nv snap delete /export/vol_euc/lvm01 snap sync snap 2004/08/12 16:44:54 removed ok. b) Create a generation. The generation can be saved to a tape device. cx206:oracle > rsh -l cli nv snap create /export/vol_euc/lvm01 snap snap 2004/08/12 17:39:54 created ok. 2-3) Restarting the Oracle database Place the database in normal operation status. cx206:oracle > sqlplus /as sysdba SQL> startup open pfile= /export/oradata13/initsnap1.ora SQL> exit Issue the command from SQL*Plus. 3) Use of cold backup data Because this is a recovery of database, the original database is in halted state. 3-1) Recovering data from a generation a) Delete the unnecessary generation named before_copyback if copy-back was executed once cx206:oracle > rsh -l cli nv snap delete /export/vol_euc/lvm01 before_copyback sync before_copyback 2004/08/11 17:03:45 removed ok

17 b) Copy a generation back and recover data cx206:oracle > rsh -l cli nv snap copyback /export/vol_euc/lvm01 snap creating "before_copyback" for safety before_copyback 2004/08/12 17:43:51 created ok. 3-2) Recovering and restarting the Oracle database By the operation in step 3-1, the contents of the volume containing the data file for the Oracle database are recovered to the backed-up point. To start the Oracle database, just apply the Online redo log and archived log in another volume to this data file. Outline of the procedure for SQL operation a) Start the database without opening. b) Recover the database. (Apply the logs.) c) Open the database. Example in the verification test cx206:oracle > sqlplus /as sysdba SQL> startup open pfile= /export/oradata13/initsnap1.ora ; --- a) SQL> recover automatic database ; --- b) SQL> alter database open ; --- c) SQL> exit In this verification test, because both Online redo log and Archived log are set to be securely stored in another volume, the database could be restored with minimum operation. Extension 1) Cold backup and restore for the whole database In addition to the above example, the following shows notes on using the snapshot function to back up all the related files in the database as cold backup: Files to be backed up are: Data file Online Redo logs Control files Archived logs The advantages to apply the snapshot function compared with the case for applying general copy tool are as follows: Less load to the host computer because snapshot is executed within NAS devices High speed

18 The procedure for creating cold backup data is the same as the one for only the data file. Since the Oracle database is stopped, consistency among data file, Control files, Online Redo logs, Archived logs is maintained. Note that database restore for not only data files but the control files, Online Redo logs and archived logs means that the data is rolled back to the point when the backup data was created. Recovery processing is not needed for restarting the restored database. point when it stopped for backup data creation.) (It is the same as to restart from the Hot backup 1) Command operation The following shows operation commands (create, delete, copy) to logical volumes in which data files exist for hot backup and their results. 1-1) Creating generation Create a generation against a logical volume subject to snapshot cx206:oracle > rsh -l cli nv snap create /export/vol_euc/lvm01 snap snap 2004/08/12 17:39:54 created ok. 1-2) Deleting generation Delete an unnecessary generation cx206:oracle > rsh -l cli nv snap delete /export/vol_euc/lvm01 snap sync snap 2004/08/12 16:44:54 removed ok. 1-3) Recovering data from generation Recover data from a generation which is saved in the same logical volume cx206:oracle > rsh -l cli nv snap copyback /export/vol_euc/lvm01 snap before_copyback 2004/08/12 17:43:51 created ok. 2) Creation of hot backup data 2-1) Beginning the hot backup mode Place each table space of the operating database in hot backup mode. SQL*Plus. Issue the command from

19 cx206:oracle > sqlplus /as sysdba SQL> alter tablespace rbs begin backup; SQL> alter tablespace temp begin backup; SQL> alter tablespace system begin backup; SQL> alter tablespace snapdata begin backup; SQL> exit 2-2) Creating generation For hot backup data, create a generation of a logical volume in which a data file exists. At this time, backup data can be saved to a tape device. a) Delete an unnecessary generation if any cx206:oracle > rsh -l cli nv snap delete /export/vol_euc/lvm01 snap sync snap 2004/08/12 17:53:26 removed ok. b) Create a generation. The generation can be saved to a tape device. cx206:oracle > rsh -l cli nv snap create /export/vol_euc/lvm01 snap snap 2004/08/12 18:12:38 created ok. 2-3) Ending the hot backup mode Displace each table space of the operating database from hot backup mode. Issue the command from SQL*Plus. Issue the command from SQL*Plus. If the snapshot function is applied, the hot backup mode need not to be continued until completion of backup processing for tapes, which makes it possible to minimize the time for hot backup mode. Issue the command from SQL*Plus. cx206:oracle > sqlplus /as sysdba SQL> alter tablespace snapdata end backup; SQL> alter tablespace system end backup; SQL> alter tablespace rbs end backup; SQL> alter tablespace temp end backup; SQL> exit That is, influence on production would be minimized. Be sure to perform end backup processing for all the table spaces that were placed in hot backup. 3) Use of hot backup data Because this is a recovery of database, the original database is in halted state. 3-1) Recovering data from generation a) Delete the unnecessary generation named before_copyback if copy-back was executed once

20 cx206:oracle > rsh -l cli nv snap delete /export/vol_euc/lvm01 before_copyback sync before_copyback 2004/08/12 17:59:30 removed ok. b) Copy a generation back and recover data cx206:oracle > rsh -l cli nv snap copyback /export/vol_euc/lvm01 snap before_copyback 2004/08/12 18:19:01 created ok. 3-2) Recovering and restarting the Oracle database By the operation in step 3-1, the contents of the volume containing the data file for the Oracle database are recovered to the backed-up point. To start the Oracle database, just apply the Online redo log and archived log in another volume to this data file. Outline of the procedure for SQL operation a) Start the database without opening. b) Recover the database. (Apply the logs.) c) Open the database. cx206:oracle > sqlplus /as sysdba SQL> startup open pfile= /export/oradata13/initsnap1.ora ; --- a) SQL> recover automatic database ; --- b) SQL> alter database open ; --- c) SQL> exit In this verification test, because both Online redo log and Archived log are set to be securely stored in another volume, the database could be restored with minimum operation Startup of a replicated database Startup of a replicated database by using the snapshot data is available as one of the applications for a replicated database. A replicated database is used for the followings: Divide of production (such as load balancing by assigning the replicated database as a search-only database) Simulation on the replicated database Startup of a backup database Procedure to startup a replicated database is as follows: 1) Creating backup data 1) Stop database operation