Administration: Backup, Restore, and Data Recovery. SAP Sybase IQ 16.0 SP03

Transcription

1 Administration: Backup, Restore, and Data Recovery SAP Sybase IQ 16.0 SP03

2 DOCUMENT ID: DC LAST REVISED: December 2013 Copyright 2013 by SAP AG or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries. Please see index.epx#trademark for additional trademark information and notices.

3 Contents Data Backup and Recovery...1 Database Backup...1 Database Validation...3 Performance Options...4 Archive Devices...6 Tape Backups...6 Disk Backups...9 Read-Only Hardware...10 Third-Party Products...12 Queries, Utilities, and Procedures...13 Backup Scenarios...15 Routine Backups...16 Cache Dbspace Backup...17 System-Level Backups...18 Virtual Backups...20 Previous Backups...22 Backup Log...24 Restoring Database Backups...27 Database Restore...27 Verifying Database Backups...30 Displaying Header Information...32 Restoring to Raw Devices...33 Restoring Cache Dbspaces...34 Moving Database Files...35 Restoring Multiplex Stores...37 Restoring Multiplex Stores to a Different Location...37 Restoring Multiplex Stores to the Same Location...39 Restoring Read-only Backups for a Coordinator...41 Administration: Backup, Restore, and Data Recovery iii

4 Contents Error Recovery...41 System Recovery and Database Repair...43 Recovery and Repair Overview...43 Normal Recovery...43 Database Verification...44 The sp_iqcheckdb Stored Procedure...44 sp_iqcheckdb Output...47 Resource Issues Running sp_iqcheckdb...50 Database Repair...50 Analysis of Index Errors...51 Index Error Repair...54 Analysis of Allocation Problems...54 Repairing Allocation Problems using DBCC...56 Forced Recovery Mode...58 Before Forced Recovery...58 Starting a Server in Forced Recovery Mode...58 Recovering Leaked Space...59 Recovering Multiplex Databases...60 Problems Reported by DBCC...61 Index Problems that DBCC Cannot Repair...61 Dropping Inconsistent Indexes, Tables, or Columns...62 DBCC Error Messages...63 Backup Reference...67 BACKUP DATABASE Statement...67 RESTORE DATABASE Statement...73 sp_iqcheckdb Procedure...80 Index...89 iv SAP Sybase IQ

5 Data Backup and Recovery Data Backup and Recovery Regular and frequent backups are your only protection against database or device failure. Consider the performance implications of various backup options, and create an appropriate backup plan. Backup plans depend on system load factors, database size, number of updates, and the relative importance of backup and recovery times. The length of time your organization can function without access to the data in your database determines the maximum recovery time. Run a full backup on a new database to provide a base point, then perform full and incremental backups on a fixed schedule. Consider time and storage requirements. Balance the time it takes to create the backup with the time it takes to restore the data. Database Backup The BACKUP DATABASE command backs up an SAP Sybase IQ database to one or more archive devices. Basic syntax identifies the type of backup you want to perform, and a TO clause that directs the output to an archive device: BACKUP DATABASE [ backup-option ] TO archive_device You must be connected to the database to back it up. You cannot use the BACKUP DATABASE command to specify another database. In a multiplex environment, execute all BACKUP DATABASE and RESTORE DATABASE commands on the coordinator. Backup Data BACKUP DATABASE issues a CHECKPOINT before starting the backup, then backs up the catalog store. Data that is not committed when this initial checkpoint occurs is not included in the backup. A second checkpoint occurs at the end of backup. Data that is committed while the backup is in progress is included in subsequent backups. Backups run concurrently with most other read-write operations, except those that affect the structure of the database. You cannot issue a CHECKPOINT or change the database metadata while a backup is in progress. If a system or media failure occurs during backup, you cannot restore uncommitted transactions. Make a copy of the params.cfg file and save the contents of the SYSDBFILE and SYSDBSPACE system views. SAP Sybase IQ does not back up the temporary store (dbname.iqtmp) and params.cfg, but does back up the metadata and other information that is necessary for re-creating the temporary store structure. Administration: Backup, Restore, and Data Recovery 1

6 Data Backup and Recovery Data Distribution BACKUP DATABASE always makes a full backup of the catalog store on the first archive device, and then backs up the data from the IQ store in parallel across all of the devices you specify. Blocks are not evenly distributed across archive media. Depending on the processing speed of individual threads, you may have more blocks on one device than others. SAP Sybase IQ backs up only those recoverable database blocks that are actually in use at the time of the backup. Free blocks are not backed up. Sets of files must be restored in the order in which they were backed up. Backup Options Dbspaces and dbfiles can be read-only (RO), read-write (RW), online, or offline. You can restrict FULL, INCREMENTAL-SINCE-FULL, or INCREMENTAL backups to the read-write files in the IQ main store (db-name.iq). The backed up files are selected when the backup command checks the read-write status in the catalog. A backup can back up a set of read-only dbspaces and read-only files. The read-only dbspaces or files must belong to the IQ main store. The backed up files are user selected. If you use symbolic links for raw device names, as recommended, make sure the system backup utility follows the symbolic link and backs up the device. Device Limits Keep backup commands small. Large numbers of devices increase I/O and hardware contention. To saturate CPU usage, use roughly 1 device per core; on faster systems, use up to 2 devices per core. Use 36 or fewer TO clauses. Failure and Recovery If a backup fails during the initial or final CHECKPOINT, normal CHECKPOINT recovery occurs. If a backup fails between the initial or final CHECKPOINT, the backup is rolled back. If the system fails between the initial and final CHECKPOINTS, use an older backup to restore the database. If the system fails during the final CHECKPOINT after a full backup, restore the database from the backup you just created. Note: See Reference: Statements and Options > SQL Statements > BACKUP DATABASE for parameter descriptions, usage, and required permissions. See also Database Validation on page 3 Performance Options on page 4 Archive Devices on page 6 Queries, Utilities, and Procedures on page 13 2 SAP Sybase IQ

7 Data Backup and Recovery Backup Scenarios on page 15 Previous Backups on page 22 Backup Log on page 24 Database Validation Although BACKUP DATABASE ensures that the database is in a usable state, RESTORE DATABASE does not check for inconsistencies in restored data. Validate the database before a backup to ensure that the database you restore is stable. The sp_iqcheckdb stored procedure is the interface to the database consistency checker (DBCC), which performs database verification. DBCC has different verification modes that perform increasing degrees of consistency checking. Run sp_iqcheckdb before a backup, or whenever you suspect a problem with the database. On multiplex, run sp_iqcheckdbonly on a write server. Set the database DBCC_LOG_PROGRESS option to write progress messages to the message file as sp_iqcheckdb executes. Table 1. DBCC Verification Modes Verification Mode Check Database Mode Description Performs an internal consistency check on all IQ indexes and checks that each database block has been allocated correctly. Syntax: sp_iqcheckdb 'check database' Verify Database Mode Reads all data pages and can detect all types of allocation problems and all types of index inconsistencies. Syntax: sp_iqcheckdb 'verify database' Database Allocation Mode Checks that each database block is allocated correctly according to the internal physical page mapping structures. Syntax: sp_iqcheckdb 'allocation database' See also Database Backup on page 1 Performance Options on page 4 Archive Devices on page 6 Queries, Utilities, and Procedures on page 13 Administration: Backup, Restore, and Data Recovery 3

8 Data Backup and Recovery Backup Scenarios on page 15 Previous Backups on page 22 Backup Log on page 24 Performance Options BLOCK FACTOR specifies the number of blocks to write to the archive device at one time. This parameter also controls the amount of memory used for buffers during the backup, and has a direct impact on backup performance. The effects of the block factor are a function of disk subsystem speed, tape speed, and processor speed. Performance also depends on your operating system and on the block size specified when the database was created. The default SAP Sybase IQ page size of 128KB for newly created databases results in a default block size of 8192 bytes. Platform Table 2. BLOCK FACTOR Settings by Platform UNIX-like operating systems Windows Set BLOCK FACTOR to at least 25 (default). With this setting, BACKUP can buffer data for most tape drives, with enough data in memory that drives are kept busy constantly throughout the backup. On AIX, use the System Management Interface Tool (SMIT) to display and change the current block mode of any tape device that you plan to use for backups. On Windows, the default BLOCK FACTOR is computed based on the database block size. The default usually achieves maximum throughput on Windows. Because of the way Windows handles tape devices, increasing the BLOCK FACTOR may not lead to faster backups. See your platform operating system documentation for information about your platform s optimal I/O size and block factor. Database Verification Concurrency Issues During database verification, sp_iqcheckdb reads every database page in use, which consumes most of the database server's time. I/O remains as efficient as possible, but other concurrent activities may run more slowly than usual. To limit DBCC CPU use, set the resources resource-percent parameter, which controls the number of threads with respect to the number of CPUs. If resource-percent = 100 (the default value), there is one thread per CPU. If resource-percent > 100, then there are more threads than CPUs, which might increase performance for some machine configurations. 4 SAP Sybase IQ

9 Data Backup and Recovery Error Checking Setting the CRC parameter to OFF deactivates 32-bit cyclical redundancy checking on a perblock basis, which improves the speed of backup and restore operations. With CRC set to ON (default), the checksums computed on backup are verified during any subsequent RESTORE operation, affecting the performance of both commands. If you turn off this checking, remember that you sacrifice a greater assurance of accurate data in exchange for faster performance. Comments The WITH COMMENT parameter lets you specifies a string as long as 32KB as part of the header information for the backup archive. If you omit this option, BACKUP enters a NULL. You can view the comment string by executing a RESTORE DATABASE... FROM... CATALOG ONLY or by displaying the backup log, backup.syb. Catalog Store Size BACKUP makes a full backup of the catalog store at the start of every backup, both full and incremental. Ordinarily, the catalog store is quite small, containing only the system tables, metadata, and other information that SAP Sybase IQ needs to manage your database. However, you can create non-iq tables in the catalog store. To improve performance keep any non-sap Sybase IQ data in a separate SAP Sybase SQL Anywhere -only database, rather than in the catalog store. BACKUP copies only the latest committed version of the database. Other version pages used by open transactions are not backed up. Spooling Backup Data You may find that it is faster and more efficient to create backups on disk, and then spool them onto tape for archival storage. If you choose this approach, unspool the data onto disk before restoring it. See also Database Backup on page 1 Database Validation on page 3 Archive Devices on page 6 Queries, Utilities, and Procedures on page 13 Backup Scenarios on page 15 Previous Backups on page 22 Backup Log on page 24 Administration: Backup, Restore, and Data Recovery 5

10 Data Backup and Recovery Archive Devices Specific information about writing to different archive devices, including tape, disk, read-only hardware, and third-party products. See also Database Backup on page 1 Database Validation on page 3 Performance Options on page 4 Queries, Utilities, and Procedures on page 13 Backup Scenarios on page 15 Previous Backups on page 22 Backup Log on page 24 Tape Backups A tape set consists of one or more backup tapes produced on a given archive device. The first tape set you specify must be large enough to hold the full backup of the catalog store, including any non-sap Sybase IQ data in the catalog store. Digital Linear Tape (DLT) is the preferred magnetic tape data storage medium. If it is supported for your platform, use DLT for tape backups. BACKUP can also support 4mm and 8mm Digital Data Storage (DDS) tape drives, and stacker devices that support multiple tape devices. BACKUP does not support jukeboxes or robotic loaders. Use a third-party media manager for these items. Large Database Backups Use multiple tape drives for large database backups. The TO clause identifies each tape drive device. The SIZE option sets the data capacity for each tape. The STACKER option indicates a backup to a multi-tape stacker device, and the number of tapes in the device. All tapes in a given stacker device must be the same size. Tape Devices on UNIX-Like Operating Systems BACKUP does not support fixed-length block-mode tape devices on any UNIX-like OS. To check the device block-size status, run this shell command: mt -f <tape device> status If the command returns a block-size greater than 0, the device is set for fixed-length blocks. To change the tape drive to variable-length blocks, use this or other appropriate OS shell command: mt -f <tape device> defblksize 0 6 SAP Sybase IQ

11 Data Backup and Recovery Table 3. Non-rewinding Tape Device Commands Platform Solaris AIX Description Insert the letter n for no rewind after the device name: /dev/rmt/0n Set the numeric value that follows the logical name of the tape drive for an no rewind setting: /dev/rmt0.1 See the AIX documentation for specific settings. HP-UX Use 0m to specify the default tape mechanism and n for no rewind: /dev/rmt/0mn Note: See your platform documentation for specific instructions and options. Tape Devices on Windows Windows does not support rewind or no rewind devices. SAP Sybase IQ requires variablelength devices, but does perform additional processing to accommodate fixed-length tape I/O on Windows. Because SAP Sybase IQ does not support tape partitioning, do not use another application to format tapes for BACKUP or RESTORE operations. On Windows, the first tape device is \ \.\tape0, the second is \\.\tape1, and so on. Note: SAP Sybase IQ treats the leading backslash in a string as an escape character, when the backslash precedes an n, an x, or another backslash. When you specify backup tape devices, double each backslash required by the Windows naming conventions. To indicate the first tape device, use \\\\.\\tape0, \\\\.\\tape1 for the second device, and so on. If you omit the extra backslashes, or otherwise misspell a tape device name, and write a name that is not a valid tape device on your system, SAP Sybase IQ interprets this name as a disk file name. Tape Backup Size Use the SIZE parameter to set the data capacity for each tape. The SIZE value can be less than or greater than the default tape size. If you omit the SIZE parameter for an unattended backup, the entire backup must fit on one tape. Table 4. Default Tape Sizes Platform UNIX-like operating systems Default Tape Size None Administration: Backup, Restore, and Data Recovery 7

12 Data Backup and Recovery Platform Windows Default Tape Size 1.5GB Windows the SIZE parameter must be a multiple of 64. Other values are rounded down to a multiple of 64. The number of tapes supplied along with the SIZE clause determines whether there is enough space to store the backed-up data. Use separate TO clauses to set the SIZE value, in kilobytes (KB), for each tape: TO '/dev/rmt/0n' SIZE TO '/dev/rmt/2n' SIZE Use separate lines for each TO clause, not a comma-delimited list. During backup, when the amount of information exceeds the tape capacity or reaches the value specified by the SIZE parameter, BACKUP closes the current tape. For attended and unattended backups that specify the SIZE and STACKER parameters, SAP Sybase IQ waits for the stacker device to auto-load the next tape, then resumes the backup. If the backup exceeds the tape capacity in an attended backup without a STACKER device, BACKUP prompts you to mount a new tape. If the backup exceeds the tape capacity in a unattended backup without a STACKER device, the backup fails. Rewind Tapes SAP Sybase IQ does not rewind tapes before using them. Set the tapes to the correct starting point before you put them in the tape device. If you use a rewinding device, tapes are rewound after backup. If your tape device automatically rewinds tapes, position the tape so that a subsequent backup does not overwrite any information on the tape. Wait for Tape Devices If SAP Sybase IQ cannot open an archive device, the server waits for 10 seconds and tries again. These attempts continue indefinitely, until the operation succeeds or is terminated. A message is written to the server.stderr file. There is no console notification that the server cannot open the archive device. Restoring Tape Backups Position the tape to the start of the IQ data. RESTORE does not reposition the tape for you. Use the same number of tape drives for the restore you used to create the backup. See also Disk Backups on page 9 Read-Only Hardware on page 10 Third-Party Products on page 12 8 SAP Sybase IQ

13 Data Backup and Recovery Disk Backups All disk backups write to the file system; raw disk backup is not supported. All disks on a redundant array of independent devices (RAID) device are treated as a single device. BACKUP appends a suffix to the archive_device name to assign file names to disk backup files. The suffix consists of "." followed by a number that increases by one for each new file. For example, if you specify /iqback/mondayinc as the archive_device, the backup files are /iqback/mondayinc.1, /iqback/mondayinc.2, and so on. This convention allows you to store as large a backup as you need, while allowing you control over the file size; see the SIZE option for details. To accommodate this convention, your file system must support long file names. Disk File Location BACKUP does not create missing directories. If you try to start a backup in a directory that does not exist, the backup fails. Do not use relative path names for disk file locations. BACKUP interprets the path name as relative to the location where the server was started, which you may not be able to identify with certainty when you do a backup. If there is data in other directories along the path, you may not have enough disk space to perform the backup. Disk Backup Size BACKUP determines how much disk space is required to back up your database. If there is not enough disk space available, the backup fails before writing any data. Use the optional SIZE parameter to define backup file sizes. The SIZE value can be less than or greater than the default disk size. Backups that do not include SIZE parameters proceed until the backup is complete (backup is less than disk capacity) or the disk is full. Platform UNIX-like operating systems Windows Default Disk Size 2GB 1.5GB Windows the SIZE must be a multiple of 64. Other values are rounded down to a multiple of 64. The SIZE value is expressed kilobytes (KB). SIZE does not limit the number of bytes per device. SIZE limits the file size. During backup, when the amount of information written to a given file exceeds the default disk size, or reaches the value specified by the SIZE parameter, BACKUP closes the current file and creates another file of the same name, with the next ascending number appended to the file name. For example, bkup1.dat1.1, bkup1.dat1.2, bkup1.dat1.3. Administration: Backup, Restore, and Data Recovery 9

14 Data Backup and Recovery An unattended backup that runs out disk space fails. An attended backup closes all open backup files and prompts you to free up additional disk space (8KB minimum). When the additional space becomes available, the backup resumes with new backup files. Previous Backups BACKUP overwrites existing disk files of the same name. To retain a previous backup, use different file or path names for the archive devices, or move the old backup to another location. Restoring Disk Backups If you back up to disk, then move the backups to tape, move the backups back to disk with the same file names you used when you created the tape backup. SAP Sybase IQ cannot restore disk backups from the tapes where you moved the tape backups. When you restore from disk, you must specify the same number of archive devices (disk files) for the restore as were used to create the backup. See also Tape Backups on page 6 Read-Only Hardware on page 10 Third-Party Products on page 12 Read-Only Hardware Write-once read-many (WORM) disk arrays provide read-only hardware functionality that allows normal read-write use of the disks until data is frozen. Disks can be frozen for an indefinite or fixed-retention period at the volume or file level. Frozen data cannot be modified, and the retention period can be extended, but never decreased. Read-only hardware functionality is not limited to WORM disk array hardware; you can also remove write privilege from a raw device or file system file after the dbspace is altered to read-only. To create and update an archive on read-only hardware: 1. Creating an Archive Assume that the database consists of a single catalog store dbspace named db.db with three main dbspaces: A, B and C. 2. Create New Dbspaces After creating the archive, create new dbspaces. 3. Examine Archived Data Examine the archived database as of time t0. 4. Update the Working Archive If a long time (for example, months or even years) have passed since time t0, you may upgrade the db.db0.working as long as ALTER DATABASE UPGRADE modifies no objects in the IQ main store. 5. Create More Archives 10 SAP Sybase IQ

15 Data Backup and Recovery Create a new archive at time t1. See also Tape Backups on page 6 Disk Backups on page 9 Third-Party Products on page 12 Creating an Archive Assume that the database consists of a single catalog store dbspace named db.db with three main dbspaces: A, B and C. 1. At time t0, alter all three main dbspaces to read-only. 2. Copy db.db to db.db0, either by shutting the database down and copying db.db or using dbbackup to make a copy while the database is still running. 3. Freeze dbspaces A, B and C at the hardware level. Store db.db0 in an immutable form, perhaps by storing it in a file system file on the WORM device and freezing it. These steps archive the database as of time t0 in an immutable form. Create New Dbspaces After creating the archive, create new dbspaces. 1. Create two new main dbspaces, D and E. 2. Continue using the database db.db as a production database. The database objects (tables, indexes, and so on) that existed as of time t0 may have changed so that db.db does not equal db.db0. The database db.db continues to read data from dbspaces A, B, and C as long as the tables that existed at time t0 exist and as long as they contain some unmodified rows of data that existed as of time t0. Even if or when this is no longer true, db.db continues to open A, B and C unless they are dropped from db.db, which is allowed only if they are empty from db.db's point of view. Examine Archived Data Examine the archived database as of time t0. 1. Copy the archived read-only db.db0 to a read-write file db.db0.working. 2. Start db.db0.working. As long as the server name db.db0.working does not conflict with the production system db.db, there is no need to stop the production system. The db.db0.working server opens A, B, C, and D in read-only mode; the catalog opens in read-write mode. Administration: Backup, Restore, and Data Recovery 11

16 Data Backup and Recovery On a UNIX-like OS, this does not interfere with db.db's use of these files. Windows returns a sharing violation. 3. Create user inv; an investigator who wants to examine the archived database. 4. Grant inv RESOURCE permission to create views, stored procedures, global or local temporary tables or any other structures necessary for the investigation. db.db0 as well as A,B, and C remain unchanged. Update the Working Archive If a long time (for example, months or even years) have passed since time t0, you may upgrade the db.db0.working as long as ALTER DATABASE UPGRADE modifies no objects in the IQ main store. 1. The temporary dbspaces that existed as of time t0 are not required to start db.db0.working. Use the server startup switch -iqnotemp to start db.db0.working. 2. Drop and create new temporary dbspaces or use the temporary space created by the - iqnotemp parameter. Create More Archives Create a new archive at time t1. 1. Alter dbspaces D and E to read-only. 2. Copy db.db to db.db1. 3. Freeze D and E. 4. Save db.db1 in an immutable form. 5. Create new main dbspaces, for example, F and G. 6. Continue to use the production system db.db. To use the archived databases db.db0 or db.db1, or even both simultaneously, copy db.db0 and db.db1 to a working file and start a server. Create an archive and follow this procedure to create any number of archived versions of db.db. Third-Party Products SAP Sybase IQ supports backup and restore operations with third-party products that support SAP Sybase databases. To perform a backup or restore with a third-party product, issue the BACKUP DATABASE or RESTORE DATABASE statement as if you were using SAP Sybase IQ to perform the operation, with the following exceptions: For each archive_device, instead of specifying the actual device name, specify a string in the following format: 12 SAP Sybase IQ

17 Data Backup and Recovery dll_name::vendor_specific_information Do not specify the STACKER or SIZE parameters. The dll_name corresponds to a dynamic link library loaded at run time. The dll_name can be from 1 to 30 bytes long, and can contain only alphanumeric and underscore characters. It must be the same for each archive_device. vendor_specific_information varies by product, and can differ for each archive_device. The total string (including dll_name:: vendor_specific_information) can be up to 255 bytes long. BACKUP DATABASE automatically passes vendor information to the third-party program. When you request a third-party backup, BACKUP DATABASE writes this information in the backup header file, and moves the header file on the first tape or disk file actually created for each archive_device. Note: See the Release Bulletin for additional usage instructions or restrictions. Before using any third-party product to back up your database, make sure it is certified with SAP Sybase IQ. See the SAP Sybase Certification Reports for the SAP Sybase IQ product in Technical Documents. See also Tape Backups on page 6 Disk Backups on page 9 Read-Only Hardware on page 10 Queries, Utilities, and Procedures Queries, utilities, and procedures return information about the tablespaces, dbspaces, and dbfiles in your database. The SYSDBFILE system view shows all the dbfiles in your database, including the catalog, message file, and dbfiles in the main and temporary dbspaces. To return dbfile and dbspace statistics, query the SYSDBFILE system view: SELECT dbf.dbfile_name, f.* FROM SYSFILE f, SYSDBFILE dbf WHERE f.file_id=dbf.dbfile_id Results are similar to this: dbfile_name file_id file_name dbspace_name store_type lob_map dbspace_id system 0 /dev/rdsk/sybaseiq/demo/iqdemo.db system 1 (NULL) 0 temporary 15 /temp/sqla0000.tmp temporary 1 (NULL) 15 IQ_SYSTEM_MAIN iqdemo.iq IQ_SYSTEM_MAIN 2 (NULL) IQ_SYSTEM_TEMP iqdemo.iqtmp IQ_SYSTEM_TEMP 2 (NULL) IQ_SYSTEM_MSG iqdemo.iqmsg IQ_SYSTEM_MSG 2 (NULL) iq_main iqdemo_main.iq iq_main 2 (NULL) The file_name column in the SYSFILE system table for the SYSTEM dbspace is not updated during a restore. For the SYSTEM dbspace, the file_name column always reflects Administration: Backup, Restore, and Data Recovery 13

18 Data Backup and Recovery the name when the database was created. The file name of the SYSTEM dbspace is the name of the database file. See Reference: Building Blocks, Tables, and Procedures > System Tables and Views > System View > Alphabetical List of System Views > SYSDBFILE System View. db_backupheader Utility Reads the first backup archive, returns backup statistics and definitions. Syntax: db_backupheader [ path ] backup_file db_backupheader is a command line utility. Output includes backup statistics, database definitions, and dbspace and dbfile specifics. sp_iqdbspace Procedure Returns details about each dbspace. Syntax: sp_iqdbspace [ dbspace-name ] Results are similar to this: DBSpaceName DBSpaceType Writable Online Usage TotalSize Reserve NumFiles NumRWFiles Stripingon StripeSize BlkTypes OkToDrop iq_main MAIN T T M 200M 1 1 T 1K 1H,3254A N IQ_SYSTEM_MAIN MAIN T T M 200M 1 1 T 1K 1H,2528F,32D,128M N IQ_SYSTEM_TEMP TEMPORARY T T 3 25M 200M 1 1 T 1K 1H,64F,16A N See Reference: Building Blocks, Tables, and Procedures > System Procedures > Alphabetical List of System Stored Procedures > sp_iqdbspace Procedure for column definitions. sp_iqfile Procedure Returns details about the dbfiles in a dbspace. Syntax: sp_iqfile [ dbspace-name ] Results are similar to this: DBSpaceName DBFileName Path SegmentType RWMode Online Usage DBFileSize Reserve StripeSize BlkTypes FirstBlk LastBlk OkToDrop IQ_SYSTEM_MAIN IQ_SYSTEM_MAIN iqdemo.iq MAIN RW T M 200M 1K 1H,2528F,32D, 128M N iq_main iq_main iqdemo_main.iq MAIN RW T M 200M 1K 1H,3254A N IQ_SYSTEM_TEMP IQ_SYSTEM_TEMP iqdemo.iqtmp TEMPORARY RW T 3 25M 200M 1K 1H,64F, 16A N See Reference: Building Blocks, Tables, and Procedures > System Procedures > Alphabetical List of System Stored Procedures > sp_iqfile Procedure. 14 SAP Sybase IQ

19 Data Backup and Recovery Renaming Dbspaces To move a database or dbspace, you need to know the name of every dbspace in the database when the backup was made. You can also run the following script in Interactive SQL. This script produces an output file that contains the set of RENAME clauses you can use, if you do not actually change the location of any files. You can substitute any new file locations, and use the resulting file in your RESTORE DATABASE statement. -- Get dbspace and IQ file names and add -- rename syntax including quotation marks select 'rename' as 'restore... rename', dbf.dbfile_name as 'IQ file', 'to' as 'to', '''' + f.file_name + '''' as 'file_path' from SYSFILE f, SYSDBFILE dbf where f.store_type=2 and f.file_id=dbf.dbfile_id -- Send output to a file in proper format -- without delimiters or extra quotation marks output to restore.tst delimited by '' quote ''; -- This produces a restore.tst file like the following: -- rename IQ_SYSTEM_MAIN to '/dev/rdsk/c2t0d1s7' -- rename IQ_SYSTEM_TEMP to '/dev/rdsk/c2t1d1s7' -- rename IQ_SYSTEM_MSG to 'all_types.iqmsg' Note: Because the database may not exist when you need to restore, you may want to run this script after you back up your database. See also Database Backup on page 1 Database Validation on page 3 Performance Options on page 4 Archive Devices on page 6 Backup Scenarios on page 15 Previous Backups on page 22 Backup Log on page 24 Cache Dbspace Backup on page 17 Backup Scenarios Background information and code samples for routine, system-level, and virtual backups. See also Database Backup on page 1 Database Validation on page 3 Administration: Backup, Restore, and Data Recovery 15

20 Data Backup and Recovery Performance Options on page 4 Archive Devices on page 6 Queries, Utilities, and Procedures on page 13 Previous Backups on page 22 Backup Log on page 24 Routine Backups Code samples, background information, and tips for standard backup options. FULL Backups FULL backups make a complete copy of the database. The FULL keyword is optional. This example backs up a database to two tape devices: BACKUP DATABASE TO '/dev/rmt/0' TO '/dev/rmt/1' WITH COMMENT 'Jan 18 full backup of iquser' This example restores the database: RESTORE DATABASE 'iquser' FROM '/dev/rmt/0' FROM '/dev/rmt/1' INCREMENTAL Backup INCREMENTAL backups copy all transactions since the last backup of any type. To make an incremental backup, use the INCREMENTAL keyword. This example creates an incremental backup of the database onto one tape device: BACKUP DATABASE INCREMENTAL TO '/dev/rmt/0' SIZE 150 WITH COMMENT 'Jan 30 incremental backup of iquser' The SIZE parameter specifies the maximum storage capacity per output device. INCREMENTAL SINCE FULL Backups INCREMENTAL SINCE FULL backups back up all changes to the database since the last full backup. This example backs up the database to two tape devices: BACKUP DATABASE INCREMENTAL SINCE FULL TO '/dev/rmt/0' SIZE TO '/dev/rmt/2' SIZE All blocks changed since the last full backup are saved to separate tape devices. Read-Only Files and Dbspaces The READWRITE and READONLY keywords limit backups to read-write or read-only files or dbspaces. The read-write dbspaces and files must be SAP Sybase IQ dbspaces. 16 SAP Sybase IQ

21 Data Backup and Recovery Back up a read-only dbspace to a tape device: BACKUP DATABASE READONLY DBSPACES dsp1 TO '/dev/rmt/0' Restore the dbspace: RESTORE DATABASE READONLY DBSPACES dsp1 FROM '/dev/rmt/0' Back up read-only files to bkp.f1f2: BACKUP DATABASE READONLY FILES dsp1_f1, dsp1_f2 TO 'bkp.f1f2' Back up read-only dbspaces and read-only files to bkp.ro: BACKUP DATABASE READONLY DBSPACES dsp2, dsp3 READONLY FILES dsp4_f1, dsp5_f2 TO 'bkp.ro' See also Cache Dbspace Backup on page 17 System-Level Backups on page 18 Virtual Backups on page 20 Cache Dbspace Backup If your installation uses the cache dbspace for direct-attached storage, follow best practices to avoid errors during backup. The cache dbspace must be online. An offline cache dbspace causes the backup to fail with error For example: Msg 21, Level 14, State 0: SQL Anywhere Error : IQ store dbspace or dbfile IQDAS1 is unavailable. Backup will not be done. (dblib/db_backupid.cxx 859) The cache dbspace must be read-write. A read-only cache dbspace causes the backup to fail with error For example: Msg 21, Level 14, State 0: SQL Anywhere Error : IQ cache dbspace IQDAS1 is read-only. Backup will not be done. (dblib/db_backupid.cxx 863) Backing up individual cache dbspace dbfiles results in an error. SAP Sybase IQ can reconstruct the cache dbspace, provided it sees a raw device, or a file of the correct length at the time the database starts. Before backing up, record the sizes of the files used to hold the cache dbspaces. Changing the path of a cache dbspace dbfile on a secondary node using the RENAME clause results in an error. Administration: Backup, Restore, and Data Recovery 17

22 Data Backup and Recovery See also Routine Backups on page 16 System-Level Backups on page 18 Virtual Backups on page 20 System-Level Backups The BACKUP DATABASE command is the most reliable method you can use to back up IQ data. If you prefer, you can perform a system-level backup, where you handle the entire data backup yourself. To ensure consistent catalog, metadata, and data, shut down the database before a system-level backup. Follow the steps below when you perform a system-level backup. If you attempt to restore your database without these safeguards in place, you are likely to cause data loss or inconsistency, either from activity in the database while the system-level backup occurred, or from missing files. 1. Shut Down the Database Shut down your SAP Sybase IQ database before performing a system-level backup. 2. Back Up the Correct Files Back up required files and optional files. 3. Restoring System-Level Backups Shut down the database server. On a multiplex, shut down all the secondary servers as well as the coordinator. See also Routine Backups on page 16 Cache Dbspace Backup on page 17 Virtual Backups on page 20 Shut Down the Database Shut down your SAP Sybase IQ database before performing a system-level backup. Ensure that no one starts the SAP Sybase IQ database until the system-level backup is complete. Ensuring That the Database is Shut Down The file protection status of the.db file is read-only when the database is shut down cleanly, and read-write when the database is in use. If you are writing a script to perform backups, make sure the script check the access mode of the file, to be sure that the database is shut down. To ensure that a database remains shut down, the script can check the size of the.iqmsg file at the start and end of the script to make sure it has not changed. If the database was started while the script was running, the.iqmsg file is larger. 18 SAP Sybase IQ

23 Data Backup and Recovery Back Up the Correct Files Back up required files and optional files. Required Files All SYSTEM dbspace files, typically named dbname.db, including any additional dbspaces in the catalog store, listed in SYSDBSPACES The transaction log file, which is required for system recovery, typically named dbname.log The IQ_SYSTEM_MAIN dbspace file and typically named dbname.iq Files for any additional dbspaces that have been added to the IQ main store Save the lengths of the following files: The IQ_SYSTEM_TEMP dbspace file, typically named dbname.iqtmp Additional files that have been added to IQ_SYSTEM_TEMP You are not required to back up temporary dbspaces. SAP Sybase IQ can reconstruct any temporary dbspace provided there is a file of the correct length at the time the database starts. Therefore, you may simply keep records of the sizes of the files or raw devices used to hold the temporary dbspaces. Optional Files Back up the ASCII message files such as dbname.iqmsg, the $IQDIR16/logfiles/ *.srvlog, and $IQDIR16/logfiles/*.stderr files, even though these files are not required for a restore. If problems occur during a restore, the.iqmsg file contains information that proves that the database was shut down before the backup started. These files may be useful in diagnosing the cause of the database failure you are recovering from. Make a copy before restoring, for use in later analysis. If IQ message log wrapping is enabled, back up the.iqmsg file so that all messages are accessible if you need them for diagnostic purposes. If message log archiving is enabled (the IQMsgMaxSize server option or the -iqmsgsz server startup switch is not equal to zero, and the IQMsgNumFiles server option or the -iqmsgnum server startup switch is not equal to zero), the server automatically backs up the message log archives. The maximum amount of message log that is archived is 128GB, which is sufficient in most cases. Note: Before a server restart, you must back up the message log archives. After the server restarts, existing log archives are ignored and a new archive is created when the dbname.iqmsg file is full. Administration: Backup, Restore, and Data Recovery 19

24 Data Backup and Recovery Keeping Your Backup List Updated It is critical to add to your system backup specification any dbspaces that are added to the database, whether they are in SYSTEM, IQ_SYSTEM_MAIN, or IQ_SYSTEM_TEMP. To ensure that you are backing up all the files you need, use a script for system-level backups. In the script, before starting the backup, compare a select from SYSFILE (for the system dbspaces) and from SYSIQFILE (for the IQ dbspaces) to a list of dbspaces known to be in the system backup specification. Raw Devices and Symbolic Links If your database files are on raw devices, be sure your system backup is backing up the raw device contents, not just the name of the device in /dev/*. If you use symbolic links for raw device names, as recommended, be sure the system backup utility follows the symbolic link and backs up the device. Restoring System-Level Backups Shut down the database server. On a multiplex, shut down all the secondary servers as well as the coordinator. Before you restore: Review the table of contents of the backup to ensure that all required SAP Sybase IQ files are present. The file list depends on your application. For temporary dbspace files, ensure that all files or raw devices are present with the correct file names (or symbolic links) and lengths. Contents of temporary dbspace files are irrelevant until the database restarts. Ensure that ownership and privilege levels do not change during the restore. Virtual Backups Virtual backups perform a full backup of the catalog store and select SAP Sybase IQ metadata, and metadata not specific to individual tables, including information specific to the freelist, backup, and checkpoint. Virtual backups do not back up table data and metadata in the IQ store. Use the VIRTUAL backup parameters as part of a full database backup. Virtual backups can quickly (in seconds or minutes) back up and restore an entire database minimizing downtime. SAN technologies create multiple mirrored copies of dbfile devices, which offload maintenance tasks, such as consistency checks, on the mirrored copies. Use the sp_iqfile procedure to create a backup list of files that comprise the database. You must make a separate OS-level copy of the corresponding IQ store. To restore from a virtual backup, restore the corresponding system-level copy of the IQ store, then proceed with the IQ full restore of the virtual backup. 20 SAP Sybase IQ

25 Data Backup and Recovery Encapsulated Virtual Backups In an encapsulated virtual backup, the 'shell-command' parameter executes arbitrary shell commands as part of the backup operation: BACKUP DATABASE FULL VIRTUAL ENCAPSULATED ' shell_command ' TO archive_device The BACKUP DATABASE command backs up the catalog and metadata; the 'shell-command' executes an OS-level script or command that backs up the user dbspaces. Encapsulated means that the two phases are wrapped in a transactional boundary. In other words, the backup is complete and consistent the catalog, metadata and user data all match. Shell commands must perform correctly for system-level backups. If a shell command fails, the backup operation throws an exception. For system-level backups of table data to be consistent with the virtual backup without additional steps, the system-level backup must be made during the backup command and by the backup transaction. This example runs the dd command to copy iqdemo: BACKUP DATABASE FULL VIRTUAL ENCAPSULATED 'dd if=iqdemo.iq of=iqdemo.iq.copy' TO 'iqdemo.full' To fully restore a database from an encapsulated virtual backup, restore the system-level backup first, then restore the virtual backup. Decoupled Virtual Backups A decoupled virtual backup copies all dbspaces after the backup operation is complete: BACKUP DATABASE FULL VIRTUAL DECOUPLED TO archive_device Decoupled virtual backups allow users to make catalog changes before the BACKUP DATABASE command completes the full backup. To gain that flexibility, you must perform an incremental backup after the decoupled virtual backup is complete, to resynchronized the catalog and data. If you perform a system-level backup outside of the backup transaction, the IQ store backup will not be consistent with the IQ backup file. However, a nonvirtual IQ incremental backup, together with the virtual full backup, represents a consistent database. This is because the IQ incremental backup copies all IQ store data and metadata that have changed during or since the virtual full backup. Even the automatic commit and checkpoint that are part of the backup command modify the IQ store, making an independent system-level backup inconsistent. Trying to use the database without applying the incremental restore gives unpredictable results. To perform a full virtual decoupled backup, first perform the full backup: Administration: Backup, Restore, and Data Recovery 21

26 Data Backup and Recovery BACKUP DATABASE FULL VIRTUAL DECOUPLED TO 'iqdemo.full' Then perform a nonvirtual incremental backup: BACKUP DATABASE INCREMENTAL SINCE FULL TO 'iqdemo.isf' For a fully restored database from a system-level backup with a virtual decoupled backup, restore the system-level backup, then restore of the virtual decoupled backup followed by an incremental-since-full restore. Virtual Backups with SAN Snapshot or Shadow Hardware Storage area network (SAN) snapshot or shadow hardware provides more flexibility in the backup process by allowing the system-level backup to take place on the shadow copy rather than on the main database. Instead of the system-level copy that is part of the virtual backup, you can create a shadow copy of the IQ store. You can then perform a system-level backup against the shadow copy. This allows the full backup to complete quickly. See also Routine Backups on page 16 Cache Dbspace Backup on page 17 System-Level Backups on page 18 Previous Backups Views, utilities, and procedures return details about previous backups. To query SYSIQBACKUPHISTORY and return details about previous backups, enter: select * from sysiqbackuphistory In these results, each row represents a successful backup operation: bu_id bu_time type selective_type virtual_type dependson_id cmd creator version :13: backup database to... DBA 8 SYSIQBACKUPHISTORY is a system view that stores details about previous backup operations. See Reference: Building Blocks, Tables, and Procedures > System Tables and Views > System Views > Alphabetical List of System Views > SYSIQBACKUPHISTORY System View. db_backupheader Utility Reads the first backup archive, returns backup statistics and definitions. Syntax: db_backupheader [ path ] backup_file 22 SAP Sybase IQ

27 Data Backup and Recovery Execute db_backupheader from the command line. Output includes backup statistics, database definitions, dbspace and dbfile specifics. sp_iqbackupsummary Procedure Summarizes backup operations. Syntax: sp_iqbackupsummary [ timestamp or backup_id ] Results are similar to: backup_id backup_time backup_type selective_type virtual_type depends_on_id creator backup_size user_comment backup_command :13:54.0 Full All inclusive Non virtual 0 DBA (NULL) backup database to... See Reference: Building Blocks, Tables, and Procedures > System Procedures > Alphabetical List of System Stored Procedures > sp_iqbackupsummary Procedure. sp_iqbackupdetails Procedure Summarizes operations performed by a particular backup. Syntax: sp_iqbackupdetails 'backup_id' Results are similar to: backup_id backup_time backup_type selective_type depends_on_id dbspace_id dbspace_name dbspace_rwstatus dbspace_createid :13:54.0 Full All inclusive 0 0 system ReadWrite :13:54.0 Full All inclusive IQ_SYSTEM_MAIN ReadWrite :13:54.0 Full All inclusive iq_main ReadWrite 6... This example omits some columns that appear in the output. See Reference: Building Blocks, Tables, and Procedures > System Procedures > Alphabetical List of System Stored Procedures > sp_iqbackupdetails Procedure. sp_iqrestoreaction Procedure Identifies actions required to restore the database to a consistent state for a given date. Syntax: sp_iqrestoreaction 'timestamp' Results are similar to: sequence_number backup_id backup_archive_list backup_time virtual_type restore_dbspace restore_dbfile backup_comment c:\\\\temp\\\\b :47:40.0 Non virtual c:\\\\temp\\\\b2.inc :47:40.0 Non virtual c:\\\\temp\\\\b3.inc :47:40.0 Non virtual Administration: Backup, Restore, and Data Recovery 23