Nexio Motion 4 Database Backup and Restore

Nexio Motion 4 Database Backup and Restore 26-March-2014 Revision: A

Publication Information 2014 Imagine Communications Corp. Proprietary and Confidential. Imagine Communications considers this document and its contents to be proprietary and confidential. Except for making a reasonable number of copies for your own internal use, you may not reproduce this publication, or any part thereof, in any form, by any method, for any purpose, or in any language other than English without the written consent of Imagine Communications. All others uses are illegal. This publication is designed to assist in the use of the product as it exists on the date of publication of this manual, and may not reflect the product at the current time or an unknown time in the future. This publication does not in any way warrant description accuracy or guarantee the use for the product to which it refers. Imagine Communications reserves the right, without notice to make such changes in equipment, design, specifications, components, or documentation as progress may warrant to improve the performance of the product. Trademarks Product names and other brands (such as ADC, D-Series, Nexio, Nexio Insight, Nexio Motion, PowerSmart, Versio ) are trademarks or trade names of Imagine Communications or its subsidiaries. Microsoft and Windows are registered trademarks of Microsoft Corporation. All other trademarks and trade names are the property of their respective companies. Contact Information Imagine Communications has office locations around the world. For domestic and international location and contact information see: http://www.imaginecommunications.com/contact-us/ Support Contact Information For domestic and international support contact information see: Support Contacts: http://www.imaginecommunications.com/services/technical-support/ ecustomer Portal: http://support.imaginecommunications.com 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 2 of 15

Contents Contents Backup and Restore strategies for SQL Server Database... 4 Introduction... 4 Impact of the Recovery Model on Backup and Restore... 5 Note... 5 Example Customer case... 6... 7 Full backups... 7 Differential Backups... 9 Transaction Log Backups... 10 Backup Strategy Planning... 11 Backup Integrity... 13 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 3 of 15

Backup and Restore strategies for SQL Server Database Backup and Restore strategies for SQL Server Database Introduction The SQL Server backup and restore component provides an essential safeguard for protecting critical data stored in SQL Server databases. To minimize the risk of catastrophic data loss, database needs to be backed up to preserve modifications to data on a regular basis. A well-planned backup and restore strategy helps protect databases against data loss caused by a variety of failures. Test strategy by restoring a set of backups and then recovering database to prepare to respond effectively to a disaster. Backup is the only reliable way to protect SQL Server data from many failures, such as: Media failure. User errors. For example, dropping a table by mistake. Hardware failures. For example, a damaged disk drive or permanent loss of a server. Natural disasters. Additionally, backups of a database are useful for routine administrative purposes, such as copying a database from one server to another, setting up AlwaysOn Availability Groups or database mirroring, and archiving. Backing up and restoring data must be customized to a particular environment and must work with the available resources. Therefore, a reliable use of backup and restore for recovery requires a backup and restore strategy. A well-designed backup and restore strategy maximizes data availability and minimizes data loss, while considering particular business requirements. Place the database and backups on separate devices. Otherwise, if the device containing the database fails, your backups will be unavailable. Placing the data and backups on separate devices also enhances the I/O performance for both writing backups and the production use of the database. Designing an effective backup and restore strategy requires careful planning, implementation, and testing. Testing is required! You do not have a backup strategy until you have successfully restored backups in all the combinations that are included in restore strategy. A variety of factors must be considered. These include the following: The production goals of organization for the databases, especially the requirements for availability and protection of data from loss. The nature of each of databases: its size, its usage patterns, the nature of its content, the requirements for its data, and so on. Constraints on resources, such as: hardware, personnel, space for storing backup media, the physical security of the stored media, and so on. 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 4 of 15

Backup and Restore strategies for SQL Server Database Impact of the Recovery Model on Backup and Restore Backup and restore operations occur within the context of a recovery model. A recovery model is a database property that controls how the transaction log is managed. Also, the recovery model of a database determines what types of backups and what restore scenarios are supported for the database. Typically a database uses either the simple recovery model or the full recovery model. The full recovery model can be supplemented by switching to the bulk-logged recovery model before bulk operations. For an introduction to these recovery models and how they affect transaction log management, see Transaction Logs (SQL Server). The best choice of recovery model for the database depends on business requirements. To avoid transaction log management and simplify backup and restore, use the simple recovery model. To minimize work-loss exposure, at the cost of administrative overhead, use the full recovery model. The following table summarizes the three recovery models. Recovery Model Description Work loss exposure Recovery to point in time Simple No log backups. Automatically reclaims log space to keep space requirements small, essentially eliminating the need to manage the transaction log space. Changes since the most recent backup are unprotected. In the event of a disaster, those changes must be redone. Can recover only to the end of a backup. Full Requires log backups. No work is lost due to a lost or damaged data file. Can recover to an arbitrary point in time (for example, prior to application or user error). Normally none. If the tail of the log is damaged, changes since the most recent log backup must be redone. Can recover to a specific point in time, assuming that backups are complete up to that point in time. Note: If there are two or more full recovery model databases that must be logically consistent, there may have to be implemented special procedures to make sure the recoverability of these databases. 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 5 of 15

Backup and Restore strategies for SQL Server Database Recovery Model Description Work loss exposure Recovery to point in time Bulk logged Requires log backups. An adjunct of the full recovery model that permits high performance bulk copy operations. Reduces log space usage by using minimal logging for most bulk operations. If the log is damaged or bulk logged operations occurred since the most recent log backup, changes since that last backup must be redone. Otherwise, no work is lost. Can recover to the end of any backup. Point-intime recovery is not supported. Example Customer case Simple recovery model is not recommended for any production use because only full database backups are possible and full backups are the most expensive operations on SQL Server. A Customer uses Invenio and Motion4, both databases are highly transactional (but not OLTP) and they are not using batched operations, therefore recommended recovery model is Full. Invenio and Motion4 databases are logically connected because of shared user management and authorization, possible inconsistencies are: User account is lost from backup set of Invenio database Lost (non-existent) user is registered in Motion4 as: Creator/Modifier of workflow Executor of workflow Initiator of Human Task Assignee of human task User information is used mostly for statistical purposes in Motion4, lost/non-existent user information in Motion4 will not lead to system malfunctions in Motion4. If this logical connection needs to be addressed at Customer site, that will require another study and will be presented in another document. 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 6 of 15

There are several different types of backups. Effective backup strategy is the combination of different types of backups. Full backups The simplest kind of backup is a full database backup. It's also possible to do a full backup of a single data file or filegroup. However, these are not commonly used and as all the principles discussed apply to them, too, we are going to focus on database-level backups. But as of SQL Server 2005, each of the more granular backup types work exactly the same (this was not true in SQL Server 2000). The same applies to differential backups they can be performed at the file, filegroup, or database level but these all work in the same way, as well, regardless of the component being backed up. A full database backup provides a complete copy of the database and provides a single point-in-time to which the database can be restored. Even though it may take many hours for the backup process to run, you can still only restore the backup to a single point (effectively at the end of the backup, but we'll discuss exactly what that point is later in this article). A full backup does not allow recovery to any point in time while the backup was running. This is also the same for differential backups. There is a misconception about this, however, fuelled by the fact that you can use the WITH STOPAT=<a time or log sequence number> option on a restore of full and differential backups. Although the syntax allows it, the option has no effect and is just there for convenience. Another misconception about full backups is that they only contain data. Both full backups and differential backups also contain some transaction log records so that the restored component (database, file, or filegroup) can be made transactionally consistent. Consider an example transaction that inserts a record into a table with a single nonclustered index. The pages of the table and index are spread through the data file. The transaction is split into two parts internally: a record insertion into a data page in the table and then the insertion of the required record in an index page in the nonclustered index. If the backup process just happens to read the nonclustered index page before the record insertion, but reads the table data page after the record insertion, then the database represented by just the data in the backup is transactionally inconsistent. This is where the transaction log comes into play. By also including some transaction log records in the backup, recovery can be run on the restored copy of the database, making it transactionally consistent. For this example transaction, depending on when it commits, the recovery part of restore may roll it forward (meaning it will appear as committed in the restored copy of the database) or roll it back (meaning it will not appear at all in the restored copy of the database). In either case, transactional consistency is maintained, which is crucial. A full backup does the following: 1. Force a database checkpoint and make a note of the log sequence number at this point. This flushes all updated-in-memory pages to disk before anything is read by the backup to help minimize the amount of work the recovery part of restore has to do. 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 7 of 15

2. Start reading from the data files in the database. 3. Stop reading from the data files and make a note of the log sequence number of the start of the oldest active transaction at that point. 4. Read as much transaction log as is necessary. Explaining how much transaction log is necessary is best done with the visual aid in Figure 1. The red numbers on the timeline are explained in this list: 1. Checkpoint and begin reading from the database. 2. The read operation reads page X. 3. Transaction A starts. 4. Transaction A makes a change to page X. The copy in the backup is now out of date. Note that the backup will not read page X again it's already passed that point in the database. 5. Transaction B starts. It does not complete before the data read operation completes. 6. Transaction A commits. This commits the changes to page X. 7. The backup data read operation completes and transaction log reading starts. Figure 1: Example Timeline of Changes during a Full Backup Backing up enough of the transaction log is required so that recovery can successfully run during the restore and so that all pages in the database are at the same point in time the time at which the data reading portion of the backup operation completed (Point 7). The transaction log from the beginning of the oldest active (or uncommitted) transaction (Point 5, Transaction B) to the end of the backup (Point 7), is required to allow recovery to run. The transaction log back from the backup checkpoint (Point 1) to the end of the backup (Point 7) is required to allow all pages to be brought to the same point in time. If the transaction log was only included from the beginning of the oldest active transaction (Point 5), then the copy of page X that was restored from the backup read at Point 2 would not be updated with the changes from transaction A, which occurred at Point 4. This means that it would not be 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 8 of 15

transactionally consistent with the rest of the database as of the time the read data operation completed (Point 7). So, the minimum log sequence number (LSN) of the transaction log that is included in the full backup is MIN (LSN of backup checkpoint, LSN of oldest active transaction) and it could be for a transaction that began even before the backup began. This ensures that the restored copy of the database (or whatever you restored page, file, filegroup, database) is completely consistent. This mechanism means that transactions are not paused in any way by backup operations, although the extra I/O workload on the database may slow them down somewhat. The downside of this mechanism is that the transaction log cannot be cleared for the duration of the full backup because it is required. If there's a lot of update activity and the full backup takes a long time to complete, this could lead to transaction log growth. The data contained in a full backup is not necessarily all of the contents of all data files. The backup will only contain the allocated pages from the data files. For example, a single-file database may be 100GB but only contain 15GB of allocated data. In that case, the full backup will only contain the 15GB of allocated data plus the necessary transaction log. (However, the restored database will always be the same size as the original in this case, 100GB.) Another misconception is that the backup process examines or changes the data it is backing up. This is untrue, except for in a single case when backup checksums are enabled. Differential Backups The other type of data backup is a differential backup. A differential backup performs the same operations as a full backup, but only contains all the data that has changed or been added since the previous full backup. A common misconception here is that differential backups are incremental. They are actually cumulative and successive differential backups after a full backup and will increase in size as more data is changed or added. In every 4GB section (called a GAM interval) of every data file there is a special database page called a differential bitmap that tracks which portions (called extents) of that 4GB section have changed since the last full backup, indicating data that has changed or been added to the database. (There are several other allocation bitmaps, too, and you can find more information about these in article "Inside The Storage Engine: GAM, SGAM, PFS and other allocation maps"). A differential backup scans through these bitmaps and only backs up the data file extents that are marked as changed. The bitmaps are reset by the next full backup, so you can see that as more and more of the database changes, more of it will be marked in the differential bitmaps and successive differential backups will be larger and larger. Eventually, if most of the database has changed, a differential backup may become as large as the full backup. You can find out how large your next differential backup will be using a script that is available from article "New script: How much of the database has changed since the last full backup?." As a side note, if you want to take an ad-hoc full backup and not have it reset the differential bitmaps, you should use the WITH COPY_ONLY option on the BACKUP statement. This is very useful, as otherwise the next differential backups would be based on the ad-hoc full backup you took, instead of on the 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 9 of 15

regular (probably scheduled) full backup. This could lead to big problems when you try to restore in a disaster situation. So why are differential backups useful? As discussed in the backup strategy section, differential backups can really speed up restore operations by allowing many transaction log backups to be skipped in the restore process. It's much faster to essentially jump forward in time using a differential backup than to have to replay a lot of transaction log records to get to the same point in time. Transaction Log Backups Transaction log backups are only possible in the FULL or BULK_LOGGED recovery models, whereas full and differential backups are also possible in the SIMPLE recovery model. A transaction log backup contains all the transaction log records generated since the last log backup (or full backup that starts a log backup chain) and is used to allow the database to be recovered to a specific point in time (usually the time right before a disaster strikes). This means they are incremental, unlike differential backups, which are cumulative. Since these are incremental, if you want to restore the database to a particular point in time, you need to have all the transaction log records necessary to replay database changes up to that point in time. These are contained in the log backup chain. A log backup chain is an unbroken series of log backups that contain all the transaction log records necessary to recover a database to a point in time. A chain starts with a full database backup, and continues until something breaks the chain, thus preventing more log backups being taken until another full (or differential) backup is taken. Operations that break the log backup chain include switching to the SIMPLE recovery model, reverting from a database snapshot, and forcibly clearing the log using the WITH NO_LOG or TRUNCATE_ONLY options (which are not available in SQL Server 2008). It is inadvisable to break the log backup chain, as it forces another (potentially large) full backup to be taken. Although a log backup chain stretches back to a full backup, you don't necessarily need to restore all those log backups during recovery. If you took a full backup, say, on Sunday night and on Wednesday night, with log backups every half hour since Sunday night, then restoring the database after a disaster on Friday could use Wednesday's full backup plus all the log backups since Wednesday night instead of having to go all the way back to Sunday night's full backup. Log backups are also required to help manage the size of the transaction log. In the FULL or BULK_LOGGED recovery models, the log will not clear until a log backup has been, so regular log backups must be performed to prevent the log file from growing out of control. If the log cannot clear, the log will grow until it runs out of space. As such, if you don't want to do point-in-time recovery using log backups, the easiest option is to switch to the SIMPLE recovery model and not use the FULL or BULK_LOGGED recovery models. There is a special case in logging that improves performance by allowing some operations to run as minimally-logged operations, where only the page allocations are logged, not the actual insertion of data. This can improve the performance of operations such as bulk loads and index rebuilds. In these cases, not everything about the operation is logged, so the backed up log records don't contain enough 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 10 of 15

information to completely replay the operation. In that case, how can restore and recovery possibly work if there isn't enough information? The answer is that the first log backup following a minimally-logged operation will also contain some data. Just like the differential bitmaps mentioned earlier, there is another bitmap called the minimallylogged bitmap (sometimes called the bulk-changed map). This bitmap tracks which extents of a data file have been changed because of a minimally-logged operation. The log backup following a minimally-logged operation will scan through these bitmaps and also back up those data extents that are marked as having changed. The bitmaps are cleared after being scanned. This means that the log backup has enough information to completely replay the effects of the minimally-logged operation in the database, when the log backup is restored. There's a twist though: There's nothing in that log backup that says when any particular data extent was changed. So a log backup that also contains data from a minimally logged operation cannot be restored to any point in time except the end of the time period covered (or beyond, if the log backup is part of a log backup chain that you're restoring from). So, while you can get performance improvements when switching to the BULK_LOGGED recovery model, you must consider changing to it as a temporary operation just to improve your batch process and once the batch process is complete, you should switch back to FULL and perform a log backup as soon as possible. There is also a special case log backup to allow the log to be backed up after a disaster that damages the data files. This is called a tail-of-the-log (or tail-log) backup, where the data files can be damaged or destroyed, but all the transaction log leading up to the disaster point can be backed up. This allows minimal work loss (called up-to-the-minute recovery) when the database is subsequently restored; however, it is supported only when the database is running in the FULL recovery model. More information on these and restrictions with minimally logged operations can be found in the Books Online topic "Tail-Log Backups." In versions of SQL Server prior to SQL Server 2005, transaction log backups could not be performed concurrently with full database or differential database backups they would block until the databaselevel backup completed. File and filegroup-based backups did not cause log backups to block. While this complicated the recovery process for file and filegroup backups, it gave them an advantage by not blocking log backups. In SQL Server 2005, all full and differential backups (regardless of component) work in the same way. The behavior now is that the concurrent transaction log backup completes, but the transaction log is not cleared until the full or differential backup (that requires the log) also completes. Backup Strategy Planning An effective backup strategy should be designed to meet organization s Recovery Time Objective (RTO) and Recovery Point Objective (RPO). More information about RTO and RPO can be found online. There are several commonly used backup strategies. With a strategy that only includes full backups you're somewhat limited in what you can do with restores. Basically, you can only restore to the time of each full backup, as in Figure 2. If disaster strikes at 23:59 on Saturday, just before the next full backup is scheduled, then all the work since the last full backup might be lost. For this reason, if data-loss needs to be avoided and the data cannot be recreated, log backups are also included, as shown in Figure 3. 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 11 of 15

Figure 2: Backup Strategy with only Full Backups Figure 3: Backup Strategy with Full and Log Backups Imagine that the log backups are being taken every 30 minutes. As long as all the backups are available, this means that the database can be restored to any point in time. However, this still may not be the best strategy. What if disaster strikes at 23:59 on Saturday with this strategy in place? First thing would be to take a tail-of-the-log backup and then start restoring. To restore the database up to the point of the disaster would mean restoring last Sunday's full backup and then 336 log backups (that's six days of 48 log backups per day, plus 47 on Saturday plus the tail-ofthe-log backup). Depending on how much churn there was in the database over the week that could be a huge amount of transaction log that will take a very long time to replay. That's clearly not an optimal restore strategy but it's a common strategy. With a strategy like this, make sure that you've practiced doing a restore so you know whether you can meet your RTO in the event of a disaster. 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 12 of 15

To mitigate this problem, some strategies use more frequent full backups but these might be prohibitively large to take every day, for instance. The alternative is to use differential backups, which only contain the data that has changed since the previous full backup. Continuing our example, that strategy is illustrated in Figure 4. Figure 4: Backup Strategy with Full, Log, and Differential Backups With this strategy, recovering from a disaster at 23:59 on Saturday is a lot faster. Remember that a differential backup is cumulative so the restore strategy is the Sunday full backup, the 00:00 Saturday differential backup, plus all the log backups from Saturday. Having the differential backup from 00:00 Saturday means that all the log backups before that can be skipped, as the differential backup contains the same as the net-result of restoring all those log backups. This was a pretty simple and contrived example, but it clearly shows the benefits of each backup type. Once a backup strategy is designed, make sure you test it to ensure that it allows you to perform your desired restores. Backup Integrity There's no point in having backups if you find that they're corrupt when you try to restore from them. Of course, the best way to check the validity of your backups is to restore them on another server, but in SQL Server 2005 a new feature was introduced that allowed some backup integrity checks to be performed without having to actually restore them. You can use the WITH CHECKSUM option when taking a backup (of any variety). This creates a checksum over the entire backup stream, which is stored in the backup itself. If the backup is a full or differential, and the database has page checksums enabled, then this option will also cause all existing page checksums to be tested as the backup process reads the pages. If a bad page checksum is found, the backup operation will fail. This offers great protection against accidentally backing up a database that's already corrupt in some way. 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 13 of 15

Once the backup has completed, it can be verified using a command like the following: RESTORE VERIFYONLY FROM <backup device(s)> WITH CHECKSUM This will recalculate the checksum over the entire backup stream and compare it against the one stored in the backup. For full and differential backups, it will also test any page checksums on pages in the backup. If any problems are found, then you know that backup has been corrupted in some way. Naturally, there are the exceptions to the rule, where you may want to back up a corrupt database (if it's the only copy of the database you have and you're going to have to run repair, for instance). In that case, you can force the backup to complete using the WITH CONTINUE_AFTER_ERROR option. Customer Case As it was explained at the beginning of this chapter, a backup strategy should meet an organization s RTO and RPO. Without specific information on a Customer, RTO and RPO are based on following assumptions: Invenio and Motion4 databases needs to be backed up Both databases are highly transactional but not OLTP Data stored in both databases is mission critical and can t be reproduced if lost Downtime in case of disaster should be minimal We propose following backup strategy: Strategy with full, log and differential backups Each backup set cycle is 1 week long, starting every Sunday 00:00 local time Create Full database backup on Sunday at 00:00 Create Differential database backup on Monday-Saturday at 00:00 Create transaction log backup every 30 minutes (48/day, yes even at 00:00 when either differential or full backup is created!!!) For backup operations large, redundant disk array with high sequential write figure is a requirement. Backup sets should be created on this location, this is called primary backup location. Second redundant storage location should be provided this is secondary backup location. This location can be anything from low-iops disc array to tape library. It is advised that secondary backup location is on different physical hardware than primary and also if possible hardware should be in two different server rooms. From Sunday-Saturday at 00:00 copy of all files from primary to secondary backup location should be performed. Weekly backup set from previous week should be backed up and sent to external protected data storage off site. Full backup sets needs to be preserved for current week + 3 previous weeks. DBA team should have test SQL Server system that is similar to production (storage capacity and same number of volumes for data storage, CPU and RAM does not have to meet the same level as production SQL Server) DBA team should have a document with step-by-step description of how to perform recovery from disaster. 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 14 of 15

Monthly DBA team needs to perform disaster recovery practice, recovering database from backup sets from productions system. 3 different exercises can be performed: Restore to the latest point-in-time from production (using log tail backups) Restore one of previous week s state Restore to arbitrary point-in-time Based on results from these exercises backup/restore strategy should be updated/refined. When a new disc volume is added to SQL Server or a new file group is introduced to either Invenio or Motion4 database, these changes need to be aligned with current backup/restore strategy. IMPORTANT: A Backup strategy can be accepted only when there is a successful recovery test performed and it meets the requirements set in RTO and RPO and produces a transactionally consistent database with no data loss (except data loss planned in backup strategy)!!!!! DBA s wisdom: The goal of backup strategy is not to create a bunch of huge files but to be able to recover from disasters! 2014 Imagine Communications Corp. Proprietary and Confidential 26-March-2014 Page 15 of 15