Recovering the master Database

Transcription

1 04_McBath_PHNJ_ /5/01 2:29 PM Page 129 F O O N U E R Recovering the master Database CHAPTER OBJECTIVES Rebuilding and Recovering the master Database Potential Problems in Rebuilding and Restoring the master Database Recovering Other Databases after master Has Been Rebuilt Notes on Logins and Users after a master Rebuild Program to Generate Logins from the User Database Remapping Orphaned Users Recovering msdb The master database has specific system information common to all databases (server logins, for example). If you lose your master database, chances are you are not ing to start your SQL Server. This chapter es over the process of rebuilding your master database, then recovering it from a backup. Additionally, we over some other critical areas that often get overlooked in a recovery, ly logins and user accounts getting out of synchronization. We ll over two tools on how to fix common problems. We ll also cover some other common problems that you may encounter when rebuilding the master database and how to work around them. Rebuilding and Recovering the master Database The master database is special. When the master database becomes damaged in any way, chances are your SQL Server is not ing to work. In this section, we ll rebuild the master database and then restore it. Remember that once you 129

2 04_McBath_PHNJ_ /5/01 2:29 PM Page Chapter 4 Recovering the master Database rebuild master, you lose all information and data. So, make sure your databases are backed up. If your database files still exist and are recoverable, you can re-attach the databases by using the SQL EM or by using the stored procedure sp_attach_db. Let s run through a typical recovery situation. Scenario Our system took a power hit and we lost a disk drive. When we try to start SQL Server backup, we discover that it won t start, so we look at the error log. The default location for the errorlog is in the (...\LOG) directory: :53:18.45 server Microsoft SQL Server (Intel X86) Aug :57:48 Copyright (c) Microsoft Corporation Enterprise Edition on Windows NT 5.0 (Build 2195: ) :53:18.50 server Copyright (C) Microsoft Corporation :53:18.50 server All rights reserved :53:18.50 server Server Process ID is :53:18.50 server Logging SQL Server messages in file 'C:\Program Files\Microsoft SQL Server\MSSQL\log\ERRORLOG' :53:18.53 server initconfig: Error 2(The system cannot find the file specified.) opening 'C:\Program Files\Microsoft SQL Server\MSSQL\data\master. mdf' for configuration information. We ve lost the master database data file. Use the rebuildm tool located in the...\tools\binn directory (C:...\tools\binn> rebuildm.exe) to rebuild the master database. Figure 4 1 Rebuild Master initial screen.

3 04_McBath_PHNJ_ /5/01 2:29 PM Page 131 Rebuilding and Recovering the master Database 131 Running the rebuildm tool displays the screens shown in Figures 4 1 through 4 5. You will be asked for information such as the location of the data files (the original data files that came on the CD), which are located in the...\x86\data directory on the CD. I strongly suggest you copy the files to a local hard disk before you attempt to run the rebuild. It takes some of the headaches out of the restore, although it s not mandatory. Figure 4 1 shows the splash screen for the Rebuild Master tool. Enter the server and the location of the data files master.mdf and master.ldf. Once you provide the location, and if your sort order is different from the default, change the sort orders for the databases. This information will be in the sp_helpdb and sp_helpsort that you saved before. You can also see the sort information from previous backups by looking at the header information. The backup stores this information on the backup device. In this example, we will take the defaults. Next, SQL Server will confirm that we want to rebuild the master and overwrite the system information stored in the system database as shown in Figure 4 2. Select Yes. SQL Server will next start copying the data files from the source to the target locations. Figure 4 3 shows what you will see. Following the files being copied over, SQL Server will configure itself so that the new master is recognized as seen in Figure 4 4. When these steps are completed, the exit dialog box appears as shown in Figure 4 5. Figure 4 2 Rebuild Master confirmation screen. Figure 4 3 Rebuild Master copying files.

4 04_McBath_PHNJ_ /5/01 2:29 PM Page Chapter 4 Recovering the master Database Figure 4 4 Rebuild Master configuration. Figure 4 5 Rebuild Master complete. At this point, we have a clean SQL Server, but one that does not know other databases exist. Now it s time to restore the backup of master from our last od backup. This will contain the locations of other databases on the server, SQL Server logins, etc. Restoring the master Database To restore the master database, you will need to execute the following steps: 1. Add the backup device: sp_addumpdevice 'disk', 'master_backup', 'c:\tmp\sql_backup\master_backup.dmp' 2. Stop SQL Server. 3. Bring SQL Server up into single-user mode via the command line. To do this, get to a DOS prompt, change directory into the...\mssql\binn directory, and execute the following command: C:\Program Files\Microsoft SQL Server\MSSQL\Binn> sqlservr.exe -c -m This will start SQL Server and the output will be directed to the console. Here s a sample of what will be displayed on your console screen (it is also in the errorlog):

5 04_McBath_PHNJ_ /5/01 2:29 PM Page 133 Potential Problems in Rebuilding and Restoring the master Database :44:08.43 spid3 SQL Server started in single user mode. Updates allowed to system catalogs :44:08.46 spid3 Starting up database 'master'. 4. Log into SQL Server as SQL Server user sa, or as a trusted administrator. 5. Restore the master database just as you would any other database. Note that master s recovery mode is simple, so you will not restore any transaction logs. restore database master from master_backup Here s the output from that command: The master database has been successfully restored. Shutting down SQL Server. SQL Server is terminating this process. 6. Restart SQL Server. Once master has been restored, SQL Server will automatically shut down. Bring it up just as you normally would. 7. Restore other user databases as needed. If your other database files are okay, then you will not need to restore them. If other databases were damaged, or if you are rebuilding the server, you will need to restore them. 8. Resynchronize SQL Server logins and users as needed. If everything is a clean restore, then there are no orphaned users and logins. If there are orphans, then see the sections on creating logins/users and resynchronizing them. Potential Problems in Rebuilding and Restoring the master Database The following are three potential rebuilding and restoring problems and fixes: PROBLEM 1: The rebuildm.exe tool gets caught in an infinite loop. It should only take a few minutes to rebuild. FIX: Rebuildm.exe may have problems with the read-only attributes on the CD for the master data files. Copy them to a local hard drive and then run the following command: C:\tmp\master> attrib -r *.* This will take the read-only attributes off and the rebuildm.exe should run. PROBLEM 2: You try and restore the master from a backup and you get the following error:

6 04_McBath_PHNJ_ /5/01 2:29 PM Page Chapter 4 Recovering the master Database restore database master from master_backup Server: Msg 3108, Level 16, State 1, Line 1 RESTORE DATABASE must be used in single user mode when trying to restore the master database. Server: Msg 3013, Level 16, State 1, Line 1 RESTORE DATABASE is terminating abnormally. FIX: Put the SQL Server into single-user mode via the command line. Execute the following from the BINN directory: sqlservr.exe -c -m PROBLEM 3: Trying to put the master database into single-user mode via sp_dboption or ALTER DATABASE results in the following error: sp_dboption master, 'single user', true /* Server: Msg 5058, Level 16, State 5, Line 1 Option 'SINGLE_USER' cannot be set in database 'MASTER'. sp_dboption command failed. */ FIX: Place the SQL Server into single-user mode via the command line. You need to stop SQL Server and restart it by executing the following from the BINN directory: sqlservr.exe -c -m Recovering Other Databases after master Has Been Rebuilt This section deals with the scenario of losing a master database and not having a back up of it. Once master has been rebuilt, you have a freshly installed server. SQL Server at this point does not know anything about your previous databases that were on this machine. If you had a backup of master you would restore it now. If you did not, you have two choices: Either restore the databases from tape or disk or use SQL EM or the sp_attach_db stored procedure to re-associate the database files at the OS level back to the database. Tip If your other data files appear to be okay (meaning the disk drives were not damaged), I strongly suggest that you first try to use the sp_attach_db procedure because it is a lot faster to attach pre-existing files than to restore a whole database. The reason is that a restore will have to lay all the bits back on the disk, while the sp_attach_db stored procedure just puts a few entries back in the system tables so SQL Server will recognize the database. The worst thing that can happen is that sp_attach_db doesn t work.

7 04_McBath_PHNJ_ /5/01 2:29 PM Page 135 Recovering Other Databases after master Has Been Rebuilt 135 Then, you drop the database and restore from tape. The worst thing is that you lose five minutes; the best is you save several hours of restore. If you proactively create the sp_attach_db scripts, it will make recovery even faster. Also, attaching the existing databases allows you to back up the tail of the log. Using the RESTORE Command Method 1. Add the backup devices. 2. Restore the database(s). 3. Synchronize the users if necessary (use sp_fixlogins later in the chapter). use master sp_addumpdevice 'disk', 'full_backup', 'c:\tmp\sql_backup\full_backup.dat' sp_addumpdevice 'disk', 'log_backup', 'c:\tmp\sql_backup\log_backup.dat' backup database mcbath to full_backup with init backup log mcbath to log_backup with init boom... lose your master database... use the rebuildm.exe tool to rebuild your master database then load your database back from tape use master sp_addumpdevice 'disk', 'full_backup', 'c:\tmp\sql_backup\full_backup.dat' sp_addumpdevice 'disk', 'log_backup', 'c:\tmp\sql_backup\log_backup.dat' restore database mcbath from full_backup restore log mcbath from log_backup Using the sp_attach_db Method You need to know the sources for the database files. These can be located from a variety of sources: Old database logs sp_helpdb Using a dir command to look for files in the data file directory Here is an example of attaching data files using sp_attach_db: sp_helpdb mcbath

8 04_McBath_PHNJ_ /5/01 2:29 PM Page Chapter 4 Recovering the master Database /* db_size owner compatibility_level mcbath MB sa fileid file filegroup size mcbath_data0 1 c:\tmp\sql_data\data0.mdf PRIMARY 1024 KB mcbath_log 2 c:\tmp\sql_log\log0.ldf NULL 5120 KB mcbath_data1 3 c:\tmp\sql_data\data1.ndf data KB mcbath_index0 4 c:\tmp\sql_data\index0.ndf index KB mcbath_data2 5 c:\tmp\sql_data\data2.ndf data 5120 KB test_filegroup_file0 6 c:\tmp\sql_data\test_filegroup0.ndf test_filegroup KB test_filegroup_file1 7 c:\tmp\sql_data\test_filegroup1.ndf test_filegroup KB test_filegroup_file2 8 c:\tmp\sql_data\test_filegroup2.ndf test_filegroup KB */ we could have just added the files via sp_attach_db which is *MUCH* faster than a restore because it s just adding the entries into system tables as opposed to loading data into the database. note, we are assuming the data files are ok. run dbcc checkdb just to be sure once the database is up and running. use master select from sysdatabases = = = = = = = = = 'c:\tmp\sql_log\log0.ldf' select from sysdatabases /*--Before master tempdb model msdb pubs Northwind (6 row(s) affected)

9 04_McBath_PHNJ_ /5/01 2:29 PM Page 137 Notes on Logins and Users after a master Rebuild After master tempdb model msdb pubs Northwind mcbath (7 row(s) affected) */ Notes on Logins and Users after a master Rebuild If you have backed up your master database and user databases at the same time, then restoring will be easy. But if for some reason you didn t back up your master, or can t recover it from a backup, things could get tricky. Specifically, user logins for the server may be out of synchronization with the users in the database. The following discussion will walk you through this scenario. For purposes of discussion, let s assume the following: You do not have the latest copy of the master database. You have added a whole lot of server logins and users to the database. After you do a rebuild of the master database and then restore it, you may encounter problems with users trying to log in. This is because the users no longer have logins in the server, but they do still have user assignments in the user database. What s important is that the SIDs are the same in both tables. For example, I could have the syslogins and sysusers tables match up by, but not SID number. If this happened, users would not be able to log in. Here s an example of that: print 'master..syslogins:' select, sid from master..syslogins order by print 'mcbath..sysusers:' select, sid from mcbath..sysusers where sid <> 0x01 and sid <> 0x00 and sid is not NULL order by

10 04_McBath_PHNJ_ /5/01 2:29 PM Page Chapter 4 Recovering the master Database master..syslogins: BUILTIN\Administrators sa test_login test_login_1 test_login_2 test_login_3 test_login_4 test_login_5 (8 row(s) affected) mcbath..sysusers: test_login test_login_1 test_login_2 test_login_3 test_login_4 test_login_5 (6 row(s) affected) sid 0x x01 0x80559A87BC2B7B49BA43EF92B6EDF87E 0x1CF91F72D5C03C49A982EBCDA3756F9B 0xFB518D2DCD41F54088F1A613B0F0E9E1 0x779A044E A53E1D4E604A27AD 0x4DB074BF2B52EF40AA7982D11528C268 0xED863C488BF15E4EA5CB677DCA0F7798 sid 0x80559A87BC2B7B49BA43EF92B6EDF87E 0x85B6F41D4C681847B475097DAC1BA085 0x89EBE069EAC0614EA33F8C4EA283C889 0xC657B3EC122ED64883F517CA717728F7 0x3E1654A4E8AA634DA5E3DAFBAFD84FA4 0xE91D1C6414BE4C4E8CD9EB BB8 For users to log in, they need to have the syslogins, sysusers, and SID numbers in the tables match up. Here s an example of what a successful restore looks like: master..syslogins: BUILTIN\Administrators sa test_login test_login_1 test_login_2 test_login_3 test_login_4 test_login_5 mcbath..sysusers: test_login test_login_1 test_login_2 test_login_3 test_login_4 test_login_5 sid 0x x01 0x80559A87BC2B7B49BA43EF92B6EDF87E 0x85B6F41D4C681847B475097DAC1BA085 0x89EBE069EAC0614EA33F8C4EA283C889 0xC657B3EC122ED64883F517CA717728F7 0x3E1654A4E8AA634DA5E3DAFBAFD84FA4 0xE91D1C6414BE4C4E8CD9EB BB8 sid 0x80559A87BC2B7B49BA43EF92B6EDF87E 0x85B6F41D4C681847B475097DAC1BA085 0x89EBE069EAC0614EA33F8C4EA283C889 0xC657B3EC122ED64883F517CA717728F7 0x3E1654A4E8AA634DA5E3DAFBAFD84FA4 0xE91D1C6414BE4C4E8CD9EB BB8

11 04_McBath_PHNJ_ /5/01 2:29 PM Page 139 Program to Generate Logins from the User Database 139 You can fix these errors by several methods: Drop all users from the user database. Drop the logins from the server. Then, recreate them either by a script that you had ready for such an occasion or enter them in manually. This method is the least risky, but takes the most time. Insert the missing logins into the server, and then update the system tables via an update statement that synchronizes the SID numbers. This is higher risk. Run a script that creates the logins in the server based on the users in the database, and then resynchronizes the SID numbers via system stored procedures and/or scripts. This is probably the fastest and safest method. Regardless, users will get new passwords. You just need to decide if they are ing to be NULL or fabricated. For security reasons, I suggest random passwords. create proc varchar(20)='test' as begin set nocount on Program to Generate Logins from the User Database This is the stored procedure to create logins in the server from a restored database that had orphaned users. There are two modes you can run it in, test and run. Test mode will show you the output before it makes any changes. This way, you can check that the passwords and logins are correct. Run mode will actually execute the changes and update the system tables. Here are the combinations in which you can run it: If you don t specify to use random passwords, then the password will be password. Note, the random password uses the RAND() function. For better password generation, use a better RAND() seed if you need it. int sys sys

12 04_McBath_PHNJ_ /5/01 2:29 PM Page Chapter 4 Recovering the master Database sys int sys varbinary(85) varchar(20) if (upper(@run) <> 'RUN') begin print '' print '**************************' print '* Test Run *' print '**************************' print '' end = count(*) from sysusers where sid <> 0x01 and sid <> 0x00 and sid is not NULL = db_(dbid) from master..sysprocesses where spid=@@spid =a. from master..syslanguages a, master..sysconfigures b where b.comment = 'default language' and b.value = a.langid declare sysusers_cursor cursor for select, sid from sysusers where sid <> 0x01 and sid <> 0x00 and sid is not NULL order by open sysusers_cursor fetch next from while (@@FETCH_STATUS = 0) begin + 1 if (upper(@random) = 'RANDOM') begin end else begin

13 04_McBath_PHNJ_ /5/01 2:29 PM Page 141 Program to Generate Logins from the User Database 141 end if = 'RUN') begin @sid end select 'Adding @sid fetch next from end close sysusers_cursor deallocate sysusers_cursor end print 'master..syslogins:' select, sid from master..syslogins order by select print '' select, sid from sysusers where sid <> 0x01 and sid <> 0x00 and sid is not NULL order by set nocount off Here is the output when the logins and users are not synchronized between the user s database and the master database: *************************** * Test Run * *************************** Adding Login: test_login mcbath us_english 0x80559A87BC2B7B49BA43EF92B6EDF87E Adding Login: test_login_ mcbath us_english 0x85B6F41D4C681847B475097DAC1BA085 Adding Login: test_login_ mcbath us_english 0x89EBE069EAC0614EA33F8C4EA283C889 Adding Login: test_login_ mcbath us_english 0xC657B3EC122ED64883F517CA717728F7 Adding Login: test_login_ mcbath us_english 0x3E1654A4E8AA634DA5E3DAFBAFD84FA4

14 04_McBath_PHNJ_ /5/01 2:29 PM Page Chapter 4 Recovering the master Database Adding Login: test_login_ mcbath us_english 0xE91D1C6414BE4C4E8CD9EB BB8 master..syslogins: sid BUILTIN\Administrators 0x sa 0x01 mcbath..sysusers: test_login test_login_1 test_login_2 test_login_3 test_login_4 test_login_5 sid 0x80559A87BC2B7B49BA43EF92B6EDF87E 0x85B6F41D4C681847B475097DAC1BA085 0x89EBE069EAC0614EA33F8C4EA283C889 0xC657B3EC122ED64883F517CA717728F7 0x3E1654A4E8AA634DA5E3DAFBAFD84FA4 0xE91D1C6414BE4C4E8CD9EB BB8 Following is the output when the logins and users have been synchronized between the user s database and the master database using the stored procedure: New login created. Adding Login: test_login mcbath us_english 0x80559A87BC2B7B49BA43EF92B6EDF87E New login created. Adding Login: test_login_ mcbath us_english 0x85B6F41D4C681847B475097DAC1BA085 New login created. Adding Login: test_login_ mcbath us_english 0x89EBE069EAC0614EA33F8C4EA283C889 New login created. Adding Login: test_login_ mcbath us_english 0xC657B3EC122ED64883F517CA717728F7 New login created. Adding Login: test_login_ mcbath us_english 0x3E1654A4E8AA634DA5E3DAFBAFD84FA4 New login created. Adding Login: test_login_ mcbath us_english 0xE91D1C6414BE4C4E8CD9EB BB8 master..syslogins: sid BUILTIN\Administrators 0x

15 04_McBath_PHNJ_ /5/01 2:29 PM Page 143 Remapping Orphaned Users 143 sa test_login test_login_1 test_login_2 test_login_3 test_login_4 test_login_5 mcbath..sysusers: test_login test_login_1 test_login_2 test_login_3 test_login_4 test_login_5 0x01 0x80559A87BC2B7B49BA43EF92B6EDF87E 0x85B6F41D4C681847B475097DAC1BA085 0x89EBE069EAC0614EA33F8C4EA283C889 0xC657B3EC122ED64883F517CA717728F7 0x3E1654A4E8AA634DA5E3DAFBAFD84FA4 0xE91D1C6414BE4C4E8CD9EB BB8 sid 0x80559A87BC2B7B49BA43EF92B6EDF87E 0x85B6F41D4C681847B475097DAC1BA085 0x89EBE069EAC0614EA33F8C4EA283C889 0xC657B3EC122ED64883F517CA717728F7 0x3E1654A4E8AA634DA5E3DAFBAFD84FA4 0xE91D1C6414BE4C4E8CD9EB BB8 Tip To avoid these types of ugly problems, back up your master database (which is very small) every time you back up your user database. Also, document your system very well. Have scripts that work and have been tested well in advance. Remapping Orphaned Users Quite often after you restore a database, there are user login, security, and permission problems because SQL Server requires a database login and you also have to be a user of the database. When you restore a database, you are quite often just restoring users to the database and not restoring server logins. Thus, you end up in a situation that I call orphaned users: a database with users but a server with different login SIDs for those users. SQL Server provides a stored procedure (sp_change_users_login) to synchronize users and logins, but it can be tedious to use if there are more than a few users. The following is a script that automates much of this and helps on large databases with many users. It matches up users to SQL Server logins by and then links the SIDs together so there are no orphaned users. What s important to remember is that the user and login have to preexist. If they don t, read the section Notes on Logins and Users after A master Rebuild, which es over creating logins when only users exist. CREATE PROCEDURE dbo.sp_fixusers AS BEGIN varchar(25) DECLARE fixusers CURSOR FOR SELECT UserName = FROM sysusers

16 04_McBath_PHNJ_ /5/01 2:29 PM Page Chapter 4 Recovering the master Database WHERE issqluser = 1 and (sid is not null and sid <> 0x0) and suser_s(sid) is null ORDER BY OPEN fixusers FETCH NEXT FROM fixusers = 0 BEGIN EXEC FETCH NEXT FROM fixusers END CLOSE fixusers DEALLOCATE fixusers END Note It is important to over the final results to ensure that the security for each user is correct. The tool will attempt to match users and logins properly, but there is a chance that a user could be mismatched and get the wrong level of security. Always check your work. Recovering msdb The msdb database contains system-wide information such as log shipping, replication information, backup history, and marked transaction information. Thus, it is critical that you back it up as part of your recovery strategy. Probably the biggest distinction with backing up msdb up to this point is that msdb is in simple mode (by default), which doesn t allow backups of transaction logs. This makes recovery faster because there is less to do. Process 1. Back up the msdb database. 2. Corruption happens. In this case, a data file could not be opened, as seen in the errorlog. 3. Since this is a simple recovery model, all we have to apply is the full backup and no transaction logs. msdb is a system database, thus you cannot drop it. Here s the error you would receive if you tried: Note The default recovery model for msdb is simple. It can be changed to full. For this example, I use the default.

17 04_McBath_PHNJ_ /5/01 2:29 PM Page 145 Recovering msdb 145 Server: Msg 3708, Level 16, State 6, Line 1 Cannot drop the database. 'msdb' because it is a system database. use master Script backup database msdb to full_backup with init /* Processed 1496 pages for database 'msdb', file 'MSDBData' on file 1. Processed 1 pages for database 'msdb', file 'MSDBLog' on file 1. BACKUP DATABASE successfully processed 1497 pages in 13,361 seconds (0.917 MB/sec). */ /* sql server was stopped and started from the errorlog, we see that MSDB will not come up. A file is missing: :02:37.37 spid8 Starting up database 'msdb' :02:37.37 spid9 Starting up database 'pubs' :02:37.37 spid10 Starting up database 'Northwind' :02:37.37 spid11 Starting up database 'mcbath' :02:37.37 spid8 udopen: Operating system error 2(The system cannot find the file specified.) during the creation/opening of physical device C:\Program Files\Microsoft SQL Server\MSSQL;data;msdbdata.mdf :02:37.59 spid8 FCB: Open failed: Could not open device C:\Program Files\Microsoft SQL Server/MSSQL/data/msdbdata.mdf for virtual device number (VDN) :02:37.75 spid8 Device activation error. The physical file 'C:\Program Files\Microsoft SQL Server\MSSQL;data;msdbdata.mdf' may be incorrect. */ restore database msdb from full_backup */ Processed 1496 pages for database 'msdb', file 'MSDBData' on file 1. Processed 1 pages for database 'msdb', file 'MSDBLog' on file 1. RESTORE DATABASE successfully processed 1497 pages in seconds (0.897 MB/sec). */

18 04_McBath_PHNJ_ /5/01 2:29 PM Page Chapter 4 Recovering the master Database select, dbid from master..sysdatabases /* dbid master 1 tempdb 2 model 3 msdb 4 pubs 5 Northwind 6 mcbath 7 (7 row(s) affected) */ Summary By having a carefully documented and tested recovery plan, restoring a server after a comprehensive failure doesn t have to be a gut-wrenching experience. I strongly suggest that at the very least, you back up the master and msdb databases in addition to your user databases with every backup. Both databases are very small and will make your recovery experience easier. If possible, you could also back up the databases (master and msdb) to a local disk drive and tape. This would make recovery even faster. Since both are small, it would be easy and fast. As a precaution, you would need to put the backup on a different disk than the database if you lost the database disk, you would also lose your disk backups! Using this chapter in conjunction with the previous chapter, you should have most critical recovery scenarios covered.