State of Wisconsin DET Managed Backup and Restore Service Offering Definition (SOD)

Transcription

1 State of Wisconsin DET Managed Backup and Restore (SOD)

2 Document Revision History Date Version Creator Notes 08/08/ Sari Awadalla Initial draft 01/29/ Nan Linde Modify based on review/discussions 03/12/ Nan Linde Edits after review 03/26/ Fatemah Panahi Edited footer and links 05/06/ Dan Proud 06/12/ Nan Linde 06/29/ Nan Linde Corrected AMPS and AMVS titles in text Add exclusion list, cost example, edits after SOD review Update exclusion list after Advisory review 7/22/ Dan Proud Final edit 8/24/ Dan Proud Updated link to FY10 Rate Sheet 08/05/ Sari Awadalla Updated retention offerings

3 Table of Contents Introduction... 4 What Is Included... 4 What Is Not Included... 4 Benefits... 5 Service Description... 5 Service Period... 6 Roles and Responsibilities... 6 Business Continuity... 6 Performance Metrics (Monitoring/Alerting/Reporting)... 7 Configuration Diagram... 8 Database Backups... 9 SQL Server... 9 Oracle... 9 How Services Are Charged... 9 DET Managed Backup and Restore 3

4 Introduction The Division of Enterprise Technology s (DET) backup environment provides for the backup and restore of client operating systems and data using IBM s Tivoli Storage Manager (TSM). The DET Managed Backup and Restore Service Offering Description (SOD) describes the service DET provides for the backup and restore of client operating systems and data and the roles and responsibilities as they relate to the management of the backup environment and associated agency servers. The service provider is the Division of Enterprise Technology and the service consumers are all state agencies who have subscribed to this service. What Is Included DET-managed backups based on agreed upon policies and data retention schedules (see Table 1) TSM Client installation and configuration of the client on the agency server Web-based management interface to manage agency restores upon request Monitoring of backup status Data recovery through incident reporting processes Storage provisioning for TSM backups Offsite copy of backup data Upon request, backup for Agency Managed Virtual Services (AMVS) Upon request and after a joint review, backup for Agency Managed Physical Services (AMPS) Backups for DET-managed servers What Is Not Included Agency ability to manage TSM environment Advanced resource utilization and reporting through TSM Data archiving or record management Open file backups Agency DR planning Advanced licensing for Tivoli Data Protection for Databases (Oracle, SQL, or Exchange) Available upon request with additional costs Bare Machine Restore (BMR) capabilities Backup of desktops or laptops DET Managed Backup and Restore 4

5 Benefits DET Managed Backup and Restore provides a means for agencies to back up and restore hosted servers and data. TSM is a centrally managed backup solution which takes advantage of Initial Full Backups and Forever Incrementals while allowing for recovery capabilities of server systems to meet requests and requirements of state agencies. DET Managed Backup and Restore relieves the agencies of the concern for data protection required to support the hosted server systems as that becomes the responsibility of DET in the management of the backup environment. Benefits of DET s backup environment include: Lower storage costs o Using TSM s Initial Full Backups and Forever Incrementals; the overall data storage is reduced significantly from the traditional GFS (Grandfather, Father, Son Monthly Full, Weekly Full, Daily Incremental) backups. Shortened backup windows o Using TSM s Initial Full Backups and Forever Incrementals, there is no longer a need for recurring full backups. Daily offsite copies of backed up data o Most agencies are only able to send offsite tapes off once a week, leaving data old and out of date. Because of DET s configuration, all backed up data is copied to an alternate offsite data center ensuring best practices for Disaster Recovery (DR) Secure isolation of backups and restores Reliable and centrally managed backups Service Description The DET Managed Backup and Restore service is designed to provide subscribers with recovery capabilities for clients hosted at DET s data centers and remote locations. These services are provided by DET at the agency s request. Backup clients are installed by request and are billed by total storage used in backups (see How Services Are Charged at the end of this document for more detail on how the service is charged back to an agency). DET retains management and maintenance of the backup environment. An agency may opt to have data restore capabilities assigned to agency IT staff. Otherwise, DET remains responsible for restore requests. The DET Backup environment has been designed to achieve industry best practices standards in backups and restores, while striving to maintain the most cost-effective system possible. One initial full backup will take place the first time a client is backed up. From then on, all future backups of that client are incremental. As files are updated and incremental backups are done, the new incremental versions (active files) merge with the unchanged files in the original backup for a new progressive full backup. The most recent backup of a file is the active version and is retained for as long as the file exists. This will be retained as a permanent full for the lifetime of the files that exist. DET Managed Backup and Restore 5

6 Backups are available for recovery purposes, not long-term record retention or archiving. The use of the Long Term Backup Policy needs a valid and documented business requirement and can significantly increase costs. Complete client restore from backup is not supported by default. In the event of a client crash, the base operating system and service packs must be reinstalled manually and then a backup client must be installed. At that point, any backed up files can be restored. Windows Server System State backups are retained for 3 days. Bare Metal Restore (BMR) is not a standard offering at this time, but may be requested through a Service Request (licensing and maintenance charges may apply). Service Period July 1 through June 30 reviewed and renewed annually. Roles and Responsibilities Roles and responsibilities for the DET Managed Backup and Restore service for AMVS and AMPS can be found here. Roles and Responsibilities for the DET Managed Backup and Restore service for Shared Backup Environment can be found here. Business Continuity The DET backup design provides for off-site replication of the backed up data. The data is shared between the Femrite Data Center and the 101 E. Wilson Data Center. Disaster recovery procedures for the backup environment will be followed as documented in DET Disaster Recovery procedures. Disaster recovery for other IT Services may or may not require backed up data. Restores will be processed as necessary according to procedures documented in Living Disaster Recovery Planning System (LDRPS). By default, Daily backups will be performed at random start times every night of the week between 6:00 p.m. & 3:30 a.m. Customers may request specific start times as necessary. Technician Normal Service Hours are from 7:45 a.m. to 4:30 p.m. Monday thru Friday. All normal problem tickets will be worked on during these hours. Currently there are no on call technicians for backup and restore. DET recognizes the need for a high level of availability for its backup/restore services. Backups are to be available between 6:00 p.m. through 6:00 a.m. 365 days a year. Scheduled maintenance and repairs to the backup environment shall be done during normal service hours. DET Managed Backup and Restore 6

7 Performance Metrics (Monitoring/Alerting/Reporting) Backup and restore speeds are dependent on various factors, including server type, processor speed, RAM, data type (small files vs. large databases), NIC, and LAN/WAN. All efforts will be made to complete backups within the designated backup windows. DET will work with agencies to get the backup process to complete within the designated backup windows, but sudden or unplanned data growth or slow bandwidth resulting from insufficient WAN speeds can impede the ability to complete the backups within the designated backup windows. Every effort will be made by DET backup staff to ensure that all data is protected. If the DET backup staff is unaware of the existence of a client it will not be backed up. It is the ultimate responsibility of the owner of the client/data to ensure that their client/data is known to the DET backup staff. Backups are reviewed by DET staff. Any problems that are found will be addressed during normal service hours to ensure backups will run successfully the next scheduled time. Failed backups are not rerun during the day unless specifically requested by the customer. The standard RMS problem procedure should be followed. The problem record will be assigned to the Backup team to work with the agency reporting the incident. The Backup team may request assistance from other DET or customer teams. DET Backup technician access for server, console, and/or operating system may be needed to help with problem resolution. Backups are not monitored by staff over the weekend, holidays, or during furloughs by default. An automated daily report documenting the status of a client s nightly backup can be requested through a service request. The requester must provide a list of clients to be included in the report and must have authorization to receive information on those clients from the owning agency. This report will be ed daily to designated agency staff for their review. DET monitors acceptable performance metrics for backups and restores. Data throughput and volume are regularly reviewed. Agencies shall be responsible for the type of data used by agency staff. All data (aside from predefined exclusions) shall be stored in both the primary and offsite storage pools for each version backed up. DET does not determine what type of data is work related and what is not. Agencies shall be charged for all data types backed up and all versions stored DET also tracks data growth and can provide trending reports per server. Any changes or updates to a client or network infrastructure by other teams may cause issues with a backup. This will be noted and repaired by the appropriate teams during normal service hours. DET Managed Backup and Restore 7

8 Configuration Diagram The number of file versions and retention times are as follows in Table 1: Standard (default) Table 1. File Versions and Retention Policy Frequency Data Exists Data Deleted Standard 7 day (upon SR request) Standard 15 day (upon SR request) Long Term (upon SR request) Daily Daily Daily Monthly Exchange Daily Exchange Long-Term Monthly System State (default) Daily 30 Days 7 Days 15 Days 62 Days 7 Versions 7 Days 12 Versions 365 Days 3 Versions 3 Days 5 Versions 30 Days 5 Versions 7 Days 5 Versions 15 Days 62 Days 7 Versions 7 Days 12 Versions 365 Days 3 Versions 3 Days Predefined standard exclusions for all clients will be applied. Exclude "*:\...\BOPDATA\...\*.dat" Exclude.compress *:\...\*.jpg Exclude "*:\...\Exchsrvr\...\*" Exclude.compress *:\...\*.Z Exclude "*:\...\system32\perf*.dat" Exclude.compress *:\...\*.zip Exclude "*:\...\tmp.edb" Exclude.dir "*:\...\macintosh volume" Exclude "*:\...\WINDOWS\system32\dhcp\tmp.edb" Exclude.dir "*:\...\microsoft uam volume" Exclude "*:\...\WINDOWS\system32\wins\winstmp.mdb" Exclude.dir "*:\...\Program Files\Cisco Systems\CSAgent" Exclude "*:\oracle\admin\...\*" Exclude.dir "*:\...\Temporary Internet Files" Exclude "*:\oracle\oradata\...\*" Exclude.dir *:\...\Recycler Exclude *:\...\*.dsm Exclude.dir *:\...\SYSTEM32\CONFIG Exclude *:\...\*.LDF Exclude.dir *:\Install Exclude *:\...\*.MDF Exclude.dir *:\SMS_PKGD Exclude *:\...\*.NDF Exclude.dir *:\SMSPKGD$ Exclude *:\...\IO.SYS Exclude.dir *:\SOFTWARE\SMSPKG Exclude *:\...\MSDOS.SYS Include * STANDARD Exclude *:\...\pagefile.sys Include *:\...\System32\Spool\...\* STANDARD Exclude *:\...\Spool\...\* Include *:\SystemStateBackup\*.bkf SSMGMT Exclude c:\...\ntuser.dat* INCLUDE.SYSTEMSTATE ALL SSMGMT Exclude c:\...\usrclass.dat* Exclude *.\...\*.gho Exclude "*:\...\BOPDATA\...\*.dat" Exclude.compress *:\...\*.jpg Exclude "*:\...\Exchsrvr\...\*" Exclude.compress *:\...\*.Z DET Managed Backup and Restore 8

9 Database Backups Database backups are provided by the database products, and as such are not directly managed by TSM. TSM backups are made only from the database backup files, not from the live database. Files that are open or in use at the time they are backed up have the potential to be a corrupt backup version or become a fuzzy backup. Database data files (SQL, Oracle, Exchange, etc.) should be identified and excluded from the cold backups. Data that would never have a need to be restored should be excluded from the backups. All requests for inclusions and exclusions must be documented in an RMS Service Request. SQL Server For SQL Server, a nightly database backup is written to disk and retained in TSM. This simple database backup will provide 1 recovery option for a given 24-hour period. A customer with a point in time recovery requirement can request that the database logs are frequently written throughout the day as part of a regular maintenance plan. On SQL Servers with disk constraints the TSM TDP agent will be required to directly manage all backups. Oracle For Oracle, a full Saturday database backup with regular log backups are kept locally for a minimum of 14 days in Production if local space is available. In non-production environments the cost of space needed will be weighed against the convenience of local backups, and more reliance is placed upon the TSM copies. The database backup sets are managed on local disk with Recovery Manager (RMAN). TSM backups of the RMAN flat file sets are used to provide the recovery period under all conditions. How Services Are Charged DET Managed Backup and Restore charges are based on total storage consumption used for backups, including the offsite copies enabled by default. For example, 10GB of storage backup will be charged 10 times the current GB rate for the backup copy located at the primary data center plus 10 times the current GB rate for the offsite backup copy located at the alternate data center. The bill for the DET Managed Backup and Restore will be on the Enterprise Billing System. Please see the IT Services Rate Sheet for rate information. DET Managed Backup and Restore 9