Contents Abstract... 2 SnapComms Solution Environment... 2 Concepts... 3 What to Protect... 3 Database Failure Scenarios... 3 Physical Infrastructure Failures... 3 Logical Data Failures... 3 Service Recovery Strategies... 4 In real life one will often find a mix of these for various components of a system.... 4 Data Backups... 4 Database Backups... 4 Data Recovery Process... 4 High Availability Considerations... 5 What Does High Availability Mean?... 5 Redundant Components... 5 Data Redundancy Options... 5 SQL Server High Availability Options... 6 Failover Clustering... 6 Failover Clustering in a Nutshell... 6 Database Mirroring... 7 Database Mirroring in a Nutshell... 8 Log Shipping... 9 Transaction Log Shipping in a Nutshell... 9 Replication... 9 SQL Server Replication in a Nutshell... 9 Data Recovery Requirements... 11 Considerations for Administrators... 12 SnapComms Recommendations... 13 Low Budget/Resource: DR using Database Mirroring... 13 Medium Budget/Resource: Cluster Database Mirroring... 13 High Budget/Resource: SAN-Based Mirroring... 14 References... 15 Books... 15 Presentations... 15 Articles... 15 1
Abstract Every company or organization relies on their data to be available all the time. For that reason protecting systems and data is very high priority. This document provides a quick overview of the various concepts and components of data protection for information systems, provides an introduction to the various backup options available for SQL Server based environments and gives specific recommendations for High Availability Redundant Solutions as recommended for use with the SnapComms solution. SnapComms Solution Environment The SnapComms solution is utilizes Microsoft SQL Server. This means ALL relevant data (users, content, statistics & reports, configuration, etc) is stored within a SQL Server database. The SnapComms application isdeveloped in ASP.NET and is fully self-contained. In case of fatal failure, a simple secondary cold standby installation which connects to the same backend SQL Server database, will provide immediate failover. In a nutshell the main focus for Data Protection of the SnapComms Solution centers on the protection of the SQL Server database. 2
Concepts When trying to make a system available without interruption all the time, one needs to consider the following degrees of protection for information systems. Business Risk Solution Data Recovery Data loss Redundant data High Availability Downtime of database service Redundant system components Disaster Recovery Downtime of business operations Redundant systems and facilities What to Protect Application data stores o Databases o Files o Other data repositories Database services o DBMS availability for applications Application services o Application availability for users and external systems Databases are the heart of most information systems - they deserve the highest affordable protection. Database Failure Scenarios There is various problem sources as to why a database can fail: Physical Infrastructure Failures Storage subsystem o Disk o Controller Network Server Power Logical Data Failures Operator errors o DBMS interruption o Drops / deletes Application defects DBMS defects Data corruption 3
Service Recovery Strategies There are several recovery strategies, the most common ones are: Standby Mode Cold standby Failover Behavior Manual intervention required to restore offline data copy SQL Server Feature Backup and restore Warm standby Data copy online and ready Manual failover required Transaction log shipping Database mirroring Hot standby Automatic failover Database mirroring Failover clustering In real life one will often find a mix of these for various components of a system. Data Backups Database backups are absolutely essential. Any High Availability setup should NEVER replace a good data backup and recovery plan. Database Backups Traditional backup types o Full backup o Differential backup o Transaction log backup Disk is better than tape o First backup to disk (separate physical disk volume) o Detect exceptions encountered during backup o Verify backup files o Copy backup files to tape or remote disk Data retention policy for backup files Data Recovery Process Backup file sets o Full baseline, differential, and transaction logs Retrieving backup files o Offsite storage o Tape o Network copy o Dependency on multiple people to get access to backup files Recovery strategy depends on failure scenario o Create comprehensive failure matrix o Devise recovery strategy for each scenario o Does worst-case recovery scenario fit within SLA parameters? Recovery time; SLA o Include future data growth in recovery plan o Fully test recovery strategies practice is essential 4
High Availability Considerations What Does High Availability Mean? Minimize or avoid service downtime o Whether planned or unplanned When components fail, service interruption is brief or non-existent o Automatic failover Eliminate single points of failure (as affordable) o Redundant components o Fault-tolerant servers Redundant Components The objective is to avoid single points of failure (where affordable). The approach to achieve this is to use redundant components for database service. This includes Database server nodes Server components o ECC RAM; failure-tolerant HW & OS DBMS instance User databases Storage devices Storage unit components o MPIO: Interfaces; paths; switches; controllers o RAID: Disks Networking o MPIO: Interfaces; paths; switches Data copies o E.g. Recovering torn page from mirror in SQL Server 2008 Data Redundancy Options Synchronous redundancy o Network bandwidth cost o Network latency and application performance o Network reliability Asynchronous redundancy o Risk of data loss o More cost-effective o Resilient to network latency issues Candidate Technologies o SQL Server database mirroring o Failover clustering with SAN-based mirroring 5
SQL Server High Availability Options SQL Server provides several options for creating high availability for a server or database. Highavailability options include the following: Failover Clustering Failover clustering provides high-availability support for an entire instance of SQL Server. A failover cluster is a combination of one or more nodes, or servers, with two or more shared disks. Applications are each installed into a Microsoft Cluster Service (MSCS) cluster group, known as a resource group. At any time, each resource group is owned by only one node in the cluster. The application service has a virtual name that is independent of the node names, and is referred to as the failover cluster instance name. An application can connect to the failover cluster instance by referencing the failover cluster instance name. The application does not have to know which node hosts the failover cluster instance. A SQL Server failover cluster instance appears on the network as a single computer, but has functionality that provides failover from one node to another if the current node becomes unavailable. For example, during a non-disk hardware failure, operating system failure, or planned operating system upgrade, you can configure an instance of SQL Server on one node of a failover cluster to fail over to any other node in the disk group. A failover cluster does not protect against disk failure. You can use failover clustering to reduce system downtime and provide higher application availability. Failover clustering is supported in SQL Server Enterprise and SQL Server Developer editions, and, with some restrictions, in SQL Server Standard edition. Failover Clustering in a Nutshell node A node B Shared Storage system DBs user DBs quorum Two clustered nodes o Active/Passive configuration MS SQL services o Running on virtual server Shared storage device o User databases o System databases o Quorum drive o Redundant internal components Redundancy at database instance level o All databases fail over together o Shared copy of system databases 6
Single data copy on shared storage device o No I/O overhead reducing throughput o Storage unit is single point of failure for cluster All database services are clustered o SQL Agent; Analysis Services; Full-Text engine, MS DTC Automatic failover (up to minutes) DBMS accessed over virtual IP Database not available from inactive node for DB client connections o Storage is controlled by one cluster node at a time Requires hardware certified by Microsoft for Microsoft Cluster Service For more information about failover clustering, see Getting Started with SQL Server 2008 R2 Failover Clustering and Installing a SQL Server 2008 R2 Failover Cluster. Database Mirroring Database mirroring is primarily a software solution to increase database availability by supporting almost instantaneous failover. Database mirroring can be used to maintain a single standby database, or mirror database, for a corresponding production database that is referred to as the principal database. The mirror database is created by restoring a database backup of the principal database with no recovery. This makes the mirror database is inaccessible to clients. However, you can use it indirectly for reporting by creating a database snapshot on the mirror database. The database snapshot provides clients with read-only access to the data in the database as it existed when the snapshot was created. Each database mirroring configuration involves a principal server that contains the principal database, and a mirror server that contains the mirror database. The mirror server continuously brings the mirror database up to date with the principal database. Database mirroring runs with either synchronous operation in high-safety mode, or asynchronous operation in high-performance mode. In high-performance mode, the transactions commit without waiting for the mirror server to write the log to disk, which maximizes performance. In high-safety mode, a committed transaction is committed on both partners, but at the risk of increased transaction latency. In its simplest configuration, database mirroring involves only the principal and mirror servers. In this configuration, if the principal server is lost, the mirror server can be used as a warm standby server, with possible data loss. High-safety mode supports an alternative configuration, high-safety mode with automatic failover. This configuration involves a third server instance, known as a witness, which enables the mirror server to act as a hot standby server. Failover from the principal database to the mirror database typically takes several seconds. Since SQL Server 2005 Service Pack 1 (SP1), database mirroring partners and witnesses have been supported by SQL Server Standard and Enterprise editions. But the partners must use the same edition, and asynchronous database mirroring (high-performance mode) is supported only by SQL Server Enterprise edition. Witnesses are also supported by SQL Server Workgroup and Express. 7
Database Mirroring in a Nutshell witness (optional) node A node B Local Storage local sys DBs source user DB Local Storage local sys DBs mirror user DB Redundancy at user database level o Duplicate copy of user database o Independent storage devices o Multiple copies of instance databases Mirrored over private network channel o Mirror always redoing transactions from principal o Negligible impact on transaction throughput Multiple mirroring modes: o High-availability: commit @ log on mirror; automatic failover o High-protection: commit @ log on mirror; manual failover o High-performance: commit when logged on principal Very fast automatic failover seconds Requires witness server Mirror-aware application client connection o Provided by client library o Database connection string must specify both servers Mirror may be available for read-only access (snapshots) Works with standard hardware With mirroring, more than one server is required to decide on failover Witness automates failover from primary to mirror Watches database availability Reports observations back to principal and mirror Runs in separate SQL Server instance (Express is OK) Prevents split brain scenario Very low resource consumption Can be witness for multiple databases Not a single point of failure For more information about database mirroring, see Database Mirroring. 8
Log Shipping Like database mirroring, log shipping operates at the database level. You can use log shipping to maintain one or more warm standby databases for a corresponding production database that is referred to as the primary database. Standby databases are also referred to as secondary databases. Each secondary database is created by restoring a database backup of the primary database with no recovery, or with standby. Restoring with standby lets you use the resulting secondary database for limited reporting. A log shipping configuration includes a single primary server that contains the primary database, one or more secondary servers that each have a secondary database, and a monitor server. Each secondary server updates its secondary database at set intervals from log backups of the primary database. Log shipping involves a user-modifiable delay between when the primary server creates a log backup of the primary database and when the secondary server restores the log backup. Before a failover can occur, a secondary database must be brought fully up-to-date by manually applying any unrestored log backups. Log shipping provides the flexibility of supporting multiple standby databases. If you require multiple standby databases, you can use log shipping alone or as a supplement to database mirroring. When these solutions are used together, the current principal database of the database mirroring configuration is also the current primary database of the log shipping configuration. Transaction Log Shipping in a Nutshell Warm standby solution Duplicate user database o Copy transaction logs to standby server & restore Database available for read-only access o Users must disconnect for logs to be applied o Two database licenses required if querying standby Manual application failover Supported on standard hardware Possible data loss (unapplied transactions) Log shipping is supported in the SQL Server Enterprise, Standard, and Workgroup editions. For more information about log shipping, see Log Shipping Overview and Log Shipping Administration. Replication Replication uses a publish-subscribe model. This lets a primary server, referred to as the Publisher, distribute data to one or more secondary servers, or Subscribers. Replication enables real-time availability and scalability across these servers. It supports filtering to provide a subset of data at Subscribers, and also allows for partitioned updates. Subscribers are online and available for reporting or other functions, without query recovery. SQL Server offers three types of replication: snapshot, transactional, and merge. Transactional replication provides the lowest latency and is usually used for high availability. SQL Server Replication in a Nutshell Transactional replication o High transaction volume o Low data latency required o Mixed technologies: Integrates with other DBMS Merge replication o Bi-directional data changes o Typically server-to-client 9
Snapshot replication o Large, infrequent data changes o Data change latency OK o Best for smaller data sets Subscriber databases available for reporting Replicate data subsets Some data loss is possible Periodically validate replicated data For more information, see Improving Scalability and Availability. Replication is supported in all editions of SQL Server. Replication publishing is not available with SQL Server Express or SQL Server Compact 3.5 SP2. 10
Data Recovery Requirements Here s a short matrix that will help to determine the necessary scale of protection for your organization. Requirements Backup and Recovery Log Shipping DB Mirroring High-Performance DB Mirroring High-Protection DB Mirroring High-Availability Failover Clustering Cost Low Low/Med Medium Medium Medium High Relative complexity Low Low Medium Medium High High Data loss Possible Latest log Possible None None None Scope of duplication Database Database Database Database Database DBMS Failover Downtime Downtime Manual Manual Seconds Up to minutes Client redirect Manual Manual Automatic Automatic Automatic Automatic Rolling upgrades & maintenance Access data on secondary Geographic separation No No OS & DB OS & DB OS & DB OS Restore Read-only Snapshot Snapshot Snapshot No OK OK OK Latency? Latency? Latency? 11
Considerations for Administrators Use identical server hardware, when possible Design network redundancies, when feasible o Consider network latency for geographic separation Always manage through virtual cluster, not individual cluster nodes Retest failover/failback/recovery after High Availability maintenance and setup Diagnose after failover o Repair alternate node o Resynchronize data, as necessary o Be aware of primary/secondary locations o Ensure application services are connected and functioning properly Keep server node configurations synchronized: o Service pack and patch levels o Duplicate non-redundant resources o Jobs; logins and permissions; OS & sys objects 12
SnapComms Recommendations Low Budget/Resource: DR using Database Mirroring Two sites: Primary and DR location SQL Server database mirroring between sites witness (optional) node A node B Local Storage local sys DBs source user DB Local Storage local sys DBs mirror user DB Medium Budget/Resource: Cluster Database Mirroring Two sites: Primary and DR location Separate failover clusters at each site SQL Server database mirroring between sites witness (optional ) failover cluster at site A failover cluster at site B node A1 node A2 database mirroring node B1 node B2 Shared Storage A local sys DBs local quorum source user DB Shared Storage B local sys DBs local quorum mirror user DB 13
High Budget/Resource: SAN-Based Mirroring Two sites: Primary and DR location Four-node failover cluster; one virtual IP address SAN-based mirroring between sites Manual cluster failover failover cluster nodes at site A failover cluster nodes at site B node A1 node A2 node B1 node B2 Shared Storage A system DBs quorum user DBs storagebased mirroring Shared Storage B system DBs quorum user DBs 14
References Books Microsoft SQL Server 2008 High Availability with Clustering & Database Mirroring by Michael Otey, 2009. Microsoft SQL Server High Availability by Paul Bertucci, 2004. Pro SQL Server 2005 High Availability by Allan Hirt, 2007. Pro SQL Server 2005 Replication by Sujoy Paul, 2006. Pro SQL Server 2005 Service Broker by Klaus Aschenbrenner, 2007. The Rational Guide to SQL Server 2005 Service Broker by Roger Wolter, 2006. Presentations Microsoft Load Balancing and Clustering http://ce.sharif.edu/courses/84-85/2/ce317/resources/root/lecture%20slides/ 14.%20Microsoft%20Load%20Balancing%20and%20Clustering.ppt SQL Server 2005 High Availability http://www.atlantamdf.com/presentations/atlantamdf_111207ha.ppt High Availability Technologies In SQL Server 2000 And SQL Server 2005 http://202.181.238.2/hk/teched2004/ppt/day_2_rm407/dat431(1330-1445).ppt Meeting the Availability Challenge http://download.microsoft.com/download/e/d/c/edcf54db-19cd-4882-9fc4-4f7d46fceaa6/highavailability.ppt Disaster Recovery Mistakes http://www.sqlsig.org/oct%2011%20dassug%20-%20jason%20hall%2010-11- 07%20MM.ppt SQL Server 2005 High Availability http://blogs.msdn.com/sql2005event/attachment/564303.ashx Effective Usage of SQL Server 2005 Database Mirroring http://www.sqlserver-qa.net/ssqa- Effective%20Usage%20of%20SQL%20Server%202005%20Database%20Mirroring_sho w.ppt Articles Achieve High Availability for SQL Server http://technet.microsoft.com/en-us/magazine/cc162477.aspx Geographically Dispersed Clusters in Windows Server 2003 http://www.microsoft.com/windowsserver2003/techinfo/overview/clustergeo.mspx Restoring file and filegroup backups http://support.microsoft.com/kb/281122/en-us Restoring specific tables or rows from backups http://support.microsoft.com/kb/321836/en-us Maintaining Availability During Upgrades http://msdn.microsoft.com/en-us/library/ms191449.aspx 15