Overview of Data Replication 1
Abstract Replication is the process of making a copy of something, or of creating a replica. In different contexts, such as art (copies of a painting), bioscience (cloning), or even furniture and autos (recreating a classic), replication can mean many different things, with varying levels of exactness and timeliness. The same is true in the world of data protection: there are several forms of replication, each with its own features and value propositions, and each with its own relative shortcomings. This paper will define the different types of data replication, and discuss the relative strengths and weaknesses of each, enabling you to make an informed decision on which technology is best for your particular needs. For completeness, we will also touch on a couple of common data protection alternatives to replication, and finish with a description of the Overland Storage Snap Enterprise Data Replicator (Snap EDR) solution. Definitions Synchronous Replication: Data written or changed on a primary storage device is copied to a secondary, or backup, device at the same time, block by block. If the data isn t committed to the secondary storage device, it doesn t get committed to the primary device and an error occurs in the writing application. Asynchronous Replication: Data that is written or changed on the primary storage device is sent to the secondary storage device, but the timeliness and/or successful completion does not impact the primary storage device or the writing applications. Real-Time Replication: All new data and all changes are captured for replicating as they occur, and are transferred to the secondary device, either synchronously or asynchronously. Point-In-Time Replication: New and changed data is transferred to the secondary device on a periodic or scheduled basis, and therefore this type of replication can only occur asynchronously. Real-Time Synchronous Replication Also known as mirroring, synchronous replication is primarily used in data center environments to make an exact duplicate of critical tier-one data for the purpose of fast recovery should the primary storage device fail or otherwise become unavailable. The value proposition of synchronous replication is RPO = Zero : the Recovery Point Objective is no lost data. Due to the nature of the data and the applications it supports, synchronous replication is usually accompanied by automated failover capabilities to automatically switch access to the secondary device quickly and without operator intervention. With synchronous replication, every byte written to the primary storage device must also be written to the secondary device immediately before the write operation is considered complete. If the secondary copy fails, the primary copy is rolled back so that the two copies remain in-sync. All real-time replication solutions copy data as it is changed, requiring a high amount of network bandwidth to process the typical give-and-take that occurs while files are open and being modified. Since changes are committed to the primary disk, they are also committed to the secondary disk, so there is no way to roll back to a known-good version of the data should it be changed or deleted by accident, or corrupted by a virus. Another solution that makes point-in-time copies of the data, such as backup or snapshot, is needed for this level of data protection granularity. Synchronous replication works well in high-bandwidth network environments, such as a Fibre Channel SAN or MAN. But when the primary and secondary devices are separated by more than a few miles, network latency can cause serious performance problems. And at WAN distances (hundreds to thousands of miles), synchronous replication simply won t work without an intermediary solution. Benefits of Real-Time Synchronous Replication: RPO: Minimal data loss from a local disaster or device failure Recovery time objective (RTO): Continuous operation of critical applications; quick recovery with automatic failover Little or no management: Continuous replication process runs in lights-out (unattended) mode 2
Typical Use Cases: Data center type environment for top-tier applications and data Protection of databases and other highly transactional applications Local and near-local disaster recovery; secondary data center in the same metro area Business continuity during scheduled maintenance of the primary storage system: secondary system acts as the primary Relative Drawbacks: File-level data protection: If a file is accidentally or maliciously deleted on the primary storage device, it will be immediately and irrevocably deleted from the secondary device Network efficiency: Requires high bandwidth and low latency; performance degrades when the storage devices are separated by more than a few miles Scalability: Typically arranged in one-to-one or one-to-two configurations Flexibility: the secondary device usually must be exactly the same as the primary, limiting vendor and configuration flexibility Cost: Synchronous is usually the most expensive type of replication software; also, synchronous replication occurs between two duplicate systems usually relatively expensive enterprise-class RAID Real-Time Asynchronous Replication Like its synchronous cousin, real-time asynchronous replication is used to protect organizations from the loss or unavailability of a primary storage device. However, since the new or changed data does not need to be sent to the secondary device at the same time it is written to the primary disk, asynchronous replication is not bound by the same bandwidth, latency, and distance constraints of synchronous replication. Therefore, asynchronous replication can be applied over wide area networks. Assuming there is enough network bandwidth to support the amount of data to be transferred, the two ends of the replication pair can be physically located anywhere a network connection is available. This higher level of network efficiency also allows several asynchronous, real-time replication solutions to copy data in a distribution (one-to-many) or aggregation (manyto-one) model. There is at least one product on the market that gives the customer the choice of sending the replicated data in a continuous manner, or on a scheduled basis. This feature can help to reduce network traffic during high traffic hours, but does not reduce the amount of data to be transferred: all new and changed bytes are cached until the scheduled transfer time, and then sent. Benefits of Real-Time Asynchronous Replication: RPO: Minimal data loss from a local disaster or device failure RTO: Quick recovery of critical applications Little or no management: Continuous replication process runs in lights-out (unattended) mode Scalability: Can be scaled to hundreds or even thousands of servers Typical Use Cases: Long distance disaster recovery Data aggregation and distribution Relative Drawbacks: File-level data protection: If a file is accidentally or maliciously deleted on the primary storage device, it will be immediately and irrevocably deleted from the secondary device Network efficiency: Sends the same amount of data as synchronous replication Flexibility: Systems may be of different types or models, but they usually need to share the same operating systems. Most real-time asynchronous replication products support only Windows operating systems. N80-DS0607-02. 3
Automated failover: The long distances and security issues involved in wide area networks makes automated failover almost impossible when a primary system dies. User accounts, access rights, firewall configurations, etc., all need to be manually adjusted when moving business to a remote backup server. Some real-time asynchronous replication vendors offer automated failover as an option, but only when the backup server is on the same local network (LAN) as the primary server. Point-In-Time Asynchronous Replication Point-in-time asynchronous replication offers an excellent compromise between systemlevel and file-level protection. Replication jobs are run on a scheduled basis, bringing the backup server up to date at preset intervals and based on rules that can be tailored to each type of data that needs to be protected. In this sense, point-in-time replication acts more like a backup than a mirror, except that it stores the copied data in its original format so it can be accessed immediately without the need for recovery and re-building. Multiple versions of each file can be saved. And there is no need to maintain similar hardware or operating system platforms on each side of the transfer, allowing for protection of heterogeneous environments. Benefits of Point-In-Time Asynchronous Replication: File level data protection: Users are able to recover from an accidental or malicious file deletion or corruption simply by accessing it from the remote backup server. Versioning: Some vendors provide the capability to store multiple versions of each file to allow a rollback to a version from a known good point-intime. Network efficiency: Only the differences between the original and replicated file are transferred, dramatically reducing the traffic on the network. And since the replication job is scheduled, the percentage of bandwidth to be used (a.k.a. bandwidth throttling) can also be set based on the time of day and day of week when replication occurs. Immediate recovery: Replicated files are stored in original format, enabling immediate access with appropriate user authorization; recovering from a lost system is the same as for other replication types the replica system is synchronized with the repaired or replaced system. Typical Use Cases: Remote backup consolidation Wide area content distribution Long distance disaster recovery Relative Drawbacks: RPO: There is the possibility of losing any new data and changes that occur during the time between replication jobs. This drawback can be limited by running the jobs more often for applications and data types that have a low tolerance to loss. Automated failover: The long distances and security issues involved in wide area networks makes automated failover almost impossible when a primary system dies. User accounts, access rights, firewall configurations, etc., all need to be manually adjusted when moving business to a remote backup server. Some point-in-time asynchronous replication vendors offer automated failover as an option, but only when the backup server is on the same local network (LAN) as the primary server. Alternatives to Replication There are a lot of things that can happen to data and the systems that store them, from a simple accidental file deletion all the way to a catastrophic destruction of equipment and facilities. Organizations need to evaluate the value and criticality of each application and type of data in its systems, and for each, determine the tolerance for data loss and the length of time they can afford to be unavailable. They also need to take into account the amount of money and the amount of network bandwidth available to get everything back up. There are a large number of data protection technologies available to address these problems. None of them covers everything, no matter how much money you have to spend. But each has its niche a combination of failure types and recovery considerations at which it excels. We ve covered three of these in the discussion above, and will discuss a few more below. N80-DS0607-02. 4
Failure Types Recovery Considerations Data Protection Solutions Site/Region Storage Server Data Types RPO Hot Site Off-Site Mirror Cluster Real-Time Replication Snapshot Corruption RTO Asynchronous WAFS User Error Bandwidth Virtual Tape D2D Backup Virus Budget Archive Tape Wide Area File Services (WAFS): Solutions built on WAFS technology use intelligent caching appliances to provide fast access to centrally stored data for remote users. There are many drawbacks to WAFS, including the need to initially migrate the remote data, a lack of scalability for large organizations, and access problems when the WAN link is down. WAFS does have the unique benefit of providing wide area filesharing capability, with file-locking mechanisms to prevent two users from changing the same file at the same time. WAFS technology does not provide any built-in data protection (it doesn t make a copy of anything usable), but does facilitate backup consolidation by keeping all data in the central site. WAFS also does not work with all applications and data types (especially large files), so some form of data protection capability may still be required in remote offices. Backup Consolidation: There are several companies that provide wide area backup consolidation solutions. Most of these software products started out in life as services, back when the SSP (Storage Services Provider) model seemed promising. One of these companies even retained their services pricing you pay for them based on the amount of data that is sent from the remote locations even though you own all of the equipment involved. These are point-in-time solutions, so there is some potential for data loss, but they provide traditional dial-back capabilities to recover from deleted or corrupted files. Snapshots: Snapshots are a growing form of disk-to-disk backup, providing point-intime copies of data files or blocks, and allowing users to roll back to a previously known-good copy when needed. Typically, the act of taking a snapshot really entails compiling the changes or differences from the last snapshot with pointers to the original data, so that when needed, a good copy of the data can be re-built. Snapshot technology usually refers to local data; snapshot products do not in themselves have the capability to move the data offsite or to a central location for protection from disasters. Disk-To-Disk (D2D) Backup: Traditional tape backup is slowly being replaced by a combination of disk-based backup technologies, with either the backup software recognizing standard disk as a backup target, or by using specialized disk systems that emulate popular tape libraries (called Virtual Tape Libraries VTL). D2D vendors promise faster backup, higher reliability, and above all, fast recovery of data. But they don t meet the need to move at least one copy of the data offsite for disaster recovery, hence the common use of D2D2T (Disk-To-Disk-To-Tape) to satisfy both the need for fast recovery of recent backup data, and long-term offsite using tape. The Snap EDR Advantage: Replication-Plus At its core, Snap EDR has a point-in-time asynchronous replication engine; built around that core is an enterprise-class management, configuration, and reporting platform for building complex remote data management solutions for even the largest organizations. For example, with a single installation of Snap EDR, customers can configure hundreds of individual, policy-based file transfer jobs to: Aggregate files from tens, hundreds, or even thousands of remote sites so that it can be centrally managed and protected Eliminate the need for tape backup in all remote offices, improve data protection reliability and recoverability, and reduce administration costs Perform long distance replication of core storage resources for disaster recovery purposes The devices at each end of the replication can be different, even with different operating systems Distribute files from a central location to remote offices Automatic updates with delivery certification provide compliance with many government regulations, and assures everyone is working from the same data N80-DS0607-02. 5
To accomplish these tasks, Snap EDR includes an impressive array of integrated features for managing, moving and securing remote data: Secure platform: Snap EDR provides three levels of configurable security user access rights, mutual authentication of the servers involved in a file transfer, and data encryption while in transit. Policy-based automation: File transfer jobs can be set up to meet the requirements of the business, automatically protecting highly critical data on a more frequent basis, and non-critical data on a less frequent basis. Centralized control: A single administrator can manage and monitor the transfer of files between thousands of servers distributed throughout the world. Resiliency: Snap EDR responds to temporary network outages based on pre-defined write-order rules developed by the user. Heterogeneous support: Supports GuardianOS, Solaris, MAC OS X, Linux and Windows systems. Network efficiency: Transfers only the net changes in files, at the byte level; and allows end users to choose the amount of bandwidth to be allocated to transfers. Scheduling: Point-in-time replication of files. Scalable: Easily scales from one-to-one to one-to-thousands and/or thousands-toone. Ease of use: Intuitive installation and graphical user interface provides step-by-step data modeling and distribution of tasks. Conclusion With the wide variety of replication and non-replication options available, any size business can find the right data protection solution to meet its cost, bandwidth, scalability, and data access needs. At the high end of the cost and bandwidth scale, synchronous replication provides the very high level of protection required by corporate data centers, but is not well-suited for protecting data distributed throughout a company. Asynchronous replication reduces the bandwidth requirements and allows data to be protected over a longer distance. Because it removes the oneto-one relationship between primary and secondary servers, data can be aggregated and distributed, as well as replicated, although only between homogeneous systems. Many companies may find that their best data protection lies in a new replication-plus breed of solutions that combine the benefits of replication with features that easily extend protection throughout the company, simplify data management, and help the company make better use of its valuable data. About Overland Storage Overland Storage is the trusted global provider of effortless data management and data protection solutions across the data lifecycle. By providing an integrated range of technologies and services for primary, nearline, offline, archival and cloud data storage, Overland makes it easy and cost effective to manage different tiers of information over time. Whether distributed data is across the hall or across the globe, Overland enables companies to focus on building their business instead of worrying about data growth. Overland SnapSAN, SnapServer, NEO and REO solutions are available through a select network of value added resellers and system integrators. For more information, visit www.overlandstorage.com. Worldwide Headquarters 4820 Overland Avenue San Diego, CA 92123 USA Tel 1.800.729.8725 1.858.571.5555 Fax 1.858.571.3664 United Kingdom (EMEA Office) Overland House, Ashville Way Wokingham, Berkshire RG41 2PL England Tel +44 (0) 118.9898000 Fax +44 (0) 118.9891897 N80-DS0607-02. EDR-WP0910-01 6