Data Mirroring Alternatives For Network Storage Pools

Transcription

1 Data Mirroring Alternatives For Network Storage Pools This white paper discusses various data mirroring alternatives available to customers of Powered by DataCore network storage pools. It suggests a preferred approach to a given data access requirement by considering the basic capabilities and limitations of each option. Starting with locally mirrored copies, the discussion entertains stretching these mirrors over increasingly longer distances. Availability, performance and economic aspects influencing the choice between real-time (synchronous) and store-and-forward (asynchronous) mirroring techniques are also presented. Throughout the document, the unique attributes of DataCore s host- and storage-independent mirroring selection will be explored, as will the compelling reduction in Total Cost of Ownership (TCO) and unprecedented flexibility attainable from advanced storage virtualization. Why Mirror There was a time when data was mirrored for only one reason to protect against a disk failure. Later, it became evident that having more than one copy of data was beneficial in many other ways. Today, with the relatively low cost of online disks, mirrored data volumes are becoming an attractive method to ensure continuous or nearly continuous data access in any of the following scenarios: Basic Failure Protection Mirrored copies are typically used to shield critical applications from equipment and operator errors. The extra copies act as hot-standby images for automatic fail-over mechanisms so that the users never sense a glitch. Example of these conditions include: Disk failures Subsystem failures Path or link failures To achieve application transparent switchover, the mirrored copies must be kept in lock step with the original online data. Generally that implies real-time synchronous mirroring more on this later. Planned Outages With good planning and well-rehearsed procedures, mirrored copies can also insulate users from planned equipment outages. Manually triggering the automatic fail-over mechanisms used for failure protection will just as easily divert the workload to standby resources during: Maintenance activities Upgrades Disaster Recovery In disaster recovery scenarios, where multiple failures have catastrophically impacted business operations, the users may experience some downtime. The objective of data mirroring under these conditions is to contain the downtime to a few minutes or a few hours, as part of an expedited restoration plan. 1

2 Data Migration Mirroring facilities are very powerful substitutes for server-centric file transfer and backup / restore methods traditionally used to migrate data from one array or one location to another. The online image is copied in the background to the new destination. Once a convenient point of synchronization is reached, the copy may be disassociated from the original image and treated as a stand-alone resource. The older equipment may be decommissioned once the new environment is fully verified. Local Access Of Distributed Data Perhaps the most recent facet of IT operations to benefit from data mirroring services are those striving to place largely distributed data in close proximity to their users. Certainly the Internet boom has brought this paradigm into focus with the emphasis on locally cached copies. The most common model relies on a single writer whose updates are mirrored to multiple sites. Known Good Working Images The underlying assumption in all the above scenarios is the notion that mirrored copies represent known good working images. This usually means not only selecting a few files to be mirrored, but also ensuring that all the objects (operating system, registry entries, application data structures, scripts, instructions, etc.) that make up the known good working set are replicated. Sometimes mirroring techniques are combined with snapshot technologies to achieve these objectives. Many Powered by DataCore solutions offer snapshot options that periodically freeze the mirrored image at a known good point in time such as prior to a software or service pack upgrade. Economics Of Managing Multiple Copies Despite the broad applicability of data mirroring and the inexpensive nature of disks, the high cost of managing and maintaining updated mirrored copies often precludes their use. Justification has been further hampered by the differences in mirrored storage administration from one vendor to the next and the excessive costs involved in many of the premium priced products. It is for this reason that DataCore approaches data mirroring as an intrinsic property of uniform storage management for network pools, rather than as a point product tied to a specific OS or a given disk array. At the same time DataCore recognizes that familiar ways die hard, and so the alternatives presented seek to leverage known operating procedures and legacy product features. Mirroring For The Masses The design point and pricing structure for DataCore s offering encourages mirroring for the masses. No longer does the IT staff have to do without these services to live within their budget. In simple terms, Powered by DataCore solutions have proven over and over that customers can do much more with less, by using a common, secure, centralized management approach that encompasses the mix of brands and suppliers typical to many organizations. The affordability mantra extends to remote mirroring as well. Distribute Access, Centralize Management, Fuel Transactions Many of DataCore s customers share the opinion that the best way to achieve processing scalability with adequate and deterministic response time is to fan out multiple instances of data where it is most needed. They see data mirroring as fuel for the engine needed to meet the everincreasing transaction demands driving corporate revenues. These critical transaction-oriented workloads must be fed by subsystems capable of higher and higher input/output operations per second (IOPS). 2

3 Such predictable performance levels are achievable in large scale only by allowing concurrent I/O threads to run undisturbed against separately mirrored and cached copies of data. Network storage pools are the proven way to accomplish these objectives. Mirroring At Its Simplest An easy way to achieve a mirrored copy is to have applications write the same information twice, once for each of two disks. In practice host-based operating system utilities, volume managers and RAID controllers as well as external RAID subsystems relieve applications of this duty (see Figure 1). All of these techniques may be used in a Powered by DataCore configuration. The virtual volumes served by the network storage pools simply appear as well behaved logical disks, but offer a great deal more flexibility. Generally, the paths traveled by the mirrored I/O streams behave very similarly, so the time to complete both I/Os is roughly the same. Reads are taken from a pre-defined preferred path a relatively arbitrary decision given similar response time characteristics. Figure 1: Different Approaches To Simple Local Mirroring Stretching The Separation Intuitively one could obtain remote copies simply by stretching one of the paths to reach a distant storage pool. SCSI and Fibre Channel extenders do just that. They are inserted into the link between the initiator system and the target device as nearly transparent cable extenders. Some of these units passively amplify the signal to overcome transmission losses incurred over the longer distances. Other products encapsulate or translate the I/O stream into an intermediate protocol better suited for wide area connections. These cable stretchers fulfill the general requirement for long distance connectivity but have no real sense for the end-to-end state of remotely mirrored information; especially when transmission or mirroring failures occur. Only the originating (initiator) end can determine if the remote copy is up to date or stale. Therefore, directly accessing the remote copy could lead to unexpected and incorrect results. Figure 2 - Cable extenders to increase separation an incomplete solution 3

4 Distributed Data Replication To better ascertain the state of a remote copy, mirroring intelligence can be placed at both ends of the connection. In this way, additional coordination between the originator and the destination allows one to determine if the copy is synchronized or if it has fallen behind. The approach is sometimes referred to as data replication. Typically the replicated I/O travels over a completely separate path from the local copy with potentially very different performance characteristics. Usually the most direct path is co-located with the originating subsystem and considered primary, while the lengthier indirect path with longer transmission and handling latencies is defined as secondary. (See Figure 3) The distributed mirroring processes take responsibility for keeping the second copy updated. At the same time they hide the underlying replication activities from the application. Both ends collaborate to inform each other of the remote copy state. The added coordination allows system administrators to better understand the state, and hence validity of the secondary copy before granting users access to them. Figure 3 Distributed data replication between cooperating subsystems Conventional replication techniques over the past years have taken either a host-centric or arraycentric tactic. Host-Centric Remote Replication In the host-centric implementations, the originating mirroring process runs in the same server as the application. The remote destination is usually another application server. The primary and secondary storage devices may be different. Host-based replication is manageable when only a pair of hosts are involved. However, there are obvious downsides when multiple hosts require replication services: The administration burden is proportional to the number of connections and host OS flavors. Though not a problem with one or two servers, it becomes a definite headache with hosts, and a complete nightmare in a large server farm such as those found at Internet Service Providers. The replication processes steal host resources (CPU cycles, memory, I/O bandwidth, I/O slots) from the resident applications potentially degrading their performance. 4

5 Array-Centric Remote Replication The array-centric alternatives move replication off-board onto cooperating intelligent arrays. While these offload the hosts and can replicate on behalf of multiple servers, they suffer three flaws: Most work only with their own proprietary disks The primary and secondary disk controllers must be of the same proprietary make and model Many rely on costly external equipment to convert disk channel protocols into the handshake of common WANs and LANs. Because of the cost and vendor-lock issues, this type of mirroring solutions has been confined largely to the very wealthy consumers. What Has Datacore Done Differently? DataCore considers data mirroring and replication as fundamental services of network storage virtualization. The advanced design integrates local and remote mirroring functions into affordable network storage control nodes known as storage domain servers (SDS). In so doing, the disadvantages of host- and array-centric alternatives have been eliminated while retaining all of their benefits. SDS nodes collaborate not only to consolidate the management of the underlying physical storage devices, but also to securely coordinate various replication activities, over short and long distances. Figure 4 - Remote mirroring between network storage pools 5

6 Uniqueness Powered by DataCore data mirroring solutions enjoy the following unique combination of attributes: Affordable Behave as natural extensions of network storage pools Encompass disk arrays and JBOD from different suppliers Support virtual volumes of arbitrary size Allow some volumes to be replicated to one site and a different set to another Support dynamic substitution of destinations, transparent to the applications Resilient to network failures Stretch over indefinitely long distances Run over common Internet Protocol (IP) WANs and LANs without conversion Can operate directly over campus and extended SANs (metropolitan area networks) Offload replication activities from hosts Operate bi-directionally (either site can act as both initiator and target) Note: Familiar host and array-centric products may continue to be used for specific roles in network storage pools alongside DataCore s advanced mirroring features. New Upgrade And Disaster Recovery Choices The ability to transcend disk array boundaries creates new opportunities for low-impact data migration and cost-effective disaster recovery. For example, many customers shy away from the hardship of replacing disk subsystem suppliers regardless of how dissatisfied they are. Most prefer to prolong the relationship rather than face lengthy downtime and uncertain upgrade results. With DataCore s cross-vendor mirroring facilities, the new equipment can be completely staged with live data in the background, while normal processing goes on. After due testing, the individual applications can be redirected to the replacement arrays by simply removing the original arrays from the mirrored volume relationship. The users never experience downtime. Other customers long for disaster recovery capabilities available from premium priced disk subsystems but know they ll never have the budget to roll out such a plan. They can now obtain these contingency services at a fraction of the cost using their choice of storage arrays at either end of the link without compromising their business continuance objectives. In fact, some accounts use intelligent arrays at their primary site and relegate relatively inexpensive JBODs (just-a-bunch-of-disks) for the remote recovery role. Tradeoffs With several alternatives for replicating data across remote sites, the final choice is largely dictated by tradeoffs between 4 variables: Distance Response time Delivery guarantees Networking costs 6

7 Disaster Recovery and Business Continuance plans, for example, count on rapidly restoring operations from unexpected catastrophes. Ideally they would maintain up-to-the-minute copies of critical online volumes at regional contingency sites. Less time critical data mining and application testing scenarios may have less rigorous replication needs. Asynchronous IP Mirroring Option The most convenient approach for widely separated locations is DataCore s Asynchronous IP Mirroring option, or AIM. AIM operates over conventional Local Area Networks (LANs) and Wide Area Networks (WANs) using the popular Internet Protocol (TCP/IP). The term asynchronous describes the store and forward nature of AIM s behavior. It also hints that a best effort will be made to keep the copy up to date, but long distances and slow connections will result in variable lag between the source and target. With very high-speed networks and short separation, AIM can minimize the differences between the two sites. Synchronous Mirroring Option Certain circumstances make it essential to keep the remote copy in lock step with the primary image. This calls for DataCore s synchronous Network Mirroring option and is generally viable with separations up to 100 kilometers. Synchronous mirroring typically takes place over dedicated high-bandwidth optical routes using the Fibre Channel protocol. Generally, the recurring cost of such links is substantially higher than the shared IP connections used for AIM. All of these product features operate independent of the hosts who generate the data stream and the underlying storage devices where their data is housed. They allow customers to centrally and securely manage remote mirroring services in a uniform manner, despite variations in operating systems and disk arrays. They also amortize the investment in the replication infrastructure over a pool of assets. Guaranteed Delivery vs. Response Time The distinction between synchronous and asynchronous mirroring is better characterized as a choice between guaranteed delivery and acceptable response time. Certainly one s instincts demand guaranteed delivery, but the laws of physics and economics sometimes overrule. Distance, performance and budget are very influential elements in the final decision. As the desired separation between the secondary copy and the replication origin grows, so too do the delay and cost factors. Similarly, lower speed (less costly) connections to the secondary copy can slow down the updates. Different applications and users have different tolerances for delay. Many transaction-oriented users find greater than 2-second turnaround unusable, while other batch users may regularly put up with 10-second waits. Extending Synchronous Mirrors Synchronous behavior guarantees delivery by having the replication service await acknowledgement that the second copy has been updated before reporting I/O completion to the application. The sequence of transmissions is illustrated in Figure 5. 7

8 Figure 5 - Synchronous mirroring sequence 1. Initial I/O issued by the application 2. SDS writes to local disk 3. SDS relays remote request* 4. Remote write performed by secondary SDS* 5. Remote copy confirmed 6. I/O completion signaled In reality, steps (2) and (3) occur simultaneously and step (4) may be cached at the remote end. Asynchronous Replication The asynchronous method anticipates various network-related delays in the transmission to, and acknowledgement from the remote destination. Rather than hold up applications and slow down user response time to unacceptable levels, AIM signals I/O complete immediately after the write has completed over the primary path (see Figure 6). Of course, there is no timely guarantee that the secondary update was posted, thus the tradeoff between response time and guaranteed delivery. At any one time the secondary may not have all the latest updates, but the remote copy maintains an ordered time-sequenced image of past I/Os. Figure 6 - Asynchronous Mirroring Sequence 8

9 Note the differences between the asynchronous sequence below, and the synchronous behavior described on Figure Initial I/O issued by the application 2. SDS writes to local disk 3. I/O completion signaled (don t wait on remote acknowledgement) 4. SDS relays remote request 5. Remote write performed by secondary SDS* 6. Remote copy confirmed *Step 5 may be postponed when the remote write is cached on the secondary SDS. Elastic buffers are maintained on the transmit and receive ends to ensure smooth operation despite network anomalies How Close Can They Keep Up? Figure 7 illustrates the fundamental difference between synchronous and asynchronous mirrors by comparing the contents of a synchronous and an asynchronous copy relative to the source volume over a period of time. Figure 7 - Updates with Synchronous and Asynchronous Mirroring The synchronous mirror remains in lockstep with the source over each successive time period. However on the asynchronous mirror, viewed over the same period, writes that occurred on the source at T 0 may not appear on the mirror until T 2 indicating that the asynchronous mirror lags behind the source. The lag will vary depending on the bandwidth of the interconnecting network and the SDS processing resources available to transmit and receive data. In the case shown in figure 7, the asynchronous mirror doesn't catch up until T 5 assuming no additional writes occur after T 3. 9

10 Network Considerations When Extending Mirrors As one might suspect, different network considerations come into play when operating synchronously and asynchronously. The inter-sds Fibre Channel links employed by synchronous Network Mirrors can take several forms depending on distance requirements. A few guidelines are presented below: 500 meters or less- native Fibre Channel point-to-point connections between SDS Host Bus Adapters will do 10 kilometers or less- use point-to-point long wave laser connections between Fibre Channel switches 100 kilometers or less- use optical extenders (including Dense Wave Division Multiplexers) to stretch the inter-switch link Greater than 100 kilometers- while achievable through IP-based Fibre Channel extenders, the longer separation and protocol conversions introduce latencies that exclude them from many real time transaction-oriented applications. WAN extenders may be more appropriate for less time sensitive situations with potentially lower update rates. Right of Way Public right-of-way issues must also be taken into account anytime the Fibre Channel connection leaves a private campus and traverses a public road. Under these circumstances, dark fiber suppliers must be brought into the picture. These companies have been granted special privileges to tunnel underneath streets, railroads and even rivers. Customers then rent or lease the optical infrastructure for their dedicated use. IP Multiplies The Selection The choice of inter-sds mirroring links is much larger with Asynchronous IP Mirroring. Essentially any network interface card (NIC) capable of running under IP can be used as the native SDS connection. These include the broad range of built-in dial-up serial ports and Ethernet LAN cards, as well as the more sophisticated and powerful Gigabit Ethernet and ATM NICs. The right-of-way concerns no longer dominate the choice of network providers. Instead, network provisioning is much more straightforward, dovetailing in with the other IP services already in use by the corporation. Downstream routers, virtual private networks and encryption services can all be leveraged to enhance the privacy and security of the remote mirroring transmissions. Resiliency And Elasticity By now everyone has become accustomed to the occasional network meltdown or the unexpected backhoe incident where vital communications lines are accidentally severed. For synchronous mirroring, the link failures trigger logging at the originating SDS. The logs are used to incrementally refresh the remote copy when the link is restored. AIM s elastic properties are even better suited to gracefully deal with network anomalies. When the network slows to a creep, AIM patiently collects I/Os at the local SDS node, only to stream them out in order as more bandwidth becomes available. In some cases, the network variations are a regular part of the day heavy contention during peak business hours, interleaved with off periods and very fast transmissions. Again, AIM ebbs and flows to match the network service level. 10

11 Conclusions Powered by DataCore solutions offer a wide range of network storage pooling alternatives capable of addressing the most common mirroring and data replication requirements over short and long distances. The compelling economics also make possible the use of these advanced facilities in fresh new opportunities, bringing both the acquisition and recurring investment in storage virtualization within the reach of the masses. Each mirroring option can be configured to circumvent failures or simply to provide convenient data access at remote locations. The underlying technology combines the best features of conventional replication techniques, without the typical lock-in to a specific disk array or host. And unlike any products on the market today, the replication services are supported on a choice of cost effective and open hardware/software platforms using a central and uniform management GUI. For more information on Data Mirroring and Network Storage Pools contact DataCore Software Corporation at or us at info@datacoresoftware.com DataCore Software Corporation Pub # P069AA 11