TECHNICAL BRIEF: SYMANTEC NETBACKUP DEDUPLICATION GENERAL......... DEPLOYMENT............. GUIDELINES.................. Symantec NetBackup deduplication general deployment guidelines Who should read this paper This document is intended for NetBackup architects who need general deployment guidelines on the various NetBackup deduplication methods available.
Content Introduction........................................................................................................... 1 NetBackup deduplication in VMware environment........................................................................ 3 NetBackup deduplication for LAN connected database servers & file servers................................................. 3 NetBackup deduplication for remote office backups....................................................................... 4 NetBackup deduplication for Network Data Management Protocol (NDMP) environments...................................... 5 NetBackup deduplication using NetBackup appliances..................................................................... 5 NetBackup deduplication with OpenStorage (OST) storage devices.......................................................... 5 Summary.............................................................................................................. 7
Introduction In February 2010 Symantec released NetBackup (NBU) 7.0, and the primary feature of NetBackup 7.0 was deduplication. NetBackup 7.0 introduced the ability to perform deduplication 3 ways: Source based deduplication (via the NBU Client) Media server based deduplication / Target based deduplication (via the Symantec NetBackup Media Server) Available in the software form factor and the appliance form factor (Symantec NetBackup 5200 appliance series) Integration with deduplication storage devices (via the NBU OpenStorage (OST) API) Available through Symantec s own deduplication storage devices (Symantec NetBackup 5000 appliance series), and 3rd party deduplication storage devices Figure 1: Symantec NetBackup Deduplication Options When implementing deduplication in an enterprise environment one size does not fit all ( source based deduplication does not work everywhere, target based deduplication does not work everywhere, and deduplication storage alone does not work everywhere ), therefore NetBackup enables all 3 deduplication methods to offer maximum flexibility with deploying deduplication solutions across an enterprise environment NetBackup deduplication can occur on the NBU client or on the NBU media server, but NetBackup deduplication does not occur when utilizing integration with backend deduplication storage devices supported via NetBackup OST. Instead, NetBackup enables the supported OST storage device to use its own deduplication technology. NetBackup integrates with OST supported deduplication storage devices to not only enable use of the OST appliance s advanced features (i.e. replication), but also to enable additional NetBackup OST feature/functionality not possible with the OST appliance alone (i.e. AIR, GRT, etc). The additional feature/functionality enabled via NetBackup OST are discussed below in the NetBackup deduplication with OST devices section below NetBackup enables global deduplication across both NetBackup deduplication methods (source & target), across all data types (VMware data, Oracle data, Microsoft Exchange data, file server data, NDMP, etc.), and across all system types (physical & virtual) by writing all deduplication data to the Media Server Deduplication Pool (MSDP) the logical backend deduplication storage component within 1
NetBackup. Regardless of the deduplication method being used (client side or target side/media server based deduplication), or the type of system (physical or virtual), NetBackup deduplication utilizes stream handlers to optimize the deduplication. NetBackup stream handlers automatically recognize the type of data that is being backed up (i.e. Oracle, VMware, Exchange, NDMP, etc.) and then using the knowledge of the application and the corresponding backup data stream NetBackup intelligently adjust the deduplication processing to maximize the deduplication of the backup data stream. The bounds of NetBackup deduplication are based solely on the capacity of the NetBackup MSDP. As of NetBackup 7.5 the NetBackup MSDP has a capacity of 64 TB of storage, and depending on the deduplication rate of the data being backed up (i.e. 50x), a single NBU MSDP can protect well beyond the capacity of the NetBackup MSDP (64-TB MSDP x 50x dedup rate = 3.2 PB). Because NetBackup enables maximum flexibility with deduplication methods, and offers global deduplication across source based deduplication & target based deduplication NetBackup solution architects often ask, Which NetBackup deduplication method(s) should I use in my enterprise environment?, and the answer is, It depends. The purpose of this whitepaper is to provide general guidance on when to use each of the NetBackup deduplication methods available. The details of how the NetBackup deduplication technology works is beyond the scope of this document, but can readily be found on-line at http://www.symantec.com/netbackup. In general, NetBackup client side deduplication should be used whenever possible. NetBackup client side deduplication has several unique key benefits over target side deduplication and stand-alone deduplication storage solutions, specifically that NetBackup client side deduplication has significant benefits to various downstream backup resources. NetBackup client side deduplication: Distributes the deduplication workload across clients to eliminate any "hot spots" of CPU processing and utilizes otherwise spare CPU cycles. Reduces the load on the network since less data is transferred during the backup process. Reduces the overhead on the NBU media servers since less data equals less I/O. Reduces the amount of storage necessary to store the backup data. NetBackup client side deduplication enables spreading the deduplication processing overhead across the enterprise to eliminate any hotspots. By using NetBackup client side deduplication customers can spread the deduplication workload across all systems across the enterprise, and not rely on a single device to do all of the deduplication processing. Customers deploying NetBackup client side deduplication often believe that NetBackup client based deduplication will result in higher overhead on the NetBackup client. While it is true that the first backup done with NetBackup client side deduplication will result in slightly more overhead on the client system, all subsequent backups will actually have much less overhead (an average savings of 95%) when compared to a normal backup of non-deduplicated data. The reason for this is because while there is additional overhead that takes place for performing the deduplication, there will be significantly less overhead necessary for performing the actual transfer of data, and the net result will be less overhead for NetBackup client side deduplication when compared to standard client side backups with non-deduplicated data. When it comes to determining which deduplication method should be used, one size does not fit all. This is true with both NetBackup client side deduplication and target side deduplication. The remainder of this whitepaper will cover common environments within an enterprise environment and provide general guidance on the appropriate NetBackup deduplication method to be used within various environments across the enterprise. It should be noted that all good backup solutions should be architected to meet a well defined Service Level Agreement (SLA), which should correspond to business requirements defined in terms of the necessary Recovery Time Objective (RTO), Recovery Point 2
Objective (RPO), and backup window ( the amount of time allowed to backup the data ). The general guidance provided in this whitepaper does not supersede using an SLA to define a specific backup solution architecture, and so the guidance in this whitepaper should be interpreted as general guidelines, and not strict deployment rules. NetBackup deduplication in VMware environment In general NetBackup target based deduplication should be used with VMware environments, particularly when the VMware vstorage API is being used to enable off-host backups. With VMware vi3 (version 3) VMware introduced VCB (VMware Consolidated Backup), and in vsphere 4 (version 4) VMware introduced the vstorage API for Data Protection (VADP). Both solutions (VCB & vstorage API) are considered best practices by VMware for performing backups of VMware environments. These are considered best practice solutions because each of these VMware solutions is an off-host backup solution that creates a quiesced snapshot of the Virtual Machines (VM s) in the VMware environment. This enables backup applications like NetBackup to perform backups of the quiesced snapshot without impacting the production VM. Details of the VMware off-host backup solutions via VCB & the vstorage API are outside the scope of this document, as are the many other benefits of NetBackup for VMware (i.e. Single pass backup with dual restore capability, without a costly staging area; Single File Restore without a costly staging area; Optimized backups; VMware Intelligent Policies). NetBackup for VMware utilizes VMware's "offhost" backup technologies ( VCB & VADP ) to perform backups and therefore target side deduplication is used. Generic backups of VM's are considered to be "crash consistent" for any applications inside the VM, and do not require an agent inside of the VM. Application aware backups (i.e. hot backup of an Oracle database) are made possible by installing an NetBackup client agent and NetBackup application agent (i.e. NetBackup for Oracle) inside the VM. Despite the presence of these NetBackup agents inside the VM they are not used for data transfer purposes, rather only to interface with the application (i.e. Oracle) to quiesce the application, send meta data, and truncate the application logs. Generic backups of VM's and application aware backups of VM's both use VMware off-host backup technologies (i.e. VCB & VADP) in order to perform off-host backups, and therefore target side deduplication is used with generic backups VM's and application aware backups of VM's. As mentioned above, both generic VM backups and application aware VM backups use VMware's off-host backup technologies (VCB & VADP), and therefore both use target side deduplication. In both cases NetBackup will utilize its knowledge of the content of the VM ( through indexing ) to backup only the used blocks within the VM and skip the deleted & unused blocks, thus significantly reducing the amount of data being backed up and the amount of storage necessary to store the backup data. This feature is called "Optimized backups", and for clarity I have provided the following example... If you have a 100 GB VM and you are only using 70GB of the VM then NetBackup will backup only 70GB, not the full 100GB VMDK. Since VM's cannot / should not run @ 100% capacity there will always be a reduction in the amount of data backed up when using NetBackup for VMware. In addition to NetBackup's ability to skip deleted and unused blocks within a VM NetBackup will use its VMware stream handlers to intelligently deduplicate the data inside the VM ( separating meta data, and data to maximize deduplication ). NetBackup deduplication for LAN connected database servers & file servers In general, when performing backups of LAN connected database servers & file servers client side deduplication is recommended. When backing up database servers a NetBackup agent and a NetBackup application agent (i.e. NetBackup for Oracle) are installed on the database server. These agents automatically work together to not only enable advanced backup features (i.e. Utilize oracle Oracle's RMAN to perform hot backups of the Oracle database), but also utilize the appropriate NetBackup stream handlers (i.e. NetBackup for Oracle stream handler) to perform client side deduplication in an optimal manner by using the appropriate NetBackup stream handlers. 3
Generally speaking, client side deduplication should be used for LAN connected database servers & file servers. This general rule does not apply to SAN connected database servers & file servers since "SAN connected" usually implies the presence of an array based snapshot of the database / file server, along with some kind of "off-host" backup solution that utilizes the SAN connected storage. When "off-host" backups are being done on database & file servers target side deduplication should be used. NetBackup deduplication for remote office backups In general, when performing remote office backups NetBackup client side deduplication should be used. The primary reason for this is because of the many down stream benefits of NetBackup client side deduplication. These benefits include needing less network bandwidth to transfer less data, less data for backup servers to process, and less storage necessary to store the deduplicated data. Client side deduplication transfers significantly less data across the network compared to traditional backups and result in network savings of up to 95%. Remote office backups are generally done across networks with very little bandwidth and high in latency. Deduplicating the data before transferring the data results in far less data being transferred and far less demand / impact on the network being used for the remote office backups. Using NetBackup client side deduplication for remote office backups also proves to be a very cost effective solution for remote backups (often a requirement) since no additional hardware is required and the deployment of the solution is simple: 1. Load a NBU client on the system 2. Add the system to the appropriate NBU policy, and you re done! New in NetBackup 7.5, remote office backups are now support in low bandwidth environments (tested on slow connections, down to 56k), high latency environments (tested up to 4,000 ms), unreliable network environments (tested with up to 10% packet loss), and going through firewalls. This means NetBackup will work well to protect & deduplicate data from servers located in remote offices, and can handle the many issues typically associated with remote office backups. 1 As mentioned in the introduction section, all good backup solutions are architected to meet a specific Service Level Agreement (SLA), and architecting a remote office backup solution is one that certainly requires a well defined SLA. When remote office backup solution requirements include a strict Recovery Time Objective (RTO) it is often necessary to add a NetBackup media server local to the remote office to enable speedy recovery times of the localized backup images. A localized NBU media server ensures that restores can be performed in a manner that meets the RTO portion of the SLA, and also opens up the possibility of using target based deduplication ( via the NetBackup media server ). In these instances the remote office can then be treated as a local office and the various guidelines provided in this whitepaper can be applied to the remote office as if it were local since there is a NetBackup media server local to the environment. In environments where customers require a localized media server in the remote office, but also need copies of backup images to be sent to a remote site (i.e. centralized data center ), NetBackup offers several replication with deduplication features. These include: Optimized Duplication (Opt Dup) for duplicating backup images within a single NetBackup domain from one OST device to another. Automatic Image Replication (AIR) for replicating backup images between multiple NetBackup domains / NetBackup masters, again from one OST device to another and usually for DR purposes Both of these OST replication solutions use a form of deduplication to replicate only unique data segments between OST devices, thus reducing the impact on the network that connects to two sites ( See the OST section below for more information on these solutions ). 1-Symantec NetBackup 7 Dedupe Update: NBU 7 Client Dedupe Support for Remote Office Backups. TECH 136354, January 14, 2010 4
NetBackup deduplication for Network Data Management Protocol (NDMP) environments When backing up NDMP environments the only NetBackup deduplication method choice that is available is target side deduplication via the NetBackup Media Server. An NDMP environment implies a third party network attached storage device without a server available to load the NetBackup client; therefore client side deduplication is not an option and target side deduplication via the NetBackup Media server must be used. When the NetBackup Media server performs the backup of the NDMP environment the NBU Media server utilizes its stream handler technology to optimize the target based deduplication from the NDMP device. NetBackup deduplication using NetBackup appliances Customers are often looking for a turnkey backup solution, and Symantec offers NetBackup appliances as an alternate form factor for NetBackup's many industry leading features for the data center, remote offices, and virtual environments. Symantec currently offers two NetBackup appliance series, the Symantec NetBackup 5000 series appliances and the Symantec NetBackup 5200 series appliances. The NetBackup 5200 series appliances are an all-in-one solution with "NetBackup in a box". The 5200 series appliances can be configured to be a NetBackup Master server and/or a NetBackup Media server, with a current backend storage capacity of 4-64TB (See the Introduction section of this document for details on how much data can be protected with and NBU MSDP with 64TB of backend storage). The NetBackup 5200 series appliances are an ideal solution for customers looking for a single appliance that contains a backup server, the backup software pre-installed, and the deduplication storage, all-in-one. The NetBackup 5000 series appliances are ideal for increasing the backend storage capacity of a NetBackup environment up to 192TB. The 5000 series appliances can be used as backend storage for a software based NetBackup environment, a 5200 series based environment, or both. Key Features of NetBackup appliances: The NetBackup appliances share many common features: Simple - Easily installed in minutes Scalable - Modular capacity, depending on data center needs Flexible - Source and/or target deduplication, a solution for the datacenter and/or remote office backups, a solution for physical and/or virtual environments Reliable - Enterprise class hardware & software NetBackup deduplication with OpenStorage (OST) storage devices Enterprise environments often include deduplication storage devices (i.e. Data Domain, ExaGrid, FalconStor, Fujitsu, HP, IBM, Quantum, Sepaton, etc), and customers who have invested in these deduplication devices often want to utilize the deduplication capabilities of the storage device that they ve already purchased. NetBackup has the flexibility to integrate with these deduplication storage devices through an API called OpenStorage (OST). The details of how the NetBackup OST API works and the many features that are available are beyond the scope of this document, however this information is readily available in the NetBackup hardware compatibility list 2. The NBU OST API enables NetBackup to not only utilize the deduplication storage device s backend deduplication capabilities, but also to manage the backup image on the deduplication storage device like any other NetBackup backup image. This implies that the backup image 2-NetBackup 7.x Server / Enterprise Server Hardware Compatibility List (HCL). Technote 76495. 5
on the deduplication storage device has been indexed by NetBackup and that NetBackup is aware of the contents of the many backup images that reside on the deduplication storage device; furthermore OST supported devices can leverage the many OST features that NetBackup makes available to OST supported devices, including the following features: Accelerator Utilizes Change Tracking technologies to minimize the overhead associated with processing changes in the data to be backed up, and reduces the total amount of data for deduplication processing - ultimately reducing backup times by as much as 100x. While the Accelerator feature alone is not a true deduplication technology, it does contain several data reduction technologies and is often used in conjunction with NetBackup client side deduplication to optimize the data deduplication and achieve the fastest possible backup along with the many benefits of client side deduplication. The NetBackup OST Accelerator feature is ideal for environments that have small backup windows, and to optimize client side deduplication. Automatic Image Replication ( AIR ) Enables OST devices to create and send copies of NetBackup backup images directly between OST devices across multiple NetBackup domains / NetBackup Masters. "AIR" utilizes a modified version of the OST Optimized Duplication technology, described below, to deduplicate data being transferred and to send only the unique data segments. "AIR" is often used to enable DR for NetBackup backup images since AIR enables multiple NetBackup domains / NetBackup Masters ( usually geographically dispersed ) to have duplicate copies of NetBackup backup images. Direct to Tape Enables OST devices to write NetBackup backup images directly to tape without having to be processed through/by the NetBackup media server. Direct to Tape enables the OST device to have instant compatibility with all NetBackup supported tape devices regardless of the OST device lacking any abilities to write to any tape devices without OST. Granular Recovery Technology Enables OST devices to store NetBackup backup images with GRT functionality. Specifically, the OST GRT functionality enables OST devices to create a single NetBackup backup image of a Microsoft application (Exchange, SharePoint, SQL, Active Directory) with the ability to perform restores of an entire MSFT application/database (i.e. Exchange database) and/or Microsoft application/database component (i.e. Single mail message), with both restore types being done from the same single backup, and without creating any copies of the data nor the use of a staging area. For example, create a single NetBackup backup image of an Exchange environment on an OST device, and from that single NetBackup backup image be able to restore both the Exchange database and/or a single mail message, without the use of a staging area nor making any copies of the data. Optimized Duplication - Enables OST devices to create copies of backup images across NetBackup media servers (within a single NetBackup Domain / under a single NetBackup Master), by sending only the unique data during the replication / duplication process. Optimized Synthetics Enables OST devices to create synthetic full backups with minimal transfer of data (and potentially no data transfer), by utilizing existing data segments that are local to the OST device and then moving pointers to existing data segments already local to the OST device to create a synthetic full backup. Replication Director Enables OST devices (i.e. Storage Arrays) to create array snapshots (i.e. NetApp Snap Manager snapshots) and then have those snapshots indexed and managed by NetBackup. Shared Tape Direct to Tape functionality with shared tape drives. Again, the purpose of this whitepaper is not to explain the details of the NetBackup OST API nor the many NetBackup OST features available, rather to explain that NetBackup enables integration with many 3rd party deduplication storage devices through the NetBackup OST API. Because the NetBackup OST API enables several deduplication type features / functionalities for OST devices, a brief description of the various NetBackup OST features have been included within this section. 6
Summary NetBackup offers maximum flexibility with various possible deduplication solutions including : source based deduplication (through NetBackup client side deduplication), target side deduplication (through NetBackup media server deduplication), and integration with deduplication storage devices (through the NetBackup OpenStorage API). Determining which deduplication solution should be used within a specific environment depends on the details of the environment and the Service Level Agreement of the backup solution for the environment, but general guidelines for appropriate NetBackup deduplication solutions have been created within this document to help NetBackup solution architects determine which NetBackup deduplication solution might be appropriate for the various environments they are looking to deploy NetBackup deduplication. A summary of the various environments and the corresponding general deduplication deployment guidelines are included below, but each section of this whitepaper should be read in order to fully understand the caveats and exceptions to each of the general guidelines provided below. Use case Corresponding NetBackup Deduplication solution VMware environment Target based deduplication ( NBU Media Server ) Database and File Servers Source based deduplication ( NBU client ) Remote Office Source based deduplication ( NBU client ) NDMP environments Target based deduplication ( NBU Media Server ) Existing deduplication storage devices NetBackup OpenStorage (OST) API More Information Symantec NetBackup Enterprise Server Information Contact: John Hilger NetBackup Product Specialist John_hilger@symantec.com Paul Sustman NetBackup Product Manager paul_sustman@symantec.com 7
About Symantec Symantec protects the world s information and is the global leader in security, backup, and availability solutions. Our innovative products and services protect people and information in any environment from the smallest mobile device to the enterprise data center to cloud-based systems. Our industry-leading expertise in protecting data, identities, and interactions gives our customers confidence in a connected world. More information is available at www.symantec.com or by connecting with Symantec at go.symantec.com/socialmedia. For specific country offices and contact numbers, please visit our website. Symantec World Headquarters 350 Ellis St. Mountain View, CA 94043 USA +1 (650) 527 8000 1 (800) 721 3934 www.symantec.com Symantec helps organizations secure and manage their information-driven world with data backup and recovery software. Copyright 2012 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo, and the Checkmark Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries.microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Other names may be trademarks of their respective owners. 9/2012 21264613