Effectively Reducing the Cost of Backup and Storage

Transcription

1 DEDUPLICATION: Effectively Reducing the Cost of Backup and Storage by Dr. Srinidhi Varadarajan CTO, AppAssure Software

2 Copyright by AppAssure Software, Inc. ALL RIGHTS RESERVED. No part of this work covered by the copyright hereon may be reproduced or used in any form or by any means graphic, electronic, or mechanical, including photocopying, recording, taping, Web distribution or information storage and retrieval systems without the written permission of the publisher. For permission to use material from this publication, contact AppAssure Software, Inc Isaac Newton Square, Suite 440 Reston, VA

3 DEDUPLICATION: EFFECTIVELY REDUCING THE COST OF BACKUP AND STORAGE Organizations have to backup and store data. Compliance requirements demand that data be stored for periods of time. Productivity demands that data is available in case of disaster or accidental loss of work. Unfortunately, meeting those demands is getting more and more expensive as the amount of data that has to be backed up and stored multiplies. Reducing the cost of backup and storage starts with reducing the amount of data that is backed up and stored. Data deduplication ensures that the same, unchanged files are not repeatedly backed up during nightly processes. Reducing storage capacity alone is a strong argument for implementing data deduplication, but when integrated with server-based backup processes that also include data compression, the combined backup and storage savings are compelling. 3

4 CONTENTS CHAPTER ONE: THE HIGH COST OF REDUNDANT DATA...5 CHAPTER TWO: DEDUPLICATION...7 CHAPTER THREE: IMPLEMENTING DEDUPLICATION...9 CHAPTER FOUR: THE DEDUPLICATION COST SAVINGS About AppAssure About Dr. Srinidhi Varadarajan

5 CHAPTER ONE: THE HIGH COST OF REDUNDANT DATA Data backup and storage is imperative for any business. Not only does lost data equal lost work, compliance mandates require that data be kept for long periods of time. That s why every business has some backup and storage process that takes periodic (daily, weekly, etc.) snapshots of data from any number of business systems including Microsoft Exchange and SQL Server, corporate websites, financial and human resources systems, and more. The big question every IT manager has to ask him or herself is: what am I backing up? Chances are, they are backing up the same data - messages that have been loitering in mailboxes for months, sales transactions from weeks ago, patient records that haven't been purged, performance reviews from last year-over and over again. Whatever was in the database and got backed up yesterday got backed up again today and will get backed up again tomorrow and forever more until it's not in the database anymore. All that redundant data leads to mounting costs without yielding any increase in benefit. Cost Contributors The cost of continually backing up the same data is clear with this example from a ComputerWeekly.com article entitled Reduce storage costs through data deduplication. (July 2007). That article considers a 10MB PowerPoint file that has been sent to 100 people as an attachment. When the nightly Microsoft Exchange backup runs, that one file will take up 1 GB of storage space. If that 1 GB of storage (which is just one file repeated 100 times) is backed up every week, it will consume 52 GB of storage space in a year. That simple example shows the multiplier effect of backing up redundant data. And, of course, that multiplier effect works on the costs that go into the creation and use of redundant tape backups. 1. Tape drives Of course, the more data you have to store, the more tape and tape drivers you ll need. You ll pay the hard costs of buying equipment and media and then the softer costs of maintaining hardware and managing the tapes. 2. Backup windows The more data you re backing up, the longer the backup process and the more bandwidth it will consume. As you require more and more time to backup larger and larger amounts of data, your organization will experience increased periods of downtime. Backups can be scheduled for nights or weekends but again, as data expands, backups can begin to ooze into the morning or work week. 5

6 3. Physical space Of course, as the number of tapes increases, the space to store them has to increase. And because data is being kept for disaster recovery and compliance, the space has to be secure which adds more overhead. 4. Data recovery and restoration Mounting tapes in order to restore data is always time consuming and the more time it takes to recover lost work, the more productivity is lost. Data recovery and restoration takes even longer when shelves of tape have to be investigated in order to find the data that s needed from the right point in time. 6

7 CHAPTER TWO: DEDUPLICATION Employing data deduplication as part of the backup process eliminates redundant data by referencing duplicate copies of data (think of that PowerPoint file) to a single reference copy and by saving only data blocks that have changed since the last backup. Since only a small percentage of data changes every day, deduplication can result in huge up to 90% - savings in storage capacity costs alone. How Deduplication Works Deduplication systems segment incoming data in this case backup data into blocks and compute a unique checksum or hash mark over each block. That allows the system to compare each block s checksum to those previously backed up. If an incoming block s checksum is unique, it is backed up. If an incoming block s checksum matches that of a previously backed up block, it will be referenced to the previously backed up block. Space requirements are reduced by backing up only unique data blocks. Deduplication systems can be accomplished at the file level or block level. The block level, as described above, is more granular than the file level. Deduplication operations may be performed online (duplicates are detected during the process of creating a backup) or offline (deduplication happens after the backup process by reading the backup file and eliminating duplicates). File level deduplication File level deduplication eliminates identical copies of whole files. Rather than marking a data block with a checksum, it marks the entire file. A file is only backed up when its checksum does not match any file previously seen by the deduplication system. File level deduplication is sometimes called single instance store. File level deduplication can be effective for read-only data but read-write data requires more granularity. Consider this example: a database file is backed up every night. If only one record in that database changes during the day, the checksum of the database will change. The deduplication system will see it as a new, unique file and a backup of the entire database will be created. The changed information will account for a tiny percentage of the space used by that mostly redundant backup. Block level deduplication Block level deduplication provides the granularity that s needed to drastically reduce backup overhead. It divides a volume into fixed or variable sized blocks and a checksum is applied to each block. Unique blocks are backed up and redundant blocks are replaced by references to a previously backed up file that matches. The smaller the specified block size, the less redundant data is backed up and stored. Of course, when block size is very small, more checksum matching has to be done. 7

8 Block level deduplication can be problematic when operations like insertions cause data shifting. Let s say a record is inserted into a file. That causes subsequent unmodified data blocks to shift. If the size of the record is not an even multiple of the block size used by the deduplication system, it will be recognized as a new block. Redundant data will be backed up. Online deduplication Online deduplication systems operate between the backup system and the nearline storage device. As data moves from the system to the device, deduplication works on the fly reducing both storage capacity and storage bandwidth. Offline deduplication Offline deduplication systems process backups after they have been written to a disk. Duplicate data is detected within and across backups at either the file or block level and replaced with references. The offline approach has two drawbacks. First, the backup has to be stored before deduplication, requiring temporary storage capacity to be available. Second, backup files have to be read, duplicated, and the new deduplicated backup written to disk which results in high disk bandwidth requirements. 8

9 CHAPTER THREE: IMPLEMENTING DEDUPLICATION Deduplication can be accomplished with software or a hardware appliance. Since a large part of the deduplication cost saving benefit comes from eliminating unnecessary equipment, we ll focus on a software solution; Replay. Replay integrates backup, data compression, and data deduplication to eliminate data redundancy. And, backups are sent to disk (on dedicated or virtual servers), rather than to tape which further reduces to overall storage footprint and increases the speed of backup operations. Employing online deduplication, Replay maximizes nearline storage performance and avoids reading and writing backup files with additional storage. Storage capacity is minimized by combining incremental backups and data compression with fine-grain block level deduplication. What is fine-grain block level deduplication? Most data deduplication systems use block sizes ranging from 4K to 128KB. Replay uses smaller sector and subsector blocks for duplicate detection. That means that redundancies are found at the individual record level. The checksum computation and deduplication algorithms have been carefully architected and heavily optimized to support such fine-grain blocks. Fine grain blocks also control for data shifting as inserted records are invariably larger than sub-sector blocks. Rather that conducting one time backups once every day, once every week Replay incrementally backs up data. Backups might run as often as every 15 minutes. In that time frame, only a relatively very small amount of data has changed. Most of the data is redundant. That efficiency and Replay s engineered optimization makes backups run very fast with negligible, usually unnoticeable, performance impact. In fact, nearline storage runs at native storage speeds. Deduplicated and backed up data is then compressed for even greater storage savings. 9

10 CHAPTER FOUR: THE DEDUPLICATION COST SAVINGS On its own, data deduplication provides savings by simply reducing the cost of storage. When deduplication is integrated in a complete backup operation, the savings are even more significant. While cost savings calculations vary depending on the amount of data you re currently backing up, the amount of data you re storing, the condition of your equipment, etc. it s easy to see that backup and storage overhead is significantly reduced when: systems do not have to be taken down in order to initiate and run backup processes backups run incrementally and continuously without requiring hands on management or oversight backed up data is retrieved from disk over high speed network connections backed up data reflects very current points in time and all changes to date protecting against lost work virtual servers further reduce the backup and storage footprint It is imperative that organizations backup data. Compliance and productivity demand it. Fortunately, disk-based backup with integrated data deduplication and compression allows organizations to meet those demands with great efficiency and cost-effectiveness. 10

11 About AppAssure AppAssure, the No. 1 backup for VM, physical and cloud environments, is a global leader in complete server, application and data protection. The company provides a unified and integrated backup and replication software that is engineered on innovative and groundbreaking technologies which offer near-zero recovery time, 100% recoverability assurance, and cross-platform virtual and physical server recovery (P2V, V2V, V2P or P2P). Building on a history of innovation and multiple industry firsts, AppAssure continues to set the standard in virtual and physical server data protection. About Dr. Srinidhi Varadarajan Dr. Varadarajan received his Ph.D. in Computer Science from the State University of New York, Stony Brook in He is the Director of the Center for High-End Computing Systems at Virginia Tech and an Associate Professor in the Department of Computer Science. Dr. Varadarajan is the recipient of the ComputerWorld Honors Award in the Science Category 2004, the CAREER award from the National Science Foundation, the New Century Technology Council Technology Innovation Award and a Faculty Fellow award from the College of Engineering, Virginia Tech. In 2004, he was honored by MIT Technology Review as one of the top 100 (TR100) young innovators transforming technology. Dr. Varadarajan is the architect of System X, the 2200 processor System X supercomputer located at Virginia Tech. AppAssure Software, Inc Isaac Newton Square East Ste 440 Reston, VA fax phone 2009 AppAssure Software. All rights reserved. The information contained in this datasheet is subject to change without notice. AppAssure Software assumes no responsibility for any errors that may appear. AppAssure Software and the AppAssure Software logo are all trademarks of AppAssure Software. All other trademarks and registered trademarks are the property of their respective owners. 11