1 Data Integrity: Backups and RAID Track SA-E AfCHIX workshop Blantyre, Malawi (Original slides by Phil Regnauld)
2 Introduction Keeping your data safe and reliable TM Backups Types of backups Strategy Tools RAID (Redundant Array of Independent Disks) Types of RAID What type to Use Disk Issues Hardware vs. Software RAID
3 Backup What is backup? backup is part of a larger domain called data security: integrity, protection: cryptography availability, redundancy: mirroring / RAID Why Backup? Software and Hardware failures are a common thing in the computer world. Any number of occurrences can cause loss of valuable data.
4 Backup Types of failures Power failures (software/hardware failure) Natural disasters (fire, flood) Security incidents (theft) Hardware Failures (disk crash) User error (rm -rf) Social issues (stolen data)
5 A common backup strategy Do nothing Not a computer program, but it is the most widely used backup strategy. There are no initial costs. There is no backup schedule to follow. Just say no. If something happens to your data, grin and bear it! If your time and your data is worth little to nothing, then Do nothing ' is the most suitable backup program for your computer. But beware, UNIX is a useful tool, you may find that within six months you have a collection of files that are valuable to you. Do nothing is the correct backup method for /usr/obj, /usr/src and other directory trees that can be exactly recreated by your computer but if in doubt, BACK IT UP!
6 Creating a backup strategy This is driven by many variables. Such as: How long can you be offline before your org disappears? Do you have legal responsibilities. Levels of backup planned: Daily Weekly Monthly Quarterly Semi-annually Annually How long must you keep the data? How do you restore the data? Does your restore need to be bare metal or just data? Bare metal, fast restore, long-term storage = more $$
7 UNIX Backup Tools Open Source options dd dump tar rsync Amanda Bacula (Apple s Time Machine uses this) (heavily used, very popular)
8 dd The lowest level type of backup Bit-for-bit copy For example: dd if=/dev/ad0s1a of=/backup/root Exact copy, but not efficient if you only use 100 MB on a 1 GB partition, you still end up with a backup of 1 GB compression helps, but you still spend time copying unused space Best for doing system recovery, or Copying media (CD-ROMs, DVDs, etc.)
9 Dump The traditional UNIX backup programs dump and restore. Works at inode level Takes backups of entire file systems, but only the used space It is unable to backup only part of a file system It does not backup across mount points (directory tree that spans more than one file system) Note: If you use dump on your / partition, you would not back up /home, /usr or or any other mounted FS. You must explicitly run dump for each FS.
10 Dump Dump can backup to several media local file remote file tape Dump can take incremental dumps only files that have changed are backup up
11 Remote dump It is possible to run dump over ssh for a secure transport: # /sbin/dump -0uan -f - /usr gzip -2 \ ssh \ dd of=/backups/dump-usr.gz Anyone asking, Where s the if parameter for dd?
12 Tar tar(1) (Tape Archive) dates back to Version 6 of AT&T UNIX (circa 1975). tar operates in cooperation with the file system; tar writes files and directories to tape or to a file. Just like with dump, one can use ssh to backup across the network: # tar -cfz - / (ssh remote; cat >/backups/backup-0425.tgz)
13 Examples using tar Let's take a backup of /etc where most configuration files reside, and place it in /home/ backups: # mkdir /home/backups # tar -cvf /home/backups/etc.tar /etc Note: The -c option to tar tells it to create an archive, -v specifies verbose output and -f specifies the file to be either written to or read from You'll see quite a lot of output as tar creates the archive at this point.
14 Examples using tar Now we check whether our archive has actually been created # cd /home/backups # ls This now show us a new file in this directory etc.tar If we now wanted to view the contents of this backup we can run # tar -tvf etc.tar
15 Examples using tar This will show you the contents of the etc directory as you backed it up. To actually restore and and unpack the contents that were backup up previously: # cd /home/backups # tar -xvf etc.tar
16 Examples using tar Notice that the restore actually creates a new directory etc where you are located not in /etc! This is because tar by default removes the leading '/' from the directories it has backed up in order not to overwrite the original files on your system when you choose to do a restore (a security consideration)
17 Rsync Another very powerful tool is rsync Rsync is very efficient: it only transfers files that have changed, and for those files, only the parts of the files that have changed This is very efficient for large trees with many files, some of them large Great for replicating a server off-site, or for doing quick backups for a migration.
18 Rsync Combined with the --link-dest option, it allows to do snapshot-like backups. --link-dest takes the newest backup, and makes links (which take 0 space) to the files that have not changed, and replicates those that have changed Allows for backup.0, backup.1, backup.2, backup.3, where backup.x is a COMPLETE copy of the replicated source, but the disk space used is ONLY the difference.
19 Rsync example script On remote backup host: # rm -rf /backups/etc.2 # mv /backups/etc.1 /backups/etc.2 # mv /backups/etc.0 /backups/etc.1 # mv /backups/etc /backups/etc.0 On machine to be backed up: # rsync -avhs \ --link-dest=etc.0 \ /etc/ host:/backups/etc/ This will backup only changed files from /etc/ to host:/etc/. Unchanged files are linked from etc.0
20 Other tools Rdiff-backup Unison Rnapshot
21 Other possible Backup methods Disk duplication Using the dd command mentioned earlier, it is possible to duplicate your entire disk block by block on another disk. However the source and destination disk should be identical in size or the destination must be bigger than the source. Another way of doing this is using RAID1 mirroring and hot swappable disks: make sure the RAID volume is rebuilt (OK) remove one of the two disks (call it backup ) replace backup with a fresh disk, let the RAID rebuild take backup home Remember: RAID or mirroring is not backup. An rm -rf / on your RAID set works very well!
22 Other possible Backup methods Disk duplication (2) instead of mirroring the two disks, make two filesystems, and use rsync to copy every night from disk 1 to disk 2 in case of user error (rm -rf), you can recover from disk 2, without having to pull the backup tapes out of the safe NOTE: IT DOES NOT HELP IF THE SERVER IS STOLEN OR THERE IS A FIRE, IF BOTH DISKS ARE IN THE MACHINE!
23 Networked backup systems There are a number of networked backup systems out there for backing up many servers to one or more backup servers, using tape drives or disk storage. In the Open Source world, two backup systems stand out: AMANDA - BACULA -
24 Amanda Advanced Maryland Automatic Network Disk Archiver Has been around for many years Networked backup Support incremental backups to disk, tape Can backup to a holding disk, flush to tape later Encrypted data flows and backup data Tape library / loader control and labeling Windows backup using a windows client All source code for Amanda is open source
25 Bacula Written by the people who invented AutoCAD Extremely popular and well-tested. Claims to be the most popular Open Source, Enterprise-level backup package around. Impressive documentation (400- pages!), including a developer's guide and tutorial Support incremental backups to disk, tape Complete SQL backend (MySQL, PgSQL, SQLite) Encrypted data flows using TLS (standard!) Tape library / loader control and labelling Native Windows client Good documented scenarios for specific backup cases, including complete bare metal restore
26 Bacula: Supported OS s The big three 1. Windows 2. Mac OS X 3. UNIX/Linux Are fully supported in Bacula. Additional, typical enterprise OS versions are supported as well (HP/UX, AIX, Solaris, etc.)
27 Reminder: Backup security 1. Take the disks / tapes / CDs off site! -> it does not help if there is a fire or if tapes are stolen 2. Consider encrypting the data on the disks / tapes / CDs -> what happens if the tapes are stolen? what happens when you throw them out?
28 RAID Redundant Array of Independent Disks Redundant Array of Inexpensive Disks RAID 0 RAID 1 RAID 3 RAID 5 RAID 6 RAID 1+0 or 10
29 Types of redundancy There are different levels of redundancy: none if a disk crashes, data is lost disk DATA RAID1 2 disks are mirrored, data is written to both disks at any time. One disk can be lost without losing data. DATA disk DATA disk
30 Types of redundancy RAID3, RAID5 data is distributed across several disks, data parity, used to rebuild a defective drive, is either placed on a dedicated drive (RAID3) or across all drives (RAID5): DATA disk DATA disk PARITY disk DATA DATA PARITY DATA PARITY PARITY DATA DATA DATA disk disk disk RAID3 RAID5
31 RAID 0 Striping Not technically RAID, but a RAID card is used to implement. Data is striped between disks. Improves I/O in most cases.
32 RAID 1 Mirroring Disks are mirrored, data is written to both disks at any time. One disk can be lost without losing data.
33 RAID 3 Striping +Dedicated Parity Data is written across multiple disks (striping). A dedicated disk is used for parity. Recovering from remaining disks plus parity disk. Lost parity disk = lost RAID array. Fast I/O.
34 RAID 5 Striping +Distributed Parity Data is written across multiple disks (striping). Parity is written across all disks. Most popular type of RAID after RAID 1. Can lose any 1 disk (set of 3) Has serious subtle issues!
35 RAID 6 Striping+Double Distributed Parity Data is written across multiple disks (striping). Parity is written across all disks multiple times. Fixes issues with RAID 5. Can lose any 2 disk (set of 4). Fixes issue with 1TB+ drives.
36 RAID 1+0 or 10 Mirrored Sets in a Striped Set Data is mirrored in multiple sets and sets are striped. Provides performance and fault tolerance. Can lose multiple disks as long as no one mirror loses all disks. Requires more disks for same storage space. Referred to as nested or hybrid RAID.
37 RAID Controller Failure What do you do? Use a hot spare RAID Card. Card must be identical. Cards must support hot -spare in BIOS. Generally connected by onboard data path, or via cable between both cards. Otherwise, at a minimum, buy 2xRAID card when building your array. If many arrays buy extra cards.
38 Fun Facts Due to quantum physics - Error rate = 100% for 1TB+ drive writes - RAID 6+ required to deal with this issue Enterprise Class Drives - Built to reduce vibration - Reduced vibration = More reliable - Cost a bit more, but essential in critical environments - May TB, 3Gb/s, 7200RPM, 32 MB Cache, 1.2 million hours MTBF SATA drives around USD $150/each. - 2TB, 6Gb/s, 7200RPM, 64 MB Cache, 1.2 million hours MTBF SATA drives around USD $250/each. Dedicated to save us from RAID 5
39 Hardware or software? In general, hardware RAID is more transparent to the user, and disk replacement is straightforward: remove defective disk install new disk RAID controller detects this and starts rebuilding on new disk (Note: real hardware RAID controllers, NOT BIOS RAID such as Promise)
40 Hardware or software? RAID3 and 5 can be complex to implement in software (in the OS), so hardware might be a better choice But what happens if the RAID controller dies? How does one recover if one does not have a spare controller? Consider having a spare controller for RAID3/RAID5/RAID6/RAID1+0 (Note: we mean real hardware RAID controllers, not BIOS software RAID such as Promise)
41 Hardware or software? RAID1 is easy to recover from and easier to implement in software (within the OS) worst case, all one needs is to skip a header at the beginning of each disk. FreeBSD and Linux have very good software RAID implementations nowadays In FreeBSD, at least 3 implementations: gmirror ccd gvinum (also RAID5, but not recommended) But you want to use ZFS
42 ZFS Incredibly flexible, incredibly powerful file system available natively in FreeBSD. For all practical purposes has no file system or file size limits. Implements functionality of RAID6 with RAID-Z2 option of double-parity striping in the ZFS volume manage. ZFS creates FS over Volume over Hardware. ZFS knows about the hardware layer. Volume recovery is fast! Only recover data, not empty areas of disk (RAID recovers all bits in an array.
43 References RAID Overview ZFS ZFS on FreeBSD ZFS on Linux (not completed)
For Small and Medium Businesses Overview May 2008 Overview 2 Contents Page 3 Page 6 Page 9 Page 11 Introduction RAID storage Software versus hardware RAID Backup hardware and software Considerations for
One Stop Data & Networking Solutions PREVENT DATA LOSS WITH REMOTE ONLINE BACKUP SERVICE Prevent Data Loss with Remote Online Backup Service The U.S. National Archives & Records Administration states that
Cyber Security: Guidelines for Backing Up Information A Non-Technical Guide Essential for Executives, Business Managers Administrative & Operations Managers This appendix is a supplement to the Cyber Security:
Options for backing up your computer Aryeh Goretsky, MVP, ZCSE NOTE: An earlier version of this white paper appeared as a guest blog at The Windows Club website. Table of Contents Both hardware and software
Summary The aim of this article is to present some basic information about Small Business Server 2003 and how it can fit in with your business. Topics What is Small Business Server? (also known as SBS)
REMOTE BACKUP-WHY SO VITAL? Any time your company s data or applications become unavailable due to system failure or other disaster, this can quickly translate into lost revenue for your business. Remote
An Oracle Technical White Paper May 2011 Oracle Optimized Solution for Enterprise Cloud Infrastructure Introduction... 1 Overview of the Oracle Optimized Solution for Enterprise Cloud Infrastructure...
Preparing for Disaster: Backup Scenarios That Work Executive Summary by Tom Mills simplitek Corporation - Technology Consultant to Small Business With some simple and inexpensive steps, any small business
Best Practices Guide McAfee epolicy Orchestrator for use with epolicy Orchestrator versions 4.5.0 and 4.0.0 COPYRIGHT Copyright 2011 McAfee, Inc. All Rights Reserved. No part of this publication may be
An Overview of the Different Technologies Available for Backup CONTENTS CONTENTS... 2 INTRODUCTION... 4 TAPE DRIVES... 4 Cost... 4 Quantity of Data... 4 Length of Archival... 5 Time to Backup... 5 Time
Data Backup Options for SME s As an IT Solutions company, Alchemy are often asked what is the best backup solution? The answer has changed over the years and depends a lot on your situation. We recognize
Relational Database Management Systems in the Cloud: Microsoft SQL Server 2008 R2 Miles Ward July 2011 Page 1 of 22 Table of Contents Introduction... 3 Relational Databases on Amazon EC2... 3 AWS vs. Your
ShadowProtect IT Edition User Guide Page 1 1. ShadowProtect IT Edition User Guide............................................................................ 3 1.1 ShadowProtect Overview..................................................................................
INTRODUCTION TO LINUX CLUSTERING DOCUMENT RELEASE 1.1 Copyright 2008 Jethro Carr This document may be freely distributed provided that it is not modified and that full credit is given to the original author.
White Paper EMC VNXe HIGH AVAILABILITY Overview Abstract This white paper discusses the high availability (HA) features in the EMC VNXe system and how you can configure a VNXe system to achieve your goals
HP StoreOnce Catalyst and HP Data Protector 7 Implementation and Best Practice Guide Release 1 Executive Summary This guide is intended to enable the reader to understand the basic technology of HP StoreOnce
Best Practices for Deploying and Managing Linux with Red Hat Network Abstract This technical whitepaper provides a best practices overview for companies deploying and managing their open source environment
EaseUS Todo Backup User guide - 1 - Table of contents Welcome... - 5 - Overview... - 6 - How to use this manual... - 7 - New in EaseUS Todo Backup 6.5... - 8 - Getting started... - 9 - Hardware requirements...
Pros 4 Technology Online Backup Features Introduction Computers are the default storage medium for most businesses and virtually all home users. Because portable media is quickly becoming an outdated and
VirtuousIT Ltd 1 Table of Contents 1 Introduction 3 The Data Explosion 3 The Importance of Rapid System Recovery 3 2 The VirtuousIT Solution 4 3 RecoveryShield - Solutions 5 Large Business 5 Small Medium
Microsoft Corporation and HP Using Network Attached Storage for Reliable Backup and Recovery Microsoft Corporation Published: March 2010 Abstract Tape-based backup and restore technology has for decades
High Availability and Scalability with Domino Clustering and Partitioning on AIX Marcelo R. Barrios, Ole Conradsen, Charles Haramoto, Didac Marin International Technical Support Organization http://www.redbooks.ibm.com