Linux Backups. Russell Adams <rladams@adamsinfoserv.com> Linux Backups p. 1

Linux Backups Russell Adams <rladams@adamsinfoserv.com> Linux Backups p. 1

Linux Backup Discuss a variety of backup software & methods Recommendations for backup HOWTO for Rsnapshot Demonstration Linux Backups p. 2

Shameless Self Promotion Principal Consultant @ Adams Information Services LLC http://www.adamsinfoserv.com/ HLUG member since 2002 IBM CATE Provides Nationwide services and administration for IBM AIX, pseries, SAN Storage, HACMP Go Debian! Linux Backups p. 3

Basic Recommendations Use separate drives/media for data and backup Use RAID or other redundancy for all disks where possible Stop open applications during a backup (ie: VMWare, SQL) to prevent open files issues and backup corruption Always have a recent full backup Keep a recent offsite copy Linux Backups p. 4

Basic Recommendations TEST YOUR BACKUPS Linux Backups p. 5

Backup Software Variety of tools & full products for Linux Only covering file level backup here Discuss Bare metal restore Linux Backups p. 6

Features / Criteria Tool / Product Tool A tool works as a standalone command that performs a single function. Product Most products perform multiple functions and have a user interface. They are typically more complex than a standalone tool. Linux Backups p. 7

Features / Criteria Ease of use Learning curve Backup Restore Backup Type Full Incremental Linux Backups p. 8

Features / Criteria Media Disk Tape Indexing Remote Backup Compression Encryption Automation Linux Backups p. 9

Software cp tar rsync Duplicity rsnapshot AMANDA BackupPC k3b Mondo Rescue mkcdrec Linux Backups p. 10

cp Standalone tool Copy files to external drive/partition Easy to use, backup and restore use the same command Full backups, can perform incremental updates Disk media only No indexing, must be manually maintained Linux Backups p. 11

cp Optional remote (ie: scp) No compression, encryption Automation requires manual scripting & cron Linux Backups p. 12

cp Example cp -a src dest Recursively copies src to dest, preserves owner/group/time/symlinks cp -au src dest As above, but only copies items that have changed (ie: incremental) Linux Backups p. 13

tar Standalone tool, Unix standard for archival, competes with cpio Learning curve is mild Complex flags Restores are more complex than backups Supports both full and incremental backups Can write to files on disk or tape Linux Backups p. 14

tar Manual indexing, incremental indexes are required but file specific Remote backups supported via remote tape Full compression, gzip & bzip2 No encryption Automation requires manual scripting & cron Linux Backups p. 15

tar Example tar -zcvpf dest.tar.gz src Create an archive of files in src, preserving permissions, with compression. tar -g dest.dat -zcvpf dest.tar.gz src As above, but save an index file for creating incremental backups Linux Backups p. 16

tar Example tar -C dest -zxvpf src.tar.gz Restore from archive to dest, preserving permissions Linux Backups p. 17

rsync Standalone tool with fantastic flexibility Easy to use, can replace cp command Moderate learning curve, many options Backup & Restore can use the same command Full backups with fast incremental updates Writes to disk files only Manual indexing Linux Backups p. 18

rsync Remote backup fully supported, native and over scp Common tool for maintaining remote mirrors Can optionally compress network traffic, no file compression No encryption, when using scp network traffic is encrypted Automation requires manual scripting & cron Linux Backups p. 19

rsync Example rsync -av src dest Copies src to dest, preserving owner/group/permission/timestamp rsync -av src user@host:dest Copy via ssh to remote host Linux Backups p. 20

rsync Complex Example rsync verbose s t a t s r e c u r s i v e perms \ times d e l e te l i n k s \ max d e l e te =50 compress itemize changes \ exclude rladams /. m o z i l l a / f i r e f o x / / Cache \ exclude rladams / home \ exclude rladams / music \ exclude rladams / tmp \ log f i l e =/home / rladams /. rsync_home. log \ / home / backup home : / backup / thinkpad / Linux Backups p. 21

Duplicity CLI Product Key feature is creating remote backups on untrusted storage Moderate learning curve Backup and restore use similar commands Supports full and incremental backups Linux Backups p. 22

Duplicity Only writes to disk or remote host Internally indexed for incrementals Remote backup via SSH Full compression Encrypted via GnuPG Automation requires manual scripting & cron Linux Backups p. 23

rsnapshot Full featured tool for creating backups Fantastic storage algorithm Easy to use Backup to a directory tree Restore is simply copying files back into place Full backup with multiple incrementals Disk only Linux Backups p. 24

rsnapshot Indexed by iteration Pull remote backups supported via rsync & scp No compression or encryption Automation is easy One line cron job Setup config file Linux Backups p. 25

rsnapshot Example Covered in depth in the HOWTO rsnapshot weekly Take an incremental backup to weekly repository Linux Backups p. 26

AMANDA Product Server side, multiple clients Intelligent backup scheduling logic & tape handling Key features are disk spooling and tape indexing & library automation Linux Backups p. 27

AMANDA Steep learning curve CLI Only Backups are automated Restores are manual via ftp-like interface Supports repeated full backups and incrementals Linux Backups p. 28

AMANDA Writes to tape, uses local disk for spooling backups (fast!) Complex indexing Supports remote backup and compression Optional encryption Fully automated on the server side Linux Backups p. 29

BackupPC Product Server side, multiple clients Intelligent backup scheduling logic Key features are user interface & rsnapshot style backend storage Linux Backups p. 30

BackupPC Moderate learning curve Fantastic web interface with multiple user support & security Backups are easy to setup Restores via web interface File explorer, download files from browser Zip file download Push to host Linux Backups p. 31

BackupPC Supports full backup with multiple incrementals Disk only Complete indexing Optional on-disk compression Encryption over network using SSH, no file encryption Complete automation, jobs created & scheduled from web Linux Backups p. 32

BackupPC Linux Backups p. 33

BackupPC Linux Backups p. 34

BackupPC Linux Backups p. 35

BackupPC Linux Backups p. 36

BackupPC Linux Backups p. 37

BackupPC Linux Backups p. 38

Other k3b Deserves a mention for a quick method to backup files & directories to CDROM. Mondo Rescue Mondo Rescue takes a bare metal backup for full system restore. Popular with Redhat mkcdrec Another bare metal backup tool Linux Backups p. 39

Rsnapshot Storage Algorithm Rsnapshot uses hard links in a creative fashion to minimize the amount of storage required for multiple backups Total size is the total for one full backup and only changes for as many incrementals that are kept Linux Backups p. 40

Sample Disk Usage 46G / backup / d a i l y. 0 / 263M / backup / d a i l y. 1 / 269M / backup / d a i l y. 2 / 248M / backup / d a i l y. 3 / 8.6G / backup / d a i l y. 4 / 3.1G / backup / d a i l y. 5 / 252M / backup / d a i l y. 6 / 252M / backup / d a i l y. 7 / 247M / backup / d a i l y. 8 / 253M / backup / d a i l y. 9 / 59G t o t a l Linux Backups p. 41

Files & Linking Legend File Inode Symlink Regular File File_A.txt Inode 1 Linux Backups p. 42

Symlinks vs Hard Links Symlinked File File_A.txt Inode 1 File_A.txt Hardlinked File Inode 1 File_B.txt File_B.txt Linux Backups p. 43

Rsnapshot Backup Cloning new backup Time.0 File_A Time.1 File_A File_B File_B File_C File_C Linux Backups p. 44

Rsnapshot Backup Update clone via rsync Time.0 File_A Time.1 File_A File_B File_B File_C File_C Linux Backups p. 45

Rsnapshot Backup Backup Repository on Disk Time.0 Time.1 Time.2 Time.3 File_A File_A File_A File_A File_B File_B File_B File_B File_C File_C File_C File_C Linux Backups p. 46

HOWTO Backup script Udev rule for automatic automount & backup Cron job Rsnapshot.conf Linux Backups p. 47

Backup.sh #! / bin / sh ex [ f /. nobackup ] && e x i t exec > / tmp / backup. log 2>&1 mount / mnt / backup rsnapshot weekly sync umount / mnt / backup Linux Backups p. 48

Udev Rule /etc/udev/rules.d/10-my.rules KERNEL=="sd?1", SUBSYSTEMS=="usb ", ATTRS{ idproduct }=="0702", \ ATTRS{ idvendor }=="05 e3 ", SYMLINK=" backuphd ", \ RUN= " / bin / sh / r o o t / backup. sh " Linux Backups p. 49

Sample Cron Job 0 0 rsnapshot c / etc / rsnapshot. conf d a i l y Linux Backups p. 50

Rsnapshot.conf c o n f i g _ v e r s i o n 1.2 snapshot_root / mnt / backup / snapshots / no_create_root 1 cmd_cp / bin / cp cmd_rm / bin / rm cmd_rsync / usr / bin / rsync cmd_logger / usr / bin / logger i n t e r v a l weekly 5 verbose 5 l o g l e v e l 5 l o g f i l e / tmp / rsnapshot. log l o c k f i l e / var / run / rsnapshot. pid backup / etc thinkpad / backup / home thinkpad / Linux Backups p. 51

Q&A Linux Backups p. 52