Automated Offsite Backup with rdiff-backup Michael Greb 2003-10-21 Contents 1 Overview 2 1.1 Conventions Used........................................... 2 2 Setting up SSH 2 2.1 Generating SSH Keys........................................ 2 2.2 Setting up Key Based Authentication on the Remote System.................. 3 2.3 Setting Custom SSH Options.................................... 4 2.4 Testing the SSH Setup........................................ 4 3 Setting up rdiff-backup 4 3.1 Creating the Location for Storing Backups............................. 4 3.2 Specifying Files to Exclude from Backup.............................. 5 4 Automating it with Cron 5 5 Testing It 6 6 Conclusion 6 1
1 Overview rdiff-backup provides a simple easy means for providing remote backups. This is intended to be a simple, easy to follow set of instructions for implementing off site backups. There are some great advantage to using rdiff-backup for this, here are some of my favorite: Files are stored on the remote site as normal files, not in some proprietary format making looking for something easy using standard *nix tools such as find, grep and locate. A full history of previous revisions is kept so you can restore a copy of that file you accidently changed from several days ago, before the change was made. Rsync s algorithms are used so only the changes from the previous day are transfered, not a full image. Previous revisions of files are stored as differences from current version saving space where backups are stored. Can use ssh for transferring files making the transmissions secure from packet sniffing. Can use RSA or DSA public key cryptography for authenticating to the remote host so automated backups are still secure without a clear text password in a config file. If you have any corrections or suggestions for this document, either technical or regarding readability, please do drop me a line at michael@thegrebs.com so your suggestions can help other people. 1.1 Conventions Used In this document program output will look like typed on a typewriter. Anything typed by the user will be shown in bold. For host-names, source will be used for the server being backed up and destination for the host storing the backups. This document assumes you already have rdiff-backup installed, if you need help with this, feel free to email me at the address above with any questions you may have. If there is enough demand, I will add instructions for getting it installed on the most popular distributions. 2 Setting up SSH This step gets ssh setup for secure unattended, password-less use. Public and private RSA keys will be generated. The private key is stored on the machine you are backing up too while the public key gets sent to your server you are backing up. These keys are used to authenticate a connection without passwords in a secure manner. Then we setup the remote end to authenticate against this key and allow only rdiff-backup to run. 2.1 Generating SSH Keys Use the ssh-keygen program to generate a password-less RSA key pair by executing the following command. In this example /root is root s home directory. Adjust accordingly for your system. 2
destination# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id rsa): /root/.ssh/id rsa backup Enter passphrase (empty for no passphrase): <ENTER> Enter same passphrase again: <ENTER> Your identification has been saved in /root/.ssh/id rsa backup. Your public key has been saved in /root/.ssh/id rsa backup.pub. The key fingerprint is: aa:63:1a:f8:0f:44:78:e9:d2:e1:25:40:51:2f:59:29 root@destination SSH will complain if it doesn t have se- Set this new private key as readable only by root for security. cure file permissions. destination# chmod go-r /root/.ssh/id rsa backup 2.2 Setting up Key Based Authentication on the Remote System Now we must transfer the public key to the source system. destination# scp /root/.ssh/id rsa backup.pub source:/root/.ssh/id rsa backup.pub root@source s password: <enter root password> id rsa backup.pub 100% 224 0.0KB/s 00:00 Now setup the entry on the source system, on the source system open id rsa backup.pub in your favorite text editor. This should be one really long line and you want to add the following to the beginning of it. Note that this line has been significantly shortened here. command= rdiff-backup server ssh-rsa AAAAB3Nz[...]iNM= root@destination The command= option restricts connections using this key to only being able to run rdiff-backup --server improving security of the authentication mechanism. Add this new line to authorized keys2. If there is no /root/.ssh/authorized keys2 already then execute the following command: source# mv /root/.ssh/id rsa backup.pub /root/.ssh/authorized keys2 If this file already exists, use these commands instead to add your new line to it: source# cat /root/.ssh/id rsa backup.pub >> /root/.ssh/authorized keys2 source# rm /root/.ssh/id rsa backup.pub Finally, set the permissions on this file to only readable by root: source# chmod go-r /root/.ssh/authorized keys2 3
2.3 Setting Custom SSH Options Now we need to tell ssh to use our new private key and a few other settings, this will lessen the CPU load on both ends. Open in your favorite editor, creating if it doesn t already exist, the file /root/.ssh/config on the destination system. You want to add the following section to this file. host source-backup hostname source identityfile /root/.ssh/id rsa backup compression yes cipher blowfish protocol 2 This causes ssh to use these settings when you ask it to connect to source-backup. The hostname specifies the real host to connect to. identityfile specifies the name of the private key to use when connecting. cipher is used to tell ssh we wish to use the blowfish cypher which has lower CPU overhead then the default. 2.4 Testing the SSH Setup rdiff-backup has a handy test mode to test connectivity. destination# rdiff-backup test-server source-backup::/ignored Testing server started by: ssh -C source-backup rdiff-backup --server Server OK If your output doesn t match hopefully you will receive an error that make sense. The most important thing here is that ssh shouldn t ask you for your password. If it doesn t work out, check you filenames and permissions. If your still at a loss, feel free to send some output my way and we ll see if we can get it knocked out. 3 Setting up rdiff-backup In this section we will setup a location for storing the backups, create an rdiff-backup configuration file, and set things up for automatic backup via a system cronjob. 3.1 Creating the Location for Storing Backups You need to decide on a location to store the backups on destination. I store my backups in /backups/<hostname>. Once you decide where you want to put your backups. Create the path. For example, to create the location I use you would type: destination# mkdir -p /backups/source 4
3.2 Specifying Files to Exclude from Backup We need to specify the files to include and exclude when backing up with rdiff-backup. This can be done on the command line when it is run, but for an automated system it is much easier to create a file with this information. I use /etc/backup-source.conf but you may use something else if you prefer. It is important to note that this file is located on destination, not on source as you might expect. - /dev - /proc - /tmp - /var/tmp - /usr/portage - /home/michael/dl In this file a - at the start of a line excludes paths matching the contents of the line. To include a path, you just place the path on the line with no preceding symbols. My example is for a Gentoo system, though the only Gentoo specific item is /usr/portage. /home/michael/dl is the location I download, extract and compile software and is easily replaceable, hence it s exclusion. The remainder of the entries you will most likely want to keep. The way we are setting stuff up, the root dir will be included, causing everything to be backed up except for the items listed in this file. 4 Automating it with Cron Finally, we setup rdiff-backup to run daily from cron. You will need to locate the directory holding scripts for system cronjobs that run daily. In most cases this is /etc/cron.daily. In this directory you need to create a new shell script that looks like this: #!/bin/sh export HOME=/root rdiff-backup --print-statistics \ --include-globbing-filelist /etc/backup-source.conf \ source-backup::/ /backup/source You may choose what ever name you desire for this script. I use rdiff-backup.sh. This script will run rdiff-backup using the config file you created earlier. The --print-statistics option is not required, it produces output that looks like this: 5
--------------[ Session statistics ]-------------- StartTime 1066622417.00 (Mon Oct 20 00:00:17 2003) EndTime 1066623159.32 (Mon Oct 20 00:12:39 2003) ElapsedTime 742.32 (12 minutes 22.32 seconds) SourceFiles 45652 SourceFileSize 429389316 (409 MB) MirrorFiles 44428 MirrorFileSize 414270694 (395 MB) NewFiles 1226 NewFileSize 15059120 (14.4 MB) DeletedFiles 2 DeletedFileSize 3501 (3.42 KB) ChangedFiles 119 ChangedSourceSize 1755983 (1.67 MB) ChangedMirrorSize 1692980 (1.61 MB) IncrementFiles 1349 IncrementFileSize 117377 (115 KB) TotalDestinationSizeChange 15235999 (14.5 MB) -------------------------------------------------- 5 Testing It Lastly you may want to run it for the first time right now. The first run will be the longest as it will need to transfer everything that needs to be backed up. A great way to do this and test out your script at the same time is by running it: destination# /etc/cron.daily/rdiff-backup.sh 6 Conclusion You should be now on your way to a happily backed up system. If you see any errors in this document, or think things could be clarified some where, please let me know! 6