Amanda The Open Source Backup & Archiving Software. Ian Turner ian@zmanda.com 11 April Copyright 2006 Zmanda, Inc. All rights reserved.

Transcription

3 What is Amanda? Most popular open source backup and archiving software Linux Journal Readers' Choice Award for "Favorite Backup System" Chapters in Linux/Unix system administration books Developed by University of Maryland and put in public domain in Thousands of users Universities, Research Labs, Technical enterprise installations. Copyright 2006 Zmanda, Inc. All rights reserved. 3

4 What is Amanda? Server-Client architecture Single server can backup multiple hosts to various media. Robust & Scalable Uses native platform tools Native data formats on the media. Well defined media interface Unattended backup runs Copyright 2006 Zmanda, Inc. All rights reserved. 4

6 What is Amanda? Active sourceforge.net project with hundreds of users and developers. Current release: 2.5 Active documentation. Amanda wiki Supports various operating systems Linux, MacOS 10, Unixes and Windows (using Samba or Cygwin). Code is very portable to all UNIX variants Server and Client support for all platforms Copyright 2006 Zmanda, Inc. All rights reserved. 6

7 What is Amanda? Available as part of Linux distributions Redhat Enterprise Server Suse Enterprise Server (SLES) Fedora Open Suse Mandrake Turbolinux Ubuntu Debian Latest release rpms are available from Copyright 2006 Zmanda, Inc. All rights reserved. 7

8 Amanda features Automatic backup level selection Consistent backup window. Estimates size of changes to be backed up. Supports multiple backup levels. Provides override option. Uses native backup/restore tools Supports dump, tar, star, and others. Uses the same format on the media. Restores from the media can be done without using Amanda tools. Copyright 2006 Zmanda, Inc. All rights reserved. 8

9 Amanda features Backs up many filesystems from many clients. Supports all Linux/Unix filesystem features sparse files hard links No file timestamp changes during backups Flexible policy in assigning backed up data to media. All configuration done on the server. Support for tapes, tape changers, virtual tape library, RAIT, disks as backup media. Copyright 2006 Zmanda, Inc. All rights reserved. 9

10 Amanda & Media Supports wide variety of media (tapes, disks) Uses basic media operations. Easy to add new media support. Media changers are supported using an API. Multiple media volumes can be used in a backup run. A single backup image can be split across multiple media volumes. Data can be backed up to Network Attached Storage (NAS) devices. Copyright 2006 Zmanda, Inc. All rights reserved. 10

11 Amanda & RAIT RAIT Redundant Array of Inexpensive Tapes Data is stripped over multiple tapes One tape contains parity data Any media (disk, tapes) can be used 2, 4, 5 tape sets are supported 2 tape set is a mirror You can use RAIT with Tape and Disk to create backups to Tape (offsite storage) and Disk at the same time D2D2T capability Amanda commands - amdd/ammt works on RAIT sets. Copyright 2006 Zmanda, Inc. All rights reserved. 11

12 Amanda backup level strategies Periodic full backups with incrementals in between. Periodic archival backup for offsite storage. Only Incremental backups. Full backups may not be needed or done manually Only Full backups. Different backup level strategy for each filesystem in a backup run. Copyright 2006 Zmanda, Inc. All rights reserved. 12

13 Amanda dynamic scheduler Traditional method: Administrator specifies when the backup level should be used for each filesystem. Administrators has to estimate the rate of change for each filesystem. Have sufficient media space available to backup all filesystems in the backup run (also called backup job). The amount of media required for backup will be different for each backup run. Backup window will be different for each backup run. Copyright 2006 Zmanda, Inc. All rights reserved. 13

14 Amanda dynamic scheduler Amanda scheduler: There is no need to specify when to do full/incremental backup level. Administrators specify Backup cycle - Do a full backup every 5 days. Backup runs per backup cycle. Media volumes per backup run. Number of media volumes per backup cycle. Copyright 2006 Zmanda, Inc. All rights reserved. 14

15 Amanda dynamic scheduler Amanda will schedule full backup as part of automatic backup level selection. Full backups are distributed over the backup cycle to balance the data backed up each run. Attempts to keep the backup window same for all backup runs. Actual dates and time of backup scheduled via crontab. Can make multiple runs in the same day. Copyright 2006 Zmanda, Inc. All rights reserved. 15

16 Amanda and Security Encrypted backups (Symmetric and Asymmetric key encryption). Client Server Custom encryption tools Server/Client communication encryption & authentication. OpenSSH Kerberos Works in SE Linux environment (Strict policy). Copyright 2006 Zmanda, Inc. All rights reserved. 16

17 Amanda and Compression Data can be compressed on the Client Server Many compression algorithms supported. gzip bzip2 custom compression algorithms can be added Hardware compression. Compression algorithm based on type of data being backed up. Copyright 2006 Zmanda, Inc. All rights reserved. 17

18 Amanda & Fault tolerance Skips the clients that could not be backed up during the scheduled run. Media errors Backed up data is kept in holding disk. Holding disk backup data can be flushed when the media problem is resolved. Does incrementals only to conserve holding disk space. Client/Server communication timeouts. Copyright 2006 Zmanda, Inc. All rights reserved. 18

19 Amanda configuration Multiple configurations in a Amanda server. Each configuration is associated with a backup schedule and media Each configuration has Amanda configuration file, amanda.conf List of filesystems/clients, disklist Amanda configuration file has all the tunable parameters. Backup run can be scheduled as cron job. No configuration required on the Amanda client. Copyright 2006 Zmanda, Inc. All rights reserved. 19

21 Amanda backup operation Amanda server requests backup size estimates from clients for multiple backup levels. Server analyzes the estimates to create a backup plan. Server asks the client to create a backup image and the backup image is sent to the media. A disk based cache (holding disk) is used for streaming data to the tape. Use of holding disk is optional. Multiple holding disks are supported. Copyright 2006 Zmanda, Inc. All rights reserved. 21

22 Amanda & data restoration Amanda stores all media, file information in a database. The database can be browsed using amrecover Amanda command. Data in the media is stored in native format and can be recovered using OS tools without Amanda commands. Backup images spanning multiple media can also be restored using OS tools. Copyright 2006 Zmanda, Inc. All rights reserved. 22

23 Amanda file header # mt -f <media_dev> fsf 1 # dd if=<media_dev> bs=32k count=1 AMANDA: FILE natasha /boot lev 1 comp N program /bin/gtar To restore, position tape at start of file and run: dd if=<tape> bs=32k skip=1 /bin/gtar -f records in 1+0 records out Copyright 2006 Zmanda, Inc. All rights reserved. 23

24 Amanda performance Consistent backup window. Amanda performs backup size estimation for all clients in a backup run in parallel. Heuristics can be used for backup size estimation The parallelism in the backup of data to the media can be configured for each configuration. Amanda uses holding disk as a cache to stream the data to a tape. Non-streaming media can be written to directly. Multiple holding disks with different chunk sizes. Copyright 2006 Zmanda, Inc. All rights reserved. 24

25 Amanda status Amanda provides status for each backup run. Amanda can be queried to get information about prior backup runs. Amanda can send notifications and reports to system administrators with status of each run. $ amstatus backup --file amdump.3 --failed Using /etc/amanda/dailyset1/amdump.3 from Wed Jan 25 17:48:28 PST 2006 localhost.localdomain:/boot 0 no estimate Copyright 2006 Zmanda, Inc. All rights reserved. 25

26 Amanda reporting Amanda provides amreport tool to obtain information of each backup run. It provides Data balance information (full backup scheduling) Summary of filesystems backed up Backup performance of each client and media Media usage for each backup image Plot of backup run backup run statistics for analysis. Identifying backup bottlenecks Copyright 2006 Zmanda, Inc. All rights reserved. 26

27 Amanda backup run statistics Total Full Incr. Estimate Time (hrs:min) 0:01 Run Time (hrs:min) 2:32 Dump Time (hrs:min) 0:36 0:36 0:00 Output Size (meg) Original Size (meg) Avg Compressed Size (%) Filesystems Dumped Avg Dump Rate (k/s) Tape Time (hrs:min) 1:52 1:52 0:00 Tape Size (meg) Tape Used (%) Filesystems Taped Avg Tp Write Rate (k/s) Copyright 2006 Zmanda, Inc. All rights reserved. 27

28 Amanda backup run statistics Backup time Backup size Performance Total Full Incr. Estimate Time (hrs:min) 0:01 Run Time (hrs:min) 2:32 Dump Time (hrs:min) 0:36 0:36 0:00 Tape Time (hrs:min) 1:52 1:52 0:00 Output Size (meg) Original Size (meg) Avg Compressed Size (%) Tape Size (meg) Filesystems Dumped Filesystems Taped Avg Dump Rate (k/s) Avg Tp Write Rate (k/s) Tape Used (%) Media usage Copyright 2006 Zmanda, Inc. All rights reserved. 28

30 Amanda verification tools Amanda can validate backup data on the media. It can do data verification in terms of media or backup run. Amanda has tools to verify configuration, media and Amanda server and client. Allows administrators to fix problems that can happen during backup run. Copyright 2006 Zmanda, Inc. All rights reserved. 30

31 Future work Integration with other open source tools. Nagios system management Mondo bare metal rescue LVM snapshots rsync and remote replication tools Backup solution for LAMP data Optimized support for Windows platforms. Graphical user interface. Copyright 2006 Zmanda, Inc. All rights reserved. 31

32 Future API enhancements Data filter API Data filtering can be done in server or client. Examples: compression, encryption, character encoding. Application API Allows support for consistent dump of live databases. More native archiving tools (star, etc.) Device API Abstraction of Media functionality. Support for Optical media, Disks, Tapes, Storage appliances. Copyright 2006 Zmanda, Inc. All rights reserved. 32