Data Storage, File Servers, Backup and Restore IMT3292 - System November 15, 2007
Administrating a File Server Making local and remote files available to users Monitoring and managing the systems disk resources Protecting against file corruption, hardware failures, and users errors via a well-planned backup schedule Ensuring data confidentiality by limiting file and system access Checking for and correcting filesystem corruption Connecting and configuring new storage devices when needed
A Simple Overview 1 HW: IDE/SCSI (SAN!) 2 HW/SW: 3 SW: /Partitioning 4 SW: Journalling filesystems 5 SW/NET: Networked filesystem (NAS!)
Redundant Array of Inexpensive/Independent Disks 0 Disk striping (best I/O performance, no redundancy!) 1 Disk mirroring (complete redundancy) 5 Disk striping with additional parity information. The parity information is split across multiple disks 0+1 A combination of 0 and 1 (mirror of stripes) 10 A combination of 1 and 0 (stripe of mirrors)
cont. www.acnc.com/04_00.html en.wikipedia.org/wiki/nested levels
Logical Volume Managers Logical Volume Managers (s) provide a higher-level view of disk storage, and gives much more flexibility in allocating storage Physical disks can be made into physical volumes by dividing them into physical partitions/ extents A volume group is a named collection of physical volumes Volume groups can be split into subunits called logical volumes A logical volume can hold a single file system, and is much more flexible than a disk partition! Logical volumes are composed of logical partitions/extents Logical partitions/extents usually map one-to-one to physical partitions/extents (in either striping or linear mode), but in case the logical volume is set up to store redundant copies of data it can map one-to-many
cont. hda1 hdc1 (PV) \ / diskvg / \ (VG) usrlv rootlv varlv (LV:s) ext2 reiserfs xfs (filesystems)
We now a lot about filesystems from the operating systems course These days, most professional file systems are journalling file systems The problem journalling file systems solve is that if your host crashes, traditional file system have to run extensive checks (fsck) on the integrity of the file systems Journalling file systems keep a transaction log (a journal) of the most recent file system operations, thus allows them to be replayed and allows for much faster recovery ext3, jfs, reiserfs, XFS
Networked Unix: NFS (Network File System) Server: /etc/exports Client: /etc/fstab MSWindows: CIFS (Common Internet File System) (formerly known as SMB) Samba is (among other things) an open source implementation of CIFS, thus MSWindows clients can mount Samba shares Other possibilities include DFS, AFS, coda, InterMezzo Problems with authentication! understand what is implied by: NFSv{2,3,4}
mount/umount File systems can be mounted with the mount command: mount -t type device dir e. g. mount -t iso9660 /dev/hdc /mount/cdrom (see also man mount of course) To unmount, simply use: umount dir e. g. umount /mount/cdrom (hint: if not able to unmount, use fuser -v dir)
/etc/fstab From man fstab: The file fstab contains descriptive information about the various file systems. fstab is only read by programs, and not written; it is the duty of the system administrator to properly create and maintain this file. e.g. # # <file system> <mount point> <type> <options> <dump> <pass> /dev/hda3 / ext3 errors=remount-ro 0 1 /dev/hda2 none swap sw 0 0 proc /proc proc defaults 0 0 /dev/fd0 /floppy auto user,noauto 0 0 /dev/cdrom /cdrom iso9660 ro,user,noauto 0 0 /dev/hda1 /boot ext3 defaults 0 2 rom:/st/rom/home /st/rom/home nfs rw,bg,hard,intr 0 0
Punchline Be interested in alle levels of data storage, never trust the vendor
Backup System = Data Restoration System Why? accidental deletion disk failure malware (virus) archival purposes
: The Questions (From Essential system administration by Frisch:) What files need to be backed up? Where are these files? Who will backup the files? Where, when and under what conditions should backups be performed? How often do these files change? How quickly does an important missing or damaged file need to be restored? What is the expected rate of loss or failure? How long do we need to retain the data? Where should the backup media be stored? Where will the data be restored?
Other Factors What media to use? magnetic tape optical disk File-based vs device-based Compression Problems with online backup file changes directory structure changes
Two Basic Strategies Full Copy all files Partial Two concepts sometimes referred to as the same: Differential All new or modified files since last full backup Incremental All new or modified files since last full or partial backup Difference between differential and incremental sometimes referred to as level x where x 0, 1, 2,...
A useful command find dir -newer /var/adm/yesterday -ls \ awk {sum+=$7}; END {print "diff =",sum} (from Frisch 2002)
Backup Schedules: Grandfather-Father-Son
Backup Schedules: Tower of Hanoi
Backup Schedules: Modified Tower of Hanoi (man dump)
Monitoring