1 Data storage A Simple Overview 1. HW: SCSI/IDE/SATA/SAS (SAN!) 2. HW/SW: RAID 3. SW: Logical volumes 4. SW: Journalling filesystems 5. SW/NET: Networked filesystem (NAS!) DAS-NAS-SAN 2 RAID Redundant Array of Inexpensive/Independent Disks RAID 0 Disk striping (best I/O performance, no redundancy!) RAID 1 Disk mirroring (complete redundancy) RAID 5 Disk striping with additional parity information. The parity information is split across multiple disks RAID 0+1 A combination of RAID 0 and 1 (mirror of stripes) RAID 10 A combination of RAID 1 and 0 (stripe of mirrors)
RAID cont. www.acnc.com/04_00.html en.wikipedia.org/wiki/nested_raid_levels 3 Volumes Logical Volume Managers Logical Volume Managers (LVMs) provide a higher-level view of disk storage, and gives much more flexibility in allocating storage Physical disks can be made into physical volumes by dividing them into physical partitions/ extents A volume group is a named collection of physical volumes Volume groups can be split into subunits called logical volumes A logical volume can hold a single file system, and is much more flexible than a disk partition! Logical volumes are composed of logical partitions/extents Logical partitions/extents usually map one-to-one to physical partitions/extents (in either striping or linear mode), but in case the logical volume is set up to store redundant copies of data it can map one-to-many LVM cont. hda1 hdc1 (PV) \ / diskvg (VG) / \ usrlv rootlv varlv (LV:s) ext3 reiserfs xfs (filesystems) 2
4 Filesystems Filesystems The problem journalling file systems solve is that if your host crashes, traditional file system have to run extensive checks (fsck) on the integrity of the file systems Journalling file systems keep a transaction log (a journal) of the most recent file system operations, thus allows them to be replayed and allows for much faster recovery ext3, jfs, reiserfs, XFS Networked Filesystems Unix: NFS (Network File System) Server: /etc/exports Client: /etc/fstab MSWindows: CIFS (Common Internet File System) (formerly known as SMB) Samba is (among other things) an open source implementation of CIFS, thus MSWindows clients can mount Samba shares Other possibilities include DFS, AFS, coda, InterMezzo Problems with authentication! understand what is implied by: NFSv{2,3,4} mount/umount File systems can be mounted with the mount command: device dir mount -t type e. g. mount -t iso9660 /dev/hdc /mount/cdrom (see also man mount of course) To unmount, simply use: umount dir e. g. umount /mount/cdrom (hint: if not able to unmount, use fuser -v dir) 3
/etc/fstab From man fstab: The file fstab contains descriptive information about the various file systems. fstab is only read by programs, and not written; it is the duty of the system administrator to properly create and maintain this file. e.g. # # <file system> <mount point> <type> <options> <dump> <pass> /dev/hda3 / ext3 errors=remount-ro 0 1 /dev/hda2 none swap sw 0 0 proc /proc proc defaults 0 0 /dev/fd0 /floppy auto user,noauto 0 0 /dev/cdrom /cdrom iso9660 ro,user,noauto 0 0 /dev/hda1 /boot ext3 defaults 0 2 rom:/st/rom/home /st/rom/home nfs rw,bg,hard,intr 0 0 Punchline Be interested in alle levels of data storage, never trust the vendor 5 Backup/Restore Backup System = Data Restoration System Why? accidental deletion disk failure malware (virus) archival purposes Backup/Restore: The Questions (From Essential system administration by Frisch:) What files need to be backed up? Where are these files? Who will backup the files? Where, when and under what conditions should backups be performed? How often do these files change? How quickly does an important missing or damaged file need to be restored? What is the expected rate of loss or failure? How long do we need to retain the data? Where should the backup media be stored? Where will the data be restored? 4
Other Factors What media to use? magnetic tape optical disk File-based vs device-based Compression Problems with online backup file changes directory structure changes Two Basic Strategies Full Copy all files Partial Two concepts sometimes referred to as the same: Differential All new or modified files since last full backup Incremental All new or modified files since last full or partial backup Difference between differential and incremental sometimes referred to as level x where x 0, 1, 2,... Backup Schedules: Grandfather-Father-Son 5
A useful command find dir -newer /var/adm/yesterday -ls \ awk {sum+=$7}; END {print "diff =",sum} (from Frisch 2002) 6 Next Week Next Week Exam Dec. 3rd 0900-1100 6