Home storage and backup options Chris Moates Head of Lettuce
Who I Am Lead Systems Architect/Administrator for Gaggle Previously employed as Staff Engineer by EarthLink/MindSpring Linux hobbyist since 1995 System Administrator/Developer since 1993 CPLUG member (off and on) since 2002
Presentation Goals Cover some disk storage concepts Explore a few NAS distributions Look at Crashplan
Home storage purposes Shared storage Dropbox and others don t store all your data Data in the cloud is not always desirable Local storage is much faster than cloud Backup location High speed backups High speed restores Local remote disks ISCSI volumes
Disk storage concepts Many choices on how to use disks to store data Depends on use case, IOPS requirements Not all are created equally
RAID Most familiar Multiple versions (levels) of RAID RAID-0 (not really RAID)
RAID 0
RAID Most familiar Multiple versions (levels) of RAID RAID-0 (not really RAID) RAID-1 (mirroring)
RAID 1
RAID Most familiar Multiple versions (levels) of RAID RAID-0 (not really RAID) RAID-1 (mirroring) RAID-5 (striped with parity)
RAID 5 / RAIDZ
RAID Most familiar Multiple versions (levels) of RAID RAID-0 (not really RAID) RAID-1 (mirroring) RAID-5 (striped with parity) RAID-6 (striped with extra parity)
RAID 6 / RAIDZ2
RAID Most familiar Multiple versions (levels) of RAID RAID-0 (not really RAID) RAID-1 (mirroring) RAID-5 (striped with parity) RAID-6 (striped with extra parity) RAID-10 (striped mirrors)
RAID 10
RAID Most familiar Multiple versions (levels) of RAID RAID-0 (not really RAID) RAID-1 (mirroring) RAID-5 (striped with parity) RAID-6 (striped with extra parity) RAID-10 (striped mirrors) RAID-50 (striped mirrors with parity)
RAID 50
RAID Most familiar Multiple versions (levels) of RAID RAID-0 (not really RAID) RAID-1 (mirroring) RAID-5 (striped with parity) RAID-6 (striped with extra parity) RAID-10 (striped mirrors) RAID-50 (striped mirrors with parity) RAID-60 (expensive)
RAID Other variants, both standard and proprietary Most are inferior to the previous choices in some way Typically can not be spun down to save on heat, power, and wear and tear
Simple Attach anywhere No automatic safety Single disks Can be spun down (doesn t consume power)
unraid Combine the intent of RAID with the idea of single disks Individual disks store individual files Parity disk is separate from the data disks Allows spinning down disks that aren t in use Catastrophic failures never mean losing all your data, just some
ZFS Complete rework of filesystem and redundancy desires from the ground up Zettabyte filesystem, designed for a future where 10,000TB disks are the norm Provides the most important features of RAID, adds features not supported by RAID Filesystem and storage layer are a single cohesive design, not separate
ZFS options Unlike RAID1, multiple disks can be mirrored, allowing for more failures RAIDZ1 = RAID5 RAIDZ2 = RAID6 RAIDZ3 = I m a paranoid sysadmin Add in a log device for fast, bursty writes Add a secondary cache device to speed up reads Like RAID, cannot be spun down
ZFS options (cont d) Compression Deduplication Snapshots Sends and receives Mix and match devices, to a degree Form clumps of smaller arrays into one large pool
NAS options Several NAS distributions exist Most are not based on Linux ZFS is very tempting, and has licensing conflicts with the GPL
NAS options OpenMediaVault (Linux) OpenFiler (Linux) FreeNAS (BSD) NAS4Free (BSD) unraid (Linux) Illumian (Solaris/Debian) NexentaStor CE (Solaris/Debian) OpenSolaris (guess)
OpenMediaVault/OpenFiler OMV appears to be more mature Offers SMB, NFS, AFS, ISCSI
OpenMediaVault
unraid Linux based Free for up to 3 disks (2 data + 1 parity) 7 disks for $70/$100 24 disks for $120/150 Runs from small flash drive, no installation Lowers power footprint by spinning down idle disks Supports SMB and NFS, others with plugins Active community
unraid
FreeNAS/NAS4Free BSD based Support ZFS Provides AFS, SMB, NFS, ISCSI
FreeNAS
NexentaStor CE Solaris kernel, Debian userspace Free version allows up to 18TB usable disk space Commercial quality web interface
NexentaStor
Illumian Solaris kernel, Debian userspace Bare OS underneath NexentaStor CE No storage restrictions No web interface ZFSguru web interface available
ZFSguru
Any linux distro Samba nfs-utils Roll your own Some web interfaces out there (notably webmin) Lots of manual setup to do things the others package up (graphs, SMART monitoring, spindown)
*yawn* Enough about storage, what about backups?
Crashplan Company offering cloud and local backup solution $4-$6/mo per computer to back up to the cloud $9-$13/mo per household (limit 10 machines) Daemon that can run on Linux and Solaris to back up locally at no cost Backup to your friend s daemon, also no cost Data is encrypted before storage, whether cloud, local, or friend Multiple backup targets supported
Crashplan Compatibility Operating System Any standard Linux unraid OpenMediaVault NexentaStor OpenSolaris 10+ FreeNAS NAS4Free Windows OS X Crashplan Compatible? Yes Yes Yes Yes (with my walkthrough) Yes Not easily Not easily Yes Yes
Possible Server Options Leftover hardware at home, of course Linux does well here, with great older hardware support ZFS essentially requires a 64bit processor, and loves it s RAM No, really, it has an unhealthy stalker like relationship with RAM 2GB is a bare recommended minimum, and the sky s the limit Off lease equipment
Possible Server Options Theserverstore.com $350: 4 bays, dual quad core, 36GB RAM $229: 8 2.5 bays, dual dual core, 8GB RAM $165: 8 bays, dual quad core, 4GB RAM Unixsurplus.com $99: 4 bay, quad core, 4GB RAM $350: 8 2.5 bays, dual quad core, 16GB RAM $520: 20 bay, dual dual core, 8GB RAM Newegg.com $400: Empty 24 bay chassis (new, Norco 4224)
Personal Evolution Started with Linux server with a RAID5 A couple of iterations of that before I wanted something nicer Moved to unraid Looked at ZFS a few times, never bit the bullet Had to find new storage system for work, chose Nexenta Core Platform (now Illumian) Switched to NCP at home as well, using mirrors of 2 and 3 disks in a large, single pool
Wrapup Lots (too many?) of options for home storage Not necessarily cheaper just for backup Provide backups for friends and family using Crashplan, at no cost Local file storage much quicker than cloud