* A disk failure where some moderate number of sectors become unreadable is far more likely than a disk going completely toast.
* Even if I were running raidz2 or raidz3 I'd still want the offsite backups in case of some disaster befalling the entire array (or more likely, me accidentally rm'ing something)
* It's a home fileserver used mostly for archival purposes. Unlike a business scenario, the loss of availability while I restore from backup is not going to hurt too badly.
Did I miss something here or do you think this is sound?
All the striped modes will (in effect) limit write rates to the bandwidth of a single disk, but ZFS is good about this: So long as the writes are reasonably large, as they'll likely be for a media server, you'll usually get the maximum streaming bandwidth of the disk. So, 150-200 MB/s. Reads are much faster.
It's not something you'd want to use in a DC, but for mostly-idle storage at home it's perfectly fine. Add in an SSD cache, and it'll be downright speedy.
That gives an effective capacity of 64 TB, given 8TB disks, and the chance of four disks dying before you can replace even a single one is pretty low. The usual limitation of ZFS applies, however: You have to decide the number of disks up-front, as you can't expand the size of a RAIDZ vdev later on.
(Well, you can, but only by replacing all the disks with larger versions.)
IOPS aren't cheap, and you will find out just how much that matters the moment you have to replace a disk in a pool with a lot of data in it. Resilver times of days can be pretty agonizing in a home setting.
I won't build a pool without at least double redundancy per rotating-drive vdev, meaning at least 3-way mirrors (zpool create foo mirror disk0 disk1 disk2 mirror disk3 disk4 disk5) or raidz2, as I have had to go to backups for failures during resilvers of raidz (never again!) and 2-way mirrors.
3-way mirrors conveniently provide high read IOPS. OpenZFS does a good job aggregating and scheduling writes to the point that writes are mainly choked by random reads. (Very full pools induce quasi-random reads while hunting for space in metaslabs, which is the main reason writes to very full pools perform poorly).
2-way mirrors are only really fine for vdevs that have negligible random-access penalty (and thus likely high IOPS), or where rebuilding an entire pool from backups takes time comparable to resilver and scrub times when dealing with a replacement leaf device that might fail during resilvering (you cannot know the data on the replacment device is OK until you have read it, which generally means scrub-after-resilver), or errors reading the surviving leaf device(s).
There is an article somewhere calculating the probability of two drives failing during resilvering and how this gets worse with the ever increasing capacities, which increases the resilvering times.
But I guess you already account for this by using a Raid 6 with +2.
Triple parity can give you pretty good integrity, but at a certain point it's wasted effort. Disk failures cluster too much for any single array to be safe; you need backups.
For just 10TB, I would recommend that most home users stay away from RAID and buy 3 Seagate 10TB drives (street price currently $399 each) for total of ~$1200. Use 1 drive as primary copy and the 2 others as backups. With 3 drives, you could even rotate the 2 backups to an offsite location.
I quit using RAID in 2007. All of them FreeNAS/drobo/Synology/etc required more troubleshooting and babysitting than I wanted to deal with. It didn't matter if it was Linux software or dedicated hardware (NAS appliance).
And how do you plan to detect if your data is silently corrupted? Unless you're using some archive file format that includes checksums/hashes, you have no way of knowing whether your backups are any good until it's too late.
If you meant that a filesystem like ZFS with checksum blocks is designed to protect itself agains corruption errors, one can use that without RAID. A single-disk ZFS is orthogonal to RAID.
To point back to the OP's question, he's running Windows with Plex. In that case, a 10TB NTFS filesystem with periodic binary comparisons/checksums (or format the disk with the newer ReFS[2] that stores checksums similar to ZFS) is simpler than RAID5/6. He also gets the extra benefit of an offsite backup without paying for cloud storage.
I'm not against RAID for all cases but a home media computer with just 10TB doesn't meet the threshold of dealing with its extra complexity. However, if one is wanting to consolidate 25TB+ of disk space -- or running a business -- or need aggregated io bandwidth, the cost/benefit drivers for RAID over JBOD would make more sense.
[1]https://www.google.com/search?q=raid+data+corruption+synolog...
[2]http://www.windowscentral.com/how-use-resilient-file-system-...
You may not need that feature, but it's a very useful option to have.
The downside for media storage is that a straight NAS is often incredibly slow compared to USB3. For decent speed you need iSCSI, which adds cost and complication.
* zfs for snapshots (and send my inc snapshots from my laptop) * zfs fs with snapshots to rsync non-zfs enabled devices (read: wife's mbp && imac) * rpi3 as brain (slow, but works) * 2x2Tb drives on a zraid
It took me a while to get it to where I wanted, but it was cheap (~230), low power and safe enough for home. I recently did a full reinstall from scratch and it took me less than 15 min to get my system back (I have my /home as a zfs fs backed up).
This comment should not be understood as offensive. I am just interested where one draws the line.
[1] https://www.synology.com/en-global/knowledgebase/DSM/tutoria...
You always want 3 copies: one on your computer, one onsite and one offsite. If there is a fire or everything electronic is stolen from your house, you still have the offsite.
The only hassle so far has been backups. The built-in backup apps can't span media (external USB drives). Something I used to do decades ago with 3.5" floppies... So I have to do a file-copy "backup" across the network.
You probably need a NAS. To store 10TB securely you are looking at a bill of materials of about ~1200usd. This includes a NAS from synology and 3 6TD Western Digital Red drives. This will give you 12TB of storage space running in a RAID5 configuration.
Synology DS416 NAS diskless: 365USD WD Red 6TB: 253USD (x3) Total: 1124
For a little more you could pick up a HP Microserver for just under 400USD and run ZFS on a pool with SMBD and get about the same results but with more hands on fun.
Alternatively, just aim your requirements a bit lower (e.g. only 6TB) and pick up a 2 disk nas and 2 drives running as a mirror and it will be much cheaper (~700-800USD).
Resources: http://www.raid-calculator.com/default.aspx