News for hackers. Stuff that matters.
Re:An Introduction to ZFS by louwrentius on Friday November 20, 2020 @03:16AM (#25158360)
ZFS is an awesome filesystem, this article gives a good overview. Very interesting for home users. I run it on my NAS.

It is very important to realize that you can’t expand VDEVS. You can only add VDEVS.

This makes expanding storage less flexible than regular MDADM RAID. Going for Mirrors is the most flexible but you lose 50% of capacity.

You also get the random IOPs performance of a single drive per VDEV. Sequential performance does scale within a VDEV.

You scale random I/O performance by adding VDEVS.

For home usage, as a NAS, you don’t need to use SSDs for a SLOG, unless you have write-intensive random I/O workloads.

Re:An Introduction to ZFS by MarkSweep on Friday November 20, 2020 @03:35AM (#25158470)
Strictly speaking you can expand a VDEV. If you replace all the leaf drives in a VDEV with larger devices, you can extend the VDEV to take the extra space.

There is work ongoing to allow adding drives to a RAIDZ VED[1]. And VDEV Removal gives some flexibility (at the cost of indirection on the read path). But yeah, ZFS is not super flexible for adding and removing random drives.

[1]: https://github.com/openzfs/zfs/pull/8853

Re:An Introduction to ZFS by louwrentius on Friday November 20, 2020 @03:57AM (#25158613)
Yes that's totally true, but it is very time consuming and cumbersome in my view.

I've detailed this also here. https://louwrentius.com/the-hidden-cost-of-using-zfs-for-you...

You may also realise from the top of my article that expanding VDEVS is a topic since 2017 but still no show.

Re:An Introduction to ZFS by iooi on Friday November 20, 2020 @12:30PM (#25162709)
I just did this operation on moving from 12x 4TB drives to 12x 16TB drives. I did it over 12 days (each resilver was ~24hours) and it wasn't too cumbersome. Considering how often you're going to grow your pool, it doesn't seem like a big drawback.
Re:An Introduction to ZFS by louwrentius on Friday November 20, 2020 @08:23PM (#25167110)
Fair enough
Re:An Introduction to ZFS by bonestamp2 on Friday November 20, 2020 @03:24AM (#25158404)
Which OS are you using for your NAS? I ruled out Unraid and I'm looking at TrueNAS Core (FreeNAS). I want it mostly for scalable backup storage where I can easily add/remove drives from the array as I need more storage. Any others that I should consider?
Re:An Introduction to ZFS by louwrentius on Friday November 20, 2020 @04:08AM (#25158693)
I am using Debian with ZoL, but it's now ancient. Just ZFS + NFS & SMB.

https://louwrentius.com/71-tib-diy-nas-based-on-zfs-on-linux...

I would now go for Ubuntu + ZoL myself. I bought all capacity up front and paid the ZFS tax. So I don't need to expand as I go.

But if you want to expand as you go, Linux + MDADM are still fine in my opinion. It's a tradeoff. Do you want to 'pay' the ZFS tax and expand at a cost, or do you want a bit more risk (on paper) but more flexibility?

I ran a RAID6 of 20 drives before that using Linux + MDADM and that worked fine. And MDADM allows you to expand as you go, exactly as you want.

I think the risks ZFS protect against are very small.

https://louwrentius.com/what-home-nas-builders-should-unders...

Re:An Introduction to ZFS by tinco on Friday November 20, 2020 @05:09AM (#25159007)
Unraid is the only one that allows you to freely add/remove drives. And only if you're OK with taking cluster down for a minute as you add it.

If you true freedom you'd need to go Ceph(FS) based, but there's no GUI to manage a Ceph cluster that I'm aware of, and they're so focused on cloud usage that you'll find little guidance on single-node clusters.

edit: Forgot to read Louwrentius' comment, of course MDADM with raid 6 allows you to add disks in pairs of two, so that's definitely also an option. Don't forget to use LVM, it can get real awkward without it.

Re:An Introduction to ZFS by magicalhippo on Friday November 20, 2020 @09:20AM (#25160628)
> Unraid is the only one that allows you to freely add/remove drives.

I think technically you can do something similar to Synology's Hybrid RAID[1] with ZFS, using partitions and creating mirror VDEVs from the partitions.

But I haven't gone through all the details so could be there are some risks I overlooked and in any case you'd have to manage it from the command line so would be a bit tedious. But I did set up a proof of concept in a VM.

[1]: https://nascompares.com/guide/shr-synology-hybrid-raid/

Re:An Introduction to ZFS by nix23 on Friday November 20, 2020 @05:23AM (#25159085)
Don't forget GlusterFS, example: Every single disk is a ZFS-pool on top of that you run your GlusterFS-Cluster, GlusterFS then looks for the redundancy and underneath you can export/import your pools aka Disks and or vdevs (if you need more performance).

>but there's no GUI to manage a Ceph cluster that I'm aware of

There are two, Open Attic and Calamari ,but Open Attic is in maintenance mode, so all the work goes now into Ceph Dashboard:

https://docs.ceph.com/en/latest/mgr/dashboard/

Re:An Introduction to ZFS by ComputerGuru on Friday November 20, 2020 @11:22PM (#25167860)
> You also get the random IOPs performance of a single drive per VDEV

This isn't strictly true. It is true for RAIDZ-n and for writing to a mirror, but ZFS can accelerate random reads with a mirror by distributing the read requests to the underlying disks.