News for hackers. Stuff that matters.
Re:ZFS for Dummies by paulmd on Tuesday September 05, 2023 @03:44AM (#37388984)
this is honestly hard because many of the decisions that matter are not things you type into zfs at all (except incidentally).

how many disks per vdev? how much memory? etc

a lot of the things you've outlined are not universal at all, just situational

Re:ZFS for Dummies by istjohn on Tuesday September 05, 2023 @04:09AM (#37389118)
Yes, there is a lot of essential complexity that is unavoidable, but there are a lot of people like me who just want a better desktop file system, and we don't need to know about SLOGs and L2ARCs and the half dozen compression algorithms, etc. It's situational, but it's a common enough situation that a targeted solution would be valuable.
Re:ZFS for Dummies by MrDresden on Tuesday September 05, 2023 @04:18AM (#37389162)
I'm one of those who wants a better desktop file system but have stayed away from ZFS (at least for the time being) due to stories about complexity.

Would you say that the defaults are sane enough for that kind of person (no real configuration needed)?

Re:ZFS for Dummies by istjohn on Tuesday September 05, 2023 @04:43AM (#37389325)
No, not really. There's a lot of unavoidable complexity like needing to understand what ZFS pools, vdevs, and datasets are and how they relate to each other. And then if you want to encrypt your data that adds another bit of functionality you need to configure. And then there are just a lot of things that aren't that important for most people, but you won't know if it's important for you until you understand what they are.

I would say the learning curve is similar but perhaps a bit steeper than configuring vim for web development for the first time or configuring nginx and let's encrypt on a webserver for the first time.

It didn't help that I started by trying to setup root on encrypted zfs. I would recommend first using zfs unencrypted, then adding encryption once you're comfortable with that, and finally trying /home on zfs or even root on zfs.

Re:ZFS for Dummies by 0mp on Tuesday September 05, 2023 @05:27AM (#37389624)
The defaults are absolutely sane. I've been using FreeBSD with root on ZFS on all sorts of workstations and laptops for many years now. It's always run just fine, even on cheap laptops with 2 GB of RAM.
Re:ZFS for Dummies by istjohn on Tuesday September 05, 2023 @06:19AM (#37389926)
I think the key thing to note there is that you are on FreeBSD. The Linux integration isn't nearly as mature.
Re:ZFS for Dummies by 5e92cb50239222b on Tuesday September 05, 2023 @05:53AM (#37389786)
I switched away from ZFS on root after using it for about a year. There were no actual problems, but I could never get over the fact that it's so complicated underneath I have to relearn everything every time I do any sort of advanced maintenance.

For example, I had to move between SSDs which is an absolutely trivial operation with any other filesystem if you know your Linux well. It took me maybe three hours of work on ZFS because it brings its own environment and does everything its own way, so I had to remember and re-discover a lot of stuff. The system booted on maybe 15th try.

If you are fine with this, or you're not going to use it on root, or you are a FreeBSD user, it's an excellent choice for pretty much any task.

Re:ZFS for Dummies by vetinari on Tuesday September 05, 2023 @08:33AM (#37390796)
I have zfs root on ubuntu; moving to another ssd was trivial: add the second drive, recreate the partition scheme (esp, bpool, rpool), add them as mirrors to existing pools, and while waiting while it mirrors up, reinstall grub on the new esp. After mirroring is done, check fstab for UUIDs, reboot, check, whether everything is ok, remove the old drive from the mirror. Done.

Though I consider zfs to be nice, I'm willing to use it only with distributions, where it comes with the distribution, built for the distro kernel (i.e. ubuntu or proxmox). In the past, I've wasted too much time with dkms, failing to build the kernel modules, or kABI modules failing to load, or other problems that left me without root filesystem at reboot. That's not an experience I'd like to repeat. Also, btrfs is nice too ;).

Re:ZFS for Dummies by waynesonfire on Tuesday September 05, 2023 @11:51AM (#37393446)
Is the integration of root zfs on linux complicated? I saw some info-graphic and it looked like there were a lot of pieces to stitch it together. Is it fragile?

Regardless, I'm surprised your one-year with ZFS didn't convinced you that it's worthwhile to invest in its mastery. The filesystem will pay dividends for your entire compute life-time and not only save your data but hours of maintenance over that time.

Major downgrade moving away from it in my view.

On a side note, FreeBSD is great and worth investing in as well. I still have more to learn but the journey has been wonderful and I will not be using Linux in the next iteration of my home / lab server and hobby projects.

Re:ZFS for Dummies by Modified3019 on Tuesday September 05, 2023 @06:07AM (#37389861)
Outside of it being a good idea to manually set `ashift=12` on each vdev creation (because disks lie and zfs’s whitelist isn’t omniscient. I’ve seen too many people burned by this) and `atime=off` for datasets (because burning IO updating the metadata because you’ve accessed something is just fucking stupid), the defaults are sane and you can basically refuse to care about them after that.

Every system I’ve used has `compression=on` set as default, which currently means lz4. People set it manually out of paranoia from earlier days I think.

For linux systems you can set `xattr=sa` and `acltype=posixacl` if you like, which offers a minor optimization that you’ll never notice.

I suppose if you don’t like how much memory zfs uses for ARC, you can reduce it. For desktop use 2-4gb is plenty. For actual active heavier storage use like working with big files or via a slow HDD filled NAS, 8GB+ is a better amount.

Dataset recordsize can be set as well, but that’s really something for nerds like me that have huge static archives (set to 1-4MiB) or virtual machines (match with qcow2’s 64KiB cluster size). The default recordsize works well enough for everything that unless you have a particular concern, you don’t need to care.

I should note, beware of rolling release linux distros and ZFS. The linux kernel breaks compatibility nonstop, and sometimes it can take a while for zfs to catch up. This means your distro can update to a new kernel version, and suddenly you can’t load your filesystem. ZFSbootmenu is probably the best way to navigate that, it makes rolling back easy.

You also want to setup automatic snapshots, snapshot pruning, and sending of snapshots to a backup machine. Highly recommend https://github.com/jimsalterjrs/sanoid

If you really find yourself wanting to gracefully deal with rollbacks and differences in a project, HotTubTimeMachine (HTTM) is nice to be aware of: https://github.com/kimono-koans/httm

Re:ZFS for Dummies by rhinoceraptor on Tuesday September 05, 2023 @05:53PM (#37398284)
If you install on ZFS from the installer with Ubuntu, FreeBSD, Illumos, etc. there's no reason to care about the complexities under the hood if you don't care about them with any other file system.

It's like saying you should know how virtual memory and pre-emptive multitasking work if you want to upgrade from Classic Mac OS to OS X.

Re:ZFS for Dummies by mastax on Tuesday September 05, 2023 @05:13AM (#37389534)
`compression=on` exists.

You don't need to know about SLOGs and L2ARCs. They're not super useful generally but people geek out a out them.

I know those were just examples and I don't disagree with the general point.

Re:ZFS for Dummies by rhinoceraptor on Tuesday September 05, 2023 @05:46PM (#37398210)
Why would you want to know about SLOGs or L2ARCs if you're using it on a desktop? Even if you are using it for a NAS, you almost certainly don't need to know about those.
Re:ZFS for Dummies by LanternLight83 on Tuesday September 05, 2023 @04:14AM (#37389137)
I get that it's meant but be wrapper for common use cases, but I use ZFS on root and had to craft a custom initfs. The other nice-ities sound great (some BTRFS inspiration?), but some of these are at distro-devel level.
Re:ZFS for Dummies by istjohn on Tuesday September 05, 2023 @04:57AM (#37389419)
I looked into doing root on ZFS, but I ended up settling for /home on ZFS. Maybe I'll revisit it next time I build a new system when I'm more comfortable with ZFS.
Re:ZFS for Dummies by realusername on Tuesday September 05, 2023 @04:13AM (#37389131)
I'm also firmly on the "convention over configuration" mindset, sensible defaults should be available and anybody else can still tinker if needed. I also admit that setting up zfs wasn't easy for me (and I'm sure I haven't done it 100% properly)