ZFS gets you all of that and raid and snapshots and has been tested time and time again. Why run BRTFS then?

ZFS is not GPL? I dunno, I like feeling safe when I do kernel upgrades knowing even if for whatever reason my ZFS module doesn't compile/work under the new kernel, I won't be left without a root FS. I have been running BTRFS for 5 years with very little issues, and enjoy not having to compile a new dkms module with every kernel.

ZFS with DKMS is a disaster, at least in my experience. Honestly, I can't recommend ZoL unless you're running a distro with relatively stable kernel releases that don't change substantially or that happens to be supported by ZoL with binary packages. ZoL on Arch was... trying at times. It worked great, but my paranoia meant that I ended up adding the kernel to IgnorePkg to force manual kernel updates (mostly for my own memory). But then, it also meant having to build all of the ZFS packages (including SPL) tied to that specific version. This usually meant waiting until the AUR packages were updated as I figured that indicated someone else must have tested ZFS on that specific kernel version.

I remembered thinking DKMS might solve the problem, but I ended up having to use recovery media just to get an environment to reinstall an older kernel and let DKMS do its thing after a botched update started provoking panics. I suspect a version mismatch based on the errors but never investigated it beyond fixing the problem and moving to the prebuilt modules. Things may have changed, but the Arch ZFS+DKMS packages were a bit flaky and required some manual modification just to boot (should've taken this as a warning!).

Granted, it was my fault entirely for being a bit too enthusiastic with ZFS on Arch. To be honest, if I were to use it again, it would be on FreeBSD. Not Linux. I recognize it's fine for other people, but in my use case it wasn't.

> ZFS with DKMS is a disaster

It is more like Arch Linux is a disaster. Upgrading the kernel package replaces the current one! Come on, any distribution worth it's salt just installs new versions alongside and you can select any of them in the boot screen. This is a ridiculous packaging policy regardless of ZFS or any other DKMS modules.

And I acknowledged that using it on a different distro would be more advisable, although I still stand by my claim that FreeBSD is far more appropriate for ZFS.

I will, however, agree that having no fallback to the prior kernel version is a problem. In practice, it's never caused me much trouble except when I do something stupid like using ZFS from the AUR. initrd generation has historically seemed to be more problematic under Arch, but I'd argue that's mostly fixed with install hooks.

In all honesty, it was probably more the fault of the zfs-dkms packages than it was either the kernel packaging policy or ZoL+DKMS itself (for reasons I elaborated on in my original post).

But, that's also what you get when you use packages from the AUR or using a distro like Arch for something that really only benefits from a wider installation base (like Ubuntu does, for instance).

I know you acknowledged that using it on a different distro would be more advisable, I just wanted to vent about more broad issue of their packaging policy. Sorry if it wasn't clear.

Oh, I understood where you're coming from.

I do agree. There are circumstances where Arch's packaging is brain dead (they only recently, within the last 2 years or so, started validating packages against signatures!). I use it for a number of applications, and as my desktop OS among others. However, I'll freely admit at least part of my choice is perhaps the fault of masochistic tendencies. After all, I migrated to Arch from Gentoo, and I used Gentoo for years! :)

In all honesty, I've been bit more by the initrd and mkinitcpio's failings than the lack of a fallback kernel. That's mostly fixed with packaging hooks that essentially guarantee it will run, but it's still a problem with the ZFS packages and may require running it manually (which is annoying). However, that wasn't always the case, and sometimes the generated initrd would be missing something important. You can imagine what happened next.

Your problem was running it on a distro the didn't support it. (As you acknowledge.)

So far it's been running absolutely great for me on several Ubuntu installs.

Indeed. Arch is good for a few things, but sometimes stability isn't one of them when it comes to unsupported packages. ;)

I'm not sure I'd be brave enough to run ZoL again, but given Ubuntu's FAR wider install base and availability of binary packages, it's the better option if you have to choose.

My personal preference would be to stick with ZFS on FreeBSD. Performance is probably better.

I have been running ZFS on Arch Linux for a little over two years across four drives, and doing weekly updates. Haven't ran into any issues yet.

I'm actually surprised by this, but I'd wager that you also didn't use the DKMS AUR packages. I also suspect you wait for the zfs-linux (etc) packages to match the kernel version before updating. Or you manually bump the kernel version and build it, hoping for the best (edgy!).

Arch Linux has a LTS kernel available in official repo. Why not using that?

I considered it, but I have some problems with the Arch LTS release cycle. If I were to choose an LTS kernel, why not just dump Arch and go with an Ubuntu LTS, which has better long term support?

The other problem is that at the time, the ZFS packages for LTS were pinned at a version that had a known issue with arc_reclaim encountering a deadlock essentially causing the file system to become unresponsive after a substantial transfer (think rsync).

Now, obviously, it wouldn't be that difficult to modify the PKGBUILD to pull a newer version of ZFS, but there's a point in time where the maintenance required to update starts to outweigh whatever benefit you can glean from the LTS kernel.

That's not the case now since the LTS packages appear to be at v0.6.5.9, which has the fixes, but I don't remember this being true about a year ago.

Ahhh yes that is the one ma in n point I missed. Can't argue with that. I've just never had issues upgrading as the zfs on Linux project us fantastic.

At the time of the system build, ZFS was user space only on CentOS7 if I recall.

BTRFS lets you make CoW copies of files. You can even retroactively merge the blocks that store identical files. BTRFS also makes it not a giant pain to remove a file from snapshots.

ZFS does seem to work better overall, but I wouldn't call either filesystem great at this point in time.

Why the remount,clear_cache ? I'm doing a monthly balance before I got problems on a few servers.

It's a little black magic -- didn't have time to completely research (this is a home NAS system.)

I believe there was a bug in the clear space cache. This could cause the system to think it didn't have free blocks... you'd have to mount another device to create more space in order to rebalance and fix.

Eventually I saw a bug fix report about a corruption in the cache... I never investigated to see if my current kernel has the fix.