bcachefs seems to have a much more coherent design.
[1] "Oh, yeah, I don't know how to handle this code path yet, let's stick a BUG_ON in there! I'm sure we'll figure something out later."
Status page says so: https://btrfs.wiki.kernel.org/index.php/Status
- RAID 1 with more than 2 disks is not what you think it is: the data will be mirrored but only once, no matter how many disks you have (meaning if you have a mirror with 3 disks, you only have 2 copies of your data). Because in BTRFS lingo, RAID 1 means '2 copies of the data' https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_dif... which is not what people expect from RAID 1 with more than 2 disks
- RAID 1 always needs 2 disks to be working, if not you can't mount the thing... Well you can, but only once... https://btrfs.wiki.kernel.org/index.php/Gotchas#raid1_volume...
- RAID 10 inherits these special RAID 1 cases as a result
Most of that stuff is described here: https://btrfs.wiki.kernel.org/index.php/Status
So as they say on the status page: "mostly ok"
My experience has been mixed, but haven't had any data loss. There was a bug for a while regarding free space so occasionally the system would seem to be full but wasn't ....and it was a real pain to correct it.
I now have a cron job that does a monthly btrfs balance along with a mount -oremount,clear_cache. I also run the latest kernels from http://elrepo.org instead of the CentOS7 kernels so that I get the latest patches.
I remembered thinking DKMS might solve the problem, but I ended up having to use recovery media just to get an environment to reinstall an older kernel and let DKMS do its thing after a botched update started provoking panics. I suspect a version mismatch based on the errors but never investigated it beyond fixing the problem and moving to the prebuilt modules. Things may have changed, but the Arch ZFS+DKMS packages were a bit flaky and required some manual modification just to boot (should've taken this as a warning!).
Granted, it was my fault entirely for being a bit too enthusiastic with ZFS on Arch. To be honest, if I were to use it again, it would be on FreeBSD. Not Linux. I recognize it's fine for other people, but in my use case it wasn't.
It is more like Arch Linux is a disaster. Upgrading the kernel package replaces the current one! Come on, any distribution worth it's salt just installs new versions alongside and you can select any of them in the boot screen. This is a ridiculous packaging policy regardless of ZFS or any other DKMS modules.
I will, however, agree that having no fallback to the prior kernel version is a problem. In practice, it's never caused me much trouble except when I do something stupid like using ZFS from the AUR. initrd generation has historically seemed to be more problematic under Arch, but I'd argue that's mostly fixed with install hooks.
In all honesty, it was probably more the fault of the zfs-dkms packages than it was either the kernel packaging policy or ZoL+DKMS itself (for reasons I elaborated on in my original post).
But, that's also what you get when you use packages from the AUR or using a distro like Arch for something that really only benefits from a wider installation base (like Ubuntu does, for instance).
I do agree. There are circumstances where Arch's packaging is brain dead (they only recently, within the last 2 years or so, started validating packages against signatures!). I use it for a number of applications, and as my desktop OS among others. However, I'll freely admit at least part of my choice is perhaps the fault of masochistic tendencies. After all, I migrated to Arch from Gentoo, and I used Gentoo for years! :)
In all honesty, I've been bit more by the initrd and mkinitcpio's failings than the lack of a fallback kernel. That's mostly fixed with packaging hooks that essentially guarantee it will run, but it's still a problem with the ZFS packages and may require running it manually (which is annoying). However, that wasn't always the case, and sometimes the generated initrd would be missing something important. You can imagine what happened next.
So far it's been running absolutely great for me on several Ubuntu installs.
I'm not sure I'd be brave enough to run ZoL again, but given Ubuntu's FAR wider install base and availability of binary packages, it's the better option if you have to choose.
My personal preference would be to stick with ZFS on FreeBSD. Performance is probably better.
The other problem is that at the time, the ZFS packages for LTS were pinned at a version that had a known issue with arc_reclaim encountering a deadlock essentially causing the file system to become unresponsive after a substantial transfer (think rsync).
Now, obviously, it wouldn't be that difficult to modify the PKGBUILD to pull a newer version of ZFS, but there's a point in time where the maintenance required to update starts to outweigh whatever benefit you can glean from the LTS kernel.
That's not the case now since the LTS packages appear to be at v0.6.5.9, which has the fixes, but I don't remember this being true about a year ago.
ZFS does seem to work better overall, but I wouldn't call either filesystem great at this point in time.
I believe there was a bug in the clear space cache. This could cause the system to think it didn't have free blocks... you'd have to mount another device to create more space in order to rebalance and fix.
Eventually I saw a bug fix report about a corruption in the cache... I never investigated to see if my current kernel has the fix.
Also been using it as / since 2012 with no issues.
Makes me wonder if anyone's tried something like Jepsen for filesystems.
I use it pretty much every day.
It is ok for a single disk FS. It is no ZFS though which is its largest problem, people keep marketing it as "linux awnser to ZFS"
No, it is not.
Maybe some day, but today it is no ZFS. I love zfs..
Further the ZFS utilities are far easier to use and understand. zfs and zpool commands well documented, and intuitive. btrfs utilities are are not, IMO
I am fine with using btrfs as a replacment for ext on my OS drive, but for my large data arrays of multiple disks it s ZoL all the way
I moved back to EXT4 and it never happened again since then.
From 2012-2014 it was mostly breakage every other month. From 2014-2016, it was semi-annual issues.
For the last ~18 months I have had ~30 machines running btrfs with no issues, some servers, some personal computers. The release notes are boring, the bugs are boring, and to me its definitely in a state I would strongly consider trusting it with any workload.
I worry that btrfs is just going to remain doomed. It wasn't stable half a decade ago, so it - for some reason - cannot be more stable now. But it has seen so much work put into it to make it as mature as it is now, and in my experience it is pretty damn mature now. All I want to see is another year and a half of perfect stability before I would start arguing to drop zfs entirely.