I am absolutely with you, FreeBSD is great!

>It's a great shame that the industry has monopolised on Linux because FreeBSD brought

No sure about that, i think it better the way it is, Linux (the ecosystem) is such a bloatware because it implements every single feature the Industry needs, then having userlandtools like toolbox, busybox or Gnu, 3 different libC's, 5 different MAC-Framework, around 50 different local filesystems (but just 3-4 are stable enough for daily use)....and so on.

If FreeBSD were as popular as Linux, it would have same problem.

We don't even need hypotheticals, how many proprietor Unices were there before Linux? Forget history, how many libre Unices are there now? Off the top of my head I can count Free/Net/Open/Dragonfly BSD and may be missing one or two.

Only difference is Linux provides a-la-carte experience, while BSDs provide the bundled options. But zoomed out, none bloat doesn't appear to me as an issue in choice/decision making department.

>If FreeBSD were as popular as Linux, it would have same problem.

That's what i meant.

>Forget history, how many libre Unices are there now?

I fail to understand what you wanna say.

> native ZFS support is FreeBSD's killer feature

Why? I never cared about my filesystem. Does it make a difference what filesystem am I using?

I hosted my coding environment on a VPS running FreeBSD and happen to jump between VPS provider quite a bit due to location and pricing. My migration method was simply:

    $ zfs snapshot zroot/home@migrate
    $ zfs send -R zroot/home@migrate | ssh newhost zfs recv zroot/home

ZFS automatic snapshots (via zfstools[1]) can also saved hours of headaches trying to recover an older version of a file. Since snapshot appears as a regular mount (ZFS even automatically expose them under /path/to/mount/.zfs/snapshot), all existing tooling just works (cp, cat, running diff against snapshot, etc.).

My Linux machine these days also runs ZFS, but the level of integration with the system is still far from that of FreeBSD's. That can range from something as simple as ZFS ARC stats in top, to native support for ZFS Boot Environment allowing the system to boot directly from snapshots or do an offline upgrade of a clone of a live system[2]. (On Linux side of things there's zfsbootmenu[3] which provides something similar but the project is still quite new.)

The filesystem may not matter that much, and I'm sure many these features can be replicated with LVM, but having a unified tooling and community around it really makes quite a difference in quality of life as an end-user.

[1]: https://github.com/bdrewery/zfstools

[2]: https://vermaden.wordpress.com/2021/02/23/upgrade-freebsd-wi...

[3]: https://zfsbootmenu.org/

Not sure if it's still relevant (article is quite old after all), but does [1] still hold true in regards to /boot on ZFS on linux, even with zfsbootmenu? What kind of issues can I expect? Is there a newer/better knowledgebase compared to the one on pthree.org?

I'm guessing the newness/missing features of zfsbootmenu is in regards to beadm [2]?

[1] https://pthree.org/2013/01/03/zfs-administration-part-xvii-b... [2] https://vermaden.files.wordpress.com/2018/11/nluug-zfs-boot-...

GRUB2 works fine with /boot on ZFS, however GRUB cannot detect ZFS dataset with native ZFS encryption (see bug[1]). The best KB right now is probably OpenZFS documentation[2] and Arch Wiki[3].

I've been using ZBM since its inclusion in void-packages. It already does everything I wanted it to, but I don't think its usage is widespread outside Void Linux community just yet. Being dependent on Dracut might be one of the reason why (I believe Arch is still using mkinitcpio). There's installation instruction for Void and Debian Buster on ZBM GitHub[4]. I used to have a separate ZBM root for void-musl and void-glibc which worked quite well (though I'm now running only void-glibc due to Nvidia).

[1]: https://savannah.gnu.org/bugs/?func=detailitem&item_id=58270

[2]: https://openzfs.github.io/openzfs-docs/Getting%20Started/ind...

[3]: https://wiki.archlinux.org/title/Install_Arch_Linux_on_ZFS

[4]: https://github.com/zbm-dev/zfsbootmenu/wiki

Well I am running Ubuntu 20.04 and have experienced boot issues after update multiple times. Currently my laptop is not directly bootable as zpools are imported in wrong order / not imported at all so I have to import them manually. As there is not enough information how the freaking boot works and I am not prepared to study grub2 to figure it out and I am in this situation for 6 months.

This is my last root ZFS install on linux.

I had problems with fedora before (where at least ZFS wasnt supported and I was expecting problems), I am having problems with Ubuntu now and I never had any issues with zfs root on FreeBSD in ~15 years.

I am sick of distributions "moving fast and breaking things" on my back.

How do ZFS snapshots compare to LVM snapshots? Question not specifically for, but anyone who might know something about this.

Night and day. I hardly hear of anyone using LVM snapshots except in special cases, but ZFS snapshots for many people are like brushing one's teeth: a habit that is done multiple times a day.

Using them for system updates/upgrades is built right into FreeBSD (copying a feature from Solaris):

* https://www.freebsd.org/cgi/man.cgi?beadm

If the changes fail just rollback the system to the previous state/snapshot (including kernel changes) via the boot loader.

Services like rsync.net allow one to have offsite backups with easy, and the destination can be encrypted such that the remote end can't see your data. All through an SSH pipe.

Do you use Linux as a daily driver?

I used Linux as a daily driver until the end of 2020 (Void Linux + ZFS + ZBM), but now running M1 Mac as a primary machine. I'm still using Linux regularly though (gaming via Proton, which is why I'm not running FreeBSD on a desktop yet).

I can't speak for your computing experience, but for mine:

There are significant differences in performance, admittedly SSDs temper these differences but there are plenty of systems with large HDDs. This isn't an abstract performance benchmark thing, I feel it when doing operations on large numbers of small files.

There are significant differences in features that impact how you use the system. For instance, sometimes I like to backup a directory I'm working in (typically before some funky git operation), normally that can be done with cp or tar, but that's slow. Much faster is a manual filesystem(/volume manager) snapshot. But I have to remember to do that. Much more convenient would be if I didn't have to explicitly create a snapshot, which HAMMER would offer me.

All non-temporary HAMMER filesystems in DragonFly by default automatically maintain 60 days worth of 1-day snapshots and 1-day worth of fine-grained (30-second) snapshots. - https://www.dragonflybsd.org/features/#index3h2

It sure does. Compression and encryption and snapshotting and bit rot protection all native to the file system and the tools around it instead of having to cobble together lvm and mdraid and such.

Snapshotting imo is one of the best things. Copy on write file systems really are awesome. And FreeBSD adopting ZFS so early was a huge win imo.

Depends. In a perfect world it doesn't matter. However in this imperfect world it is nice to know that there is an automatic backup of your files from a few days ago, and if a hard drive crashes you just replace it (requires paying for redundant hard drives!). The ability to add more space to an existing filesystem is nice just in case.

There are a lot of other features that might or might not matter to you. If the above don't it is because you haven't been burned yet in life. (Or you use one of several other filesystems that support those features)

Short answer - automatically cache frequently-accessed files on expensive but small NVMe drives while using bigger cheaper slower drives for mass storage.

Yes, I agree, and sorry for the ignorant question. Yet a lone "educate yourself" may be a bit harsh on a site where informal technical conversation happens. Care to share a couple of "killer" features of ZFS for somebody who has never cared about the FS their files are stored in?

Too much to write and too many features, that's what i meant with educate yourself:

https://en.wikipedia.org/wiki/ZFS

The FreeBSD project doesn't have the manpower to maintain ZFS. So FreeBSD is now relying on zfsonlinux.org, the Linux port of ZFS. In general, FreeBSD is increasingly having to rely on ports of Linux things, or emulating Linux things, because without enough volunteers or paid professionals to write native BSD solutions that is the only way to stay in the game. (Linus, too, complains of lack of maintainers for Linux.)

EDIT: Adding reference: "FreeBSD moving to ZFS-on-Linux"

https://forums.freebsd.org/threads/freebsd-moving-to-zfs-on-...

FreeBSD is a first-class citizen in the OpenZFS project. It's not a hack or compatibility layer on top of the linux codebase.

It makes a great deal of sense for the Linux and FreeBSD communities to collaborate on ZFS rather than both maintaining their own separate forks and merging back and forth. It means you much more easily maintain feature & bugfix parity, and makes migrating filesystems between OSes easier.

It's a win for everyone involved, not a negative.

True, but Linux is also a first-class citizen in the OpenZFS project. The argument that was made earlier is that FreeBSD is so nice, because of its superior ZFS integration. This has been true for a long time and as such I agree with OP.

However, since the ZFS code basis is now the same for Linux and FreeBSD (which I also agree is a good thing) I don't think we will have the same superior FreeBSD ZFS argument anymore. And people who are in favor of FreeBSD over Linux have to provide other pro arguments than ZFS in order to make a case for FreeBSD.

Sure, the ZFS features are the same, but the integration is still not all there on Linux, and is on FreeBSD.

FreeBSD has features like 'bootonce', to switch ZFS boot environments on the next boot only. Better integration with the boot loader, more boot environment tooling, and other things that can only come when the filesystem is part of the OS.

I agree there are still some FreeBSD only features for ZFS, but the gap is getting smaller and smaller. So unless you are really depended on some of those features, there is no particular reason of why to prefer FreeBSD over Linux, unlike a few years ago.

You've literally just read the headline and not the content. What's happening is that FreeBSD are now collaborating with ZoL rather than the two teams working in parallel to create the same features -- thus needlessly duplicating effort.

To any sane person this news should be viewed as a victory not a loss.

    [W]orking through the git history of ZoL I have also discovered that
    many races and locking bugs have been fixed in ZoL and never made it
    back to Illumos and thus FreeBSD. This state of affairs has led to a
    general agreement among the stakeholders that I have spoken to that it
    makes sense to rebase FreeBSD's ZFS on ZoL.

    This port will provide FreeBSD users with multi modifier protection,
    project quotas, encrypted datasets, allocation classes, vectorized
    raidz, vectorized checksums, and various command line improvements.

From content, parent reply is correct. ZoL was far ahead FreeBSD's native ZFS port, that FreeBSD devs found that is better using ZoL instead.

Edit: Also related presentation by A. Jude at https://2019.eurobsdcon.org/slides/The%20Future%20of%20OpenZ...

> From content, parent reply is correct. ZoL was far ahead FreeBSD's native ZFS port, that FreeBSD devs found that is better using ZoL instead.

The FreeBSD fork also had a bunch of ZFS features that hadn't yet made it to ZoL as well as their own bug fixes too. I know this first hand from porting volumes between ZoL and FreeBSD.

FreeBSD didn't switch their code base and dump everything they had in their own fork. They switched upstream so that everyone worked on the same common code base and merged what was practical and kept any of their own OS dependent code separate. Just as I said in my previous post and just as the presentation you linked to also confirms. And for what it's worth, it's the OS dependant stuff that makes ZFS's experience better on FreeBSD than it is on Linux.

I also don't understand why so many people want to assume a negative spin on the story. Projects fork and change upstreams all the time when it's identified that there's a commonality they can share resources on. This isn't any different.

I don't see it as negative relying to an upstream with larger developer base, and as you said this allows better collaboration. But this dependence makes it hard for me to see how, though once it was, this is FreeBSD's killer feature. How is ZFS, now based on ZFS-on-Linux, more native on FreeBSD than Linux?

On the nativeness, as I understands:

ZFS relies heavily on Solaris VFS API, which is inherently different from both Linux's VFS and FreeBSD's VFS implementation (e.g., behavior of vnode[1]). Both FreeBSD and Linux implements this VFS API with a layer called Solaris Porting Layer (SPL).

In the original Illumos-based FreeBSD ZFS, this SPL layer was implemented as a kernel module (opensolaris.ko). OpenZFS originally maintain this shim in other repo, but later incorporate this shim into its codebase. FreeBSD port of OpenZFS involved importing SPL layer from FreeBSD tree into OpenZFS. On this part, there's not much difference in nativeness of the port.

However, this SPL layer also serve the purpose of adding one degree of separation between GPL code and CDDL code in order to comply with GPL in order to prevent ZFS code from becoming a derivative of the Linux kernel. The shim in OpenZFS is in three licenses; the Linux part of SPL is GPLv2, the FreeBSD part is 2-BSD, and the part that interacts directly with ZFS code is CDDL.

This is the area where FreeBSD can be more native with regarding to ZFS. Since CDDL is compatible with BSD license, FreeBSD kernel and its userland can integrate directly with ZFS without any licensing issue, e.g., FreeBSD bootloader can simply linked against ZFS, whereas GRUB2 must reimplement a part of ZFS in order to do probing.

[1]: https://wiki.freebsd.org/AndriyGapon/AvgVfsSolarisVsFreeBSD

For example, on Solaris and FreeBSD, it is possible to force-unmount a filesystem. on Linux it is not.

I'd already answered that -- both in my OP and in the comment you've just replied to. It's the OS specific code. Using ZFS on FreeBSD is as natural as using ext4 on Linux. Sure, ZoL exists and some distro's ship pre-compiled binaries for ZFS, but if your server drops into an emergency shell or even fails to boot entirely, if you need to boot from a live image (CD/USB) or your boot volume gets cocked up... fixing all of these requires extra hurdles on Linux if root is ZFS. Whereas that isn't the case on FreeBSD.

There's other OS specific features that FreeBSD does better with regards to ZFS such as ACLs too. But like with trashed volumes, they're edge cases.

FreeBSD also has better bundled tooling for working with ZFS. By this I don't mean `zfs` and `zpool` but little shell scripts et al that are FreeBSD specific.

Then there is the documentation. If you're running Linux then all the documentation regarding server management is about standard server management (ie with an ext4 or xfs root). But FreeBSD documentation will talk about how to do management with the assumption you're running ZFS. The FreeBSD docs (which are excellent, by the way) has pages and pages on ZFS outside of the stuff published by ZoL (which is pretty poor it has to be said) and OpenZFS. And I say the last points as someone who's had to manually compile kernel modules on Linux to get the latest patch to support a feature that made it to FreeBSD first.

I do have servers running ZFS + Linux and they work really well. So I'm not taking anything away from that architecture. But when things go wrong or you're trying to do something a little more specialised outside of basic file storage, ZFS has always felt like an awesome hack on top of Linux rather than an integral part of it's design. Unlike with FreeBSD where the same people managing the kernel and userland are the same people pushing ZFS as the default file system. If Torvalds, Pottering or others held ZoL in the same esteem then I'm sure the edge cases I've described would go away (Linux certainly has the larger developer community).

I see. It isn't the implementation, as I understood the argument to be about, but rather the environment around it. What you're concluding is basically the core difference between Linux distros and BSDs. Whereas the former are built as an assemble, the later define a base system making them more coherent. The better documentation can be said is result of this. Disappointingly the submitted article mentions nothing of those.

That article is much better:

https://vermaden.wordpress.com/2020/09/07/quare-freebsd/

That's a fair summary.

Today that's easy to answer: Try setting up ZFS boot with a mirrored boot drive and encryption on FreeBSD, then try it on Redhat, or SLES, or Ubuntu Server.

One of the experiences is very much "native" - the other is possible, but a GIANT pain to accomplish.

At some point I'm sure, Ubuntu at least, will make installing to ZFS on a server OS more straightforward. I've got several systems doing it today, but I can tell you it's not for the faint-hearted and it took a TON of trial and error to get it to do what I wanted consistently. I also cringe every time I apply an update because I know it's not something that is getting any significant testing by Canonical or the community at large.

My production Linux+ZFS server is still running a LVM mirror with ext4 on root because of exactly the fear you mentioned. All the FreeBSD systems have had mirrored ZFS root since v10.x (released 20 January 2014 according to Wikipedia). 7+ years later and I'm still not confident enough to run the same set up in production using Linux.

Do i smell Proxmox here? Fantastic system!!

Not here but I have used Proxmox before (v1 -- back when containerisation in Linux required a custom kernel) and it was pretty nice.

As always, it's about tools and integration.

ZFS as root is completely normal on FreeBSD, is it on Linux? You have tools like zfs-stats, bectl, zfs or dataset-jails, poudriere (packagebuilder) fully integrated with zfs, the list goes on and on. So even when the base of OpenZFS came from ZoL, it's use on freebsd feels much more integrated...or let's call it native.

Integrated with Kernel, vs not. Other tools in the OS that can take advantage of ZFS, vs not.

That's BS and you know it, it's OpenZFS and the code from zfsonlinux was brought in, why the hell should someone do work on a Filesystem but just for one OS?

Filesystem implementations are normally OS-specific because each kernel has different infrastructure for filesystems. e.g. Linux has the unified page cache, dentry cache etc, and ZFS just ignores all that and ships its own ARC and dnode caches, that then have to compete for shared system resources.

That's why SPL is such a nice feature, very little OS specific code. ZFS was ported in ~10 Days over to FreeBSD. And is (hopefully) in the near future THE open and "real" portable filesystem for Mac/Linux/BSD/Windows.

FreeBSD.org forums: "FreeBSD moving to ZFS-on-Linux"

https://forums.freebsd.org/threads/freebsd-moving-to-zfs-on-...

They took the zfsonlinux-code into OpenZFS, so OpenZFS is based on the zfsonLinux features and NOT the illumos (solaris) one:

https://openzfs.org/wiki/Main_Page

>The OpenZFS project brings together developers from the Linux, FreeBSD, illumos, MacOS, and Windows platforms. OpenZFS is supported by a wide range of companies.

I disagree. ZFS is great on Linux. The killer feature of FBSD is its simplicity and lack of incoherence in focus, which all the Linux distros suffer from. People don't see to be able to differentiate between these issues: Linux distributions' diversity of features and options is not a problem - it is the lack of focus & coherency of their management. This makes it harder to help other people on the same distro, to collaborate about bugs on the same distro, etc... Then you read an OBSD/FBSD thread from developers who aren't really better than developers on Linux threads - and you see much more coherency; because of the coherency in the systems they're working on.

Btrfs is used on millions of production servers (Facebook...) and can essentially solve the same storage problems that ZFS was designed to solve, using similar principles (CoW...). Btrfs deservedly got a bad wrap for many years because it had unacceptable bugs, but they seem to have finally quieted. SUSE ships and supports Btrfs for its enterprise Linux, and even Red Hat made it the default fs for Fedora. Also, Oracle can't sue you for using Btrfs, whereas Oracle has refused to have a lawyer send Linus a letter saying Oracle won't sue people for using ZFS on Linux, despite Linus' request to Oracle for such a letter saying its OK.

Comments like yours come up whenever anyone dares discuss ZFS or FreeBSD. Sure, you can use Btrfs, xfs, ext4 or whatever the hell you want on Linux if you want. Nobody is trying to take away your toys. But Linux is already dominant -- you don't need to convince people to use Linux. FreeBSD, however, doesn't get attention it deserves. Hence why threads like these occasionally pop up.

And for what it's worth, ZFS development pre-dates Btrfs. You can make all the arguments for Btrfs that you want but ZFS is still the vastly more mature filesystem out of the two. Hence why the few examples people can come up with regards to Btrfs usage are companies with the resources of Facebook (as opposed to ZFS which is used by companies large and small).

btrfs is used by orgs like facebook in a very limited way, and afaik not for big multi-disk storage pools with raid-style devices like ZFS is generally used.

btrfs is still very rough for things like raid5/6 (where you want to prioritise available capacity), and unfortunately there are lots of reports of recent corruption (even on single disk FSes) every time we have a thread about it on HN. I'm one of those people with recent experience of corruption with btrfs + compression.

ZFS isn't flawless, and the license situation means distribution is more awkward on Linux - except for Ubuntu, who have a different view to other distros on what is allowed with it legally. This means that root on ZFS is a real pain on many distros, and possibly not something you should really do, but ZFS has really solid multi-disk pool support & working + battletested support for things like RAIDZ.

But this isn't linux, this is FreeBSD. The license issues with CDDL & GPL don't apply here, so FreeBSD has native ZFS support. It works well (especially in 13, where it's up to date with the main OSS OpenZFS work), and the installer will quite happily assist you with root on ZFS.

I tend not to rely on what facebook does with its software stack. Even by looking at what they release as open source, it's usually "works for me" and not tailored for general public and I think it's in their culture not to care about other use than for themselves.

It's hard to think it's a good proof that btrfs is ready for everyone just because facebook is using it.

I'm surprised to learn that btrfs still has corruption issues, even for simple cases. It's been in development for 12 years already; I wonder what the factors behind this are?

I worked a bit with SLES (the enterprise suse), they use btrfs as root (snapshot feature), but the HARD recomendation from them is:

-Use btrfs JUST for the OS (just use mirror if you have to) nothing else..don't change any switch, don't defrag...don't touch it with the exception of making and restoring/deleting snapshots.

-Use XFS for Data

It's disappointing :(

I'd love to use btrfs on root on linux, and I try it on my home server and VMs reasonably often (and this is a xeon + ECC etc, and when I use ZFS/xfs/ext4 I don't get FS issues). I keep finding it's often still fragile unfortunately (and snapshotting is still painful compared to ZFS!).

Conversely, I haven't had a single btrfs issue on my home server, which has gradually expanded from a 2x2TB RAID1. New disks get added to the pool via 'device add' and off we go. That's one of the biggest reasons why I use btrfs, it lets me turn storage into a large generic pool instead of individual partitions and drives, and it's very flexible when combining drives of different sizes.

The only issue I did have was a bad ASM1061-based SATA controller, which would corrupt data in weird ways, throw up alignment errors, generic I/O errors, all kinds of issue. That can hardly be blamed on btrfs.

Because it was a RAID1 pool and a checksumming filesystem, I could switch the affected drive to a known good SATA controller and correct all of the corrupted data, to bring everything back in line. I found it very robust and undramatic. I strongly assume it would have worked more or less the same on ZFS, but ZFS is less flexible when adding disks (have to add new separate vdevs instead of integrating individual drives), so I stick with btrfs.

Don't buy ASM1061-based SATA controllers. At the very least, get something Marvell-based, or an LSI-based SAS controller in IT mode, if you want to get fancy.

Yeah, I notice some people have very different experiences. It's odd.

For me it's a mix of NVMe drives, SATA SSDs on Intel SATA, or SATA SSDs on LSI SAS HBAs (in IT mode).

It doesn't seem to be pinned to specific hardware, sadly - and the exact same hardware works fine with zfs/ext4/xfs.

It would be tempting to chalk it up to marginal storage controllers and/or memory or even flaky power supplies. I've seen all three result in very weird and hard to diagnose issues, which presumably a more critical and checksumming filesystem is simply better at sussing out than a more old-fashioned FS like ext4 or XFS.

But from the impression of the type of hardware you're running, I highly doubt that is the case. And of course ZFS would be affected, too.

> And of course ZFS would be affected, too.

In theory, yes. But one of the design goals of ZFS was to be resilient against faulty hardware. On paper that seems an impossible task but I've seen first hand just how impressive it is where I had a faulty raid controller which would randomly disconnect HDDs but only under heavy load. It took weeks to debug this -- it was so bad it actually caused Solaris to crash (which made fault diagnosis all the harder). I then switched to FreeBSD, which was a little more stable so enabled me to get a little more meaningful diagnostics and when I realised what the fault was, I was amazed I hadn't actually lost any data. At one point the fault was so bad it completely trashed the superblock but FreeBSD's ZFS tooling meant I could recover that superblock and the entire outage took all of 10 minutes.

On the flip-side, the few times I've used btrfs I've either been left scratching my head at the confusing tooling, or pulling my hair out because of data loss. On one system, the storage pool -- which literally just consisted of one HDD and no clever raiding -- reverted itself by 6 months. I was an early adopter of ZFS and never ever experienced so much as a blip despite running it on hardware that really should have caused ZFS to go tits up. Yet Btrfs would get it's knickers in a twist on stable hardware.

I get my stories here are purely anecdotal but I her the same kinds of stories crop up over and over again from other people too. So I find it very hard to take Btrfs seriously as a ZFS replacement. But then given ZFS is an older and more mature project and ZFS already works pretty damn well, I also question why anyone would want a replacement to ZFS when they could just use ZFS.

> But then given ZFS is an older and more mature project and ZFS already works pretty damn well, I also question why anyone would want a replacement to ZFS when they could just use ZFS.

Basically just licencing. The great irony is that btrfs was started by Oracle (not sure if they're still involved) before they acquired Sun. A relicensing effort to GPL would probably be better, but, you know, Oracle.

I argued in the past that OpenZFS should have contributors sign a small CLA which permits relicensing to GPL to prepare for this nonetheless, it's a small effort and could save a lot of effort later.

Your experience with a faulty controller mirrors the issues I experienced with the faulty ASM1061 controller; unstable and flaky under load.

Recovery was quite undramatic once the affected disks were put on a known stable controller. Btrfs-check followed by a scrub got everything back in order.

And btrfs gives me one thing that ZFS can't do (yet): a completely flexible RAID1 storage pool where I can add and remove disks as I want, the only limitation being that no one disk can be larger than half the total pool size. For me, this flexibility is a killer feature.

I agree - but in cases of marginal hardware I'd expect to see more strange issues than just specifically FS corruption with btrfs.

At one point it was limited to if I was using compression, but using compression was the major reason I wanted btrfs for that use rather than ext4/xfs.

I guess there's a reason redhat dropped to support btrfs.

I guess only facebook and suse are big users.

I suspect they wrote thousands of lines of code "Agile" style, without nearly enough upfront design. Now there is too much code to rewrite and it's too hard to change. I would be more sympathetic if it just lacked functionality, but instead the marketing claims all these features that are likely to eat your data. IMHO, it's an example of where open-source and the "bizarre model" failed to deliver.

Oracle cannot sue anyone for using ZFS, thanks to the patent protection clause in CDDL.