FreeBSD has had fully integrated, working ZFS for over 10 years. You were so excited to deploy the BSD killer app, you forgot to do so for ten years.

I can only tell you, whatever scary differences you expect between Linux and FreeBSD are probably no more than between any two Linux distros with different packaging systems.

Ten years ago, fresh out of a failed stint at university, I applied for a junior position at a Linux shop. Would nowadays probably be called junior system engineer or so. The night before the interview, I read around a bit in Stevens' TCP/IP Illustrated as well as Design and Implementation of BSD (to calm my nerves). I told them honestly, I had maybe 15 lifetime minutes on a Linux shell. But I started with FreeBSD4.4 and had by then already 8'ish years of experience on general *nix administration.

I'm still there. Pivoted around and upward a couple of times internally. But I still run FreeBSD on my workstation to get things done. And we're still fundamentally a Linux shop.

But the root cause of my career is a friend at university handing me a FreeBSD 4.4 CD. It is a tremendous system if you want to learn about the services a kernel offers to its userland. If you care to make the dive, it not only tells you the what and how, but the why and all the compromises that had to be made along the way. And that understanding is universal.

FreeBSD may be well known as a solid production platform. It's true strength is as the foundation for not only a lifetime of learning, but a lifetime of understanding.

Good for you! FreeBSD taught you Unix which applies perfectly well at least at that time, to Linux as well. That was a good bet! Somewhat the same as learning Slackware at the time would teach you Unix, while chosing another distro would be more limited.

I have submitted multiple bugs for ZoL, from prohibiting boot of zfs as root (for the sake of boot beeing faster (?) they are not doing zpool import -a(f) but rather rely on cache file which is... crazy). Also the .zfs/snapshot still doesnt work. Both are vital issues and still not beeing fixed.

Boot on Linux laptop now looks like:

- power on

- wait for recovery shell

- `zpool import -a`

- `exit`

In all my years I have never observed this on FreeBSD and... as I have said, I expect from FreeBSD to be rock solid and it never dissapointed.

ZoL? I wouldn't be running it on my linux servers for a while. The basic mistakes that they are doing are not something that apriciate from file system that is meant to be as reliable as possible.

The actual problem you are having is that there isn't a setup option on virtually any distro that will set this up for you but you are clearly setup incorrectly. I was using Funtoo with a zfs root for years until the machines CPU died.

The process of booting up looked like hit power button wait for desktop to show up. The "basic mistakes they are doing" are completely a function of your distro not having a way to easily set this up and you not knowing how to use it correctly. As far as I know the only distro that supports zfs on root via its installer is at

https://project-trident.org/

You might try it in a virtual machine.

Ironically, Project Trident was the desktop part of TrueOS Desktop, which in turn used to be PC-BSD, a derivative of FreeBSD.

Stop spreading FUD. ".zfs/snapshot" has been working on Linux for years now. I don't know when you last used ZFS on Linux, but just because you haven't used ZoL for a while doesn't mean that the developers stopeed working on it.

Also, me and hundreds of other people have been booting off of ZFS on Linux for years. Your issue is most likely in your distro's initramfs setup or you must have botched the initramfs config yourself. If you're just used to FreeBSD, just stick to commenting about FreeBSD without maligning other operating systems.

Again, if it doesn't work, then it doesn't work; to users it doesn't matter _why_ - whether it's a problem with ZoL code itself, or with distro-provided initramfs. It's the end result that counts.

Well if it's working for everyone but a select few, then I'd argue the problem is with those select few and not the product.

> they are not doing zpool import -a(f)

That's an initramfs bug; the initramfs scripts are not actually supplied by the ZoL project. You should file a bug report against your distro's initramfs package.

That's part of the problem. From the user point of view it doesn't matter if the bug is due to initramfs, or the kernel, or ZoL, or whatever else. What matters is it doesn't work. With FreeBSD development model it's easier to get it to work, since it doesn't need to go through so many separate groups of people.

I am on fedora with added zol repository and fedora 30 doesn't supply any zfs modules or initramfs scripts. I had to do my scripts to verify that the modules are included and don't break my boot process. And it worked like a charm... for a while.

So... the bug is in your own scripts? I'm now thoroughly confused what your complaint is :)

> .zfs/snapshot still doesnt work

I've been using this as a server backup solution for over a year, I assure you zol snapshots work, including access via the .zfs directory.

I have ZFS images that I can't send from my freebsd server to my 16.04 server with the ZoL repo enabled. Given that it's my personal data I can't really submit until I narrow down to a minimal reproducing case, but yeah, there are bugs.

> Also the .zfs/snapshot still doesnt work.

What do you mean by this? I’m using ZoL and snapshots seem to be working fine.

zfs has a feature where you can `cd` to hidden .zfs folder (you wont see it by `ls`, imagine it like /procfs but hidden) on root of zfs filesystem (if you have tank/whatever mounted to /whatever this would be /whatever/.zfs) with some interesting content: within .zfs/snapshot you will get a list of snapshots for that filesystem with materialized state (!!!) of fs when the snapshot was taken. Imagine (instead of rollback the snapshot), just copying the original file(s) from there, with content from the time when snapshot was created.

This is imho, the killer feature of zfs, maybe matching only by VMS or DragonflyBSD hammer filesystems. They were/are both better at it (`undo`!) but zfs comes closely.

And it is broken for months now.

> And it is broken for months now.

This guy speaks without knowing.

I just tried going in my .zfs/snapshot on my CentOS 7 system running kernel 3.10 and ZoL 0.8.3 and it's working. I can list snapshots and cat file off a snapshot.

> CentOS 7 system running kernel 3.10

what the actual f*?

  tag name v3.10 (bc1b510b8979ecc322f8d930dde56658967b7355)
  tag date 2013-06-30 15:13:42 -0700

Even with stable fixes and CentOS distro sugar on top, running 3.10.x in 2020 feels batshit crazy. I always thought such reptilian kernels are only found on shitty embedded devices...

RHEL kernels are like that. They backport a ton of stuff and the major version number is of little relevance, but it doesn't change because they maintain a stable kABI.

By now the CentOS 7 kernel is probably closer to mainline 4.x and 5.x than the original 3.10

Wait until you find 2.6 (not many changes before 3.2 allegedly) and 2.4 kernels in embedded systems.

I'm well aware of 2.6 in embedded systems (and have to work with such systems T_T), I just didn't expect to see 3.10 in a system that also runs ZFS...

After 10+ years of using zfs, I just "might" know something... ;) While your 1 year old versions of ZoL just "might" not prove anything.

if you had actually been using it during the last 1.5/2 years you would know it work.

otherwise you're just talking about stuff (zfs on gnu/linux) you know little about.

if you use zfs on freebsd then good for you, but that doesn't mean you can spread disinformation about zfs-on-linux.

btw: freebsd dropped its own implementation of zfs and is rebasing onto zfs-on-linux (from 2018: https://lists.freebsd.org/pipermail/freebsd-fs/2018-December...). that's how good zfs-on-linux is.

Just update your outdated ZoL to latest version. Than we can talk. And you might just agree.

I run ZFS on both Linux and FreeBSD. On Linux I compile from the latest releases. I’ve honestly not noticed any issues either.

Are you absolutely sure this isn’t something you’ve misconfigured yourself?

For what it’s worth, I do prefer FreeBSD over Linux but not not for any reasons relating to ZFS.

  $ cat /saspool/debian/master-buster/.zfs/snapshot/ci/etc/issue     
  Debian GNU/Linux 10 \n \l

ZFS 0.8.4 on Linux 5.4.43 — just as a data point in case you're trying to track this down. Both built from source, no distro patches.

(Yes my "tank" is named "pool"; /saspool/debian/master-buster is a template for my containers, "ci" is one of the containers.)

Works for me on Debian Unstable...

   sudo zpool --version
   zfs-0.8.4-1
   zfs-kmod-0.8.4-1

   uname -a
   Linux xxx 5.6.0-2-amd64 #1 SMP Debian 5.6.14-1 (2020-05-23) x86_64 GNU/Linux

I haven't noticed that issue on Proxmox (been using zfs with it for approx. 6 months). Currently on zfs 0.8.4 (and I've done that previously).

Are you using a distro supplied zfs (potentially old) or rolling something more bleeding edge?

Just for any case, I would really love to solve this, I accept the fact that it is connected with upgrading the modules but this just is no reason for breaking backward compatibility.

root> zpool --version

zfs-0.8.4-1

zfs-kmod-0.8.4-1

root> ll /.zfs/snapshot

total 0

drwxrwxrwx 2 root root 2 jun 15 08:48 ./

drwxrwxrwx 1 root root 0 jun 10 07:53 ../

root> cat /etc/yum.repos.d/zfs.repo

[zfs]

name=ZFS on Linux for Fedora $releasever

baseurl=http://download.zfsonlinux.org/fedora/$releasever /$basearch/

enabled=1

metadata_expire=7d

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux

[zfs-source]

name=ZFS on Linux for Fedora $releasever - Source baseurl=http://download.zfsonlinux.org/fedora/$releasever/SRPMS/

enabled=0

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux

But nevermind, this is something that was working since zfs as root was brought into FreeBSD (and we are talking about FreeBSD not ZoL), never had issues about it.

FWIW, I think you have a good point here. I really like that FreeBSD has a focus on quality, that the base is one whole coherent system and that the documentation is very good. But as you point out, it does seem like they have a hard time keeping up with the world outside.

I see it as a huge issue that the whole world has point on marketing which is horrible wrong. The last two projects I was working on, both are doing mission critical task (for obvious reasons I can't name them or what they are doing):

one is really piece of s*, full of beginner mistakes, lasigna/spaghetti/ravioli mixed in horrible way, done in java, wasting resources like olympic games. But with polished webui, full of colors, nice icons etc.

Second one, c++, with kernel module for linux, driver for windows, really nice and clean code (I was surprised). Not really nice webui but it did do the job well. Passed the benchmarks with flying colors, almost once faster than the first project.

At the end: the first one was a bestseller. There is no technical reason to be better selling, they both were doing the same task, from technical perspective, the second one was far better. Benchmarks, way of doing its work. Just far better. But the worse product with better "colors" won.

Marketing, marketing, marketing. The pestilence of our time and probably the Dark Ages of technology. I doubt I will see the Renaissance.

Yes that works for 'end user software' but in the long run NOT for Operating-systems/Database/Filesystem/Network/Hardware...they need to be rock-solid, everything on top is your problem.

Both of those are doing quite complex tasks for sysadmins (probably it should be guessable what they are doing) that NEED to work. There is no excuse for them to fail. They need to be rock solid, "Operating-systems/Database/Filesystem/Network/Hardware" can fail, those should not (words like "criminal liability" comes to mind).

Thing is, FreeBSD is quite good at outliving various fads. Or adapting them, if they turn out to live long enough.

And in some cases predicting them (eg: jails)

> I'm not questioning whether FreeBSD is hip or trendy.

> A trend where the OS is abstracted away as much as possible.

It seems like you're asking about following trends.