ZFS is an extremely complex filesystem, and it took Sun _years_ of internal testing first, and hundreds of angry customers later (sadly, at some point, the only way to improve a product is through real world testing), to reach a milestone where it was really production ready.
I know that on these days of dockers and unicorns, "production ready" has very different meaning than years ago, but still...
It depends of course, I would always choose FreeBSD over Linux when it comes to ZFS, but then again I would always choose a BSD-OS regardless of whether I want to use ZFS or not.
Take it certainly with a grain of salt, but the developers themselves announced ZoL to be production ready in 2013/03[1], Richard Yao (also a developer of ZoL) argued in a blog post in 2014/09[2] that ZoL was stable and production ready and elaborated on this. Some interesting comments also are here[3].
And all this also was quite a while ago, ZoL has been used by users for years now, it's come a long way, I have used it to some extent a while ago and had no bad experiences.
In the end everyone has to decide for themselves what "production ready" really means, and how high the threshold is to deserve that label. But I think that you can reasonably make the argument, that it is...
[1] https://groups.google.com/a/zfsonlinux.org/forum/m/?fromgrou...
[2] https://clusterhq.com/2014/09/11/state-zfs-on-linux/
[3] http://linux.slashdot.org/story/14/09/11/1421201/the-state-o...
I periodically have unexpected behavior where my filesystem doesnt mount right and I have to re-mount, or the nfs exports have to be re-exported, etc. I've assumed this was due to incomplete configuration on my part but it's clearly not as brainless to set up as native fs.
I know a zraid3 FreeBSD/ZFS root array will still boot with two drives failed and one drive acting wonky. Not so sure about Linux.
The huge practical difference right now is that on Linux, Docker and LXC (as in linuxcontainers.org) provide a high-level, fairly opinioned framework and set of infrastructure pieces to build, run and manage containers — and there are no fully equivalent tools on BSD, at least none that are as featureful or mature.
If you want to do containerized deploys on BSD, you'll be writing scripts that set up jails, call rctl, set kernel parameters with sysctl, and so on. There's ezjail [1], but it's nowhere near Docker in terms of scope.
I actually wish there was something on Linux that was less monolithic and better designed than Docker. Its image management is badly designed and annoying to use, and after all this time it still suffers from design flaws such as needing to be the parent of all containers (completely unnecessary given that we have cgroups).
[1] https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/j...
I'm working on a simple FreeBSD jail runner: https://github.com/myfreeweb/sandblast The main idea is machine-friendly configuration: your higher level software (PaaS, CI, etc.) produces a JSON config and pipes it to sandblast, which runs a script in the container.
in jail.conf i just have
stud {
path = "/usr/local/jails/jail_name";
ip4 = "inherit";
host.hostname = "jail_name.example.net";
securelevel = 4;
devfs_ruleset = 10;
exec.start = "/bin/service --config=/etc/service.conf";
mount.devfs;
exec.system_jail_user;
exec.jail_user=jail_name;
exec.poststart="/usr/local/bin/jail_ipfw";
}
The only non-obvious thing is you need /libexec/ld-elf.so.1 in order to run dynamically linked libraries.
There's also the PC-BSD Warden. With it, one generally does not write scripts to "set up jails, call rctl, set kernel parameters with sysctl".
See the PC-BSD 10.1 Handbook chapter 8:
* http://download.pcbsd.org/iso/10.1-RELEASE/amd64/docs/html/c...
I use FreeBSD and (obviously) prefer it, but:
1) ZFSonLinux has made huge steps forward and actually (at least in my opinion) is production ready and rock solid (of course it's not as good as ZFS on FreeBSD though, for example it's lacking some features and of course it's not as proven as of now)
2) Usable containers finally have come to Linux in the form of Docker and everybody is absolutely hyped about it (I would argue though that Jails has more features, is more stable and mature)
The most obvious reason for me to use FreeBSD though is just the difference in philosophy. Don't rapidly make big changes, think everything through, don't surprise users, don't make upgrades that call for massive user intervention, and it's democratically governed and yes, it's a complete OS. Code quality is also better (have you actually looked at what code makes it into the Linux Kernel?), and I would argue therefore security and stability is way better.
Just my 2 cents, nothing wrong with Linux, have used it for years, now I would not want to go back though.
And a small Side Note: No systemd, not to start a flame war, but it is taking over the linux-sphere and I find it's design just horrific