News for hackers. Stuff that matters.
Re:ZFS for Dummies by dontupvoteme on Tuesday September 05, 2023 @12:05AM (#37387716)
As nice as the technology is as long as there's the potential of a Damoclean license issue I'll always feel hesitant around ZFS.
Re:ZFS for Dummies by wkat4242 on Tuesday September 05, 2023 @12:36AM (#37387925)
It's not really an issue with ZFS itself. It's a great filesystem. GPL isn't the only way to do open source. It's the way Linux has chosen and that has pros and cons. Not integrating so well with other open source licenses is one of the cons.
Re:ZFS for Dummies by tylercrompton on Tuesday September 05, 2023 @12:55AM (#37388027)
> Not integrating so well with other open source licenses is one of the cons.

That depends on what you want. If you want a license that will play well with closed source software, then yeah, it's a downside. But the GPL family comes from the perspective of a developer who wants to retain their rights while respecting others' desire for the same. If you care about your rights, then this is an upside.

Re:ZFS for Dummies by kaba0 on Tuesday September 05, 2023 @02:15AM (#37388437)
CDDL (ZFS’s license) is also like that, the problem is exactly the two “strong” copyleft licenses.
Re:ZFS for Dummies by jakobson14 on Tuesday September 05, 2023 @02:58PM (#37396155)
https://www.gnu.org/licenses/license-list.en.html#CDDL

>"It has a weak per-file copyleft (like version 1 of the Mozilla Public License)"

The CDDL isn't a strong copyleft licence. It doesn't give a shit what larger project you include the code into, and it doesn't give a shit how you licence the resulting binary. It's considerably more permissive than the GPL.

The conflict is entirely on the GPL side.

Re:ZFS for Dummies by judge2020 on Tuesday September 05, 2023 @12:22AM (#37387831)
Even in such an event, I don’t see how it would affect end users in an extreme way. OpenZFS doesn’t call home to make sure it’s still legal to use, so worst that could happen is that they are forced to stop distribution and you need to stay on old software until you can move to another storage system.
Re:ZFS for Dummies by NavinF on Tuesday September 05, 2023 @12:37AM (#37387932)
Not to mention Ubuntu has included ZFS by default since 2016. I still remember all the doomsday predictions people made back then. People on HN and proggit kinda suck at legal
Re:ZFS for Dummies by arjvik on Tuesday September 05, 2023 @01:44AM (#37388254)
Playing devil's advocate here, but what if Oracle decides to sue you, a company with only a few thousands in revenue, for their multi-million dollar licensing fee?

I agree this is unlikely, but so is someone being born with as much litigiousness as Larry Ellison.

Re:ZFS for Dummies by kaba0 on Tuesday September 05, 2023 @02:13AM (#37388421)
On what grounds? ZFS has an open-source license. The only semi-open question related to the topic is packaging it together with another, different open-source license that is viral and mandates more things (GPL). That part is not being “violated” by an enduser, but the distro maintainer.
Re:ZFS for Dummies by jakobson14 on Wednesday September 06, 2023 @12:41AM (#37401412)
Oracle's hands are tied in so far as they can't revoke the CDDL licence that Sun provided.

They could sue you as a kernel contributor, but only if you do something to violate the GPL. The only thing that would clearly violate the GPL, in the era of proprietary nvidia blobs, is if you tried to ship the ZFS source code in-tree.

Everybody agrees that's a bad idea, nobody does it. ZFS instead is shipped as a separate kernel module.

Re:ZFS for Dummies by TheDong on Tuesday September 05, 2023 @02:10AM (#37388401)
This is built on a misunderstanding of what the ZFS licensing issue is.

ZFS is licensed under the CDDL [0], which is a copyleft license that grants you full rights to use the ZFS source code for basically anything, providing that any derivative works incorporating its source code are also are under the CDDL. There is no "licensing fee" that you can pay Oracle related to this.

Linux is licensed under the GPL, which just like the CDDL, is a copyleft license which requires derivative works to be distributed under the GPL.

The conflict here is that both require the entire derived work to be distributed under CDDL/GPL, but obviously only one license can be picked.... even though realistically, GPL and CDDL are quite similar licenses.

The additional fact that is useful to know is that zfs.ko, the zfs kernel module, is distributed under the CDDL by ubuntu, and any other distro distributing it that I know of.

So, with that knowledge, who can sue who and why?

1. The people that can be sued are people distributing compiled zfs binaries (the 'zfs.ko' kernel binary). In practice, this means canonical, and anyone else who distributes disk images using ZFS (such as snapshotted EC2 AMIs)

2. The people who can sue are the holders of the Linux GPL license, and they can do so by claiming zfs.ko violates their copyright by being a derived work of the linux kernel, but being distributed under the CDDL rather than the GPL.

So, first of all, the vast majority of companies using ZFS aren't also distributing binary distributions containing zfs.ko and the linux kernel together, therefore this whole thing is moot. Realistically, most companies using ZFS only have the risk of _canonical_ being sued, and that breaking their filesystem, not them running into legal issues themselves.

And second, it would require a holder of Linux copyright to file suit, and realistically, no court could find significant damages here.

Canonical also has lawyers that claim 2 wouldn't legally follow, that 'zfs.ko' is not in fact a derived work of the linux kernel. There's differing opinions there, but pretty much everyone agrees that the people who could sue if this interpretation is wrong is the holders of Linux copyright, not oracle.

[0]: https://en.wikipedia.org/wiki/Common_Development_and_Distrib...

Re:ZFS for Dummies by arjvik on Tuesday September 05, 2023 @02:19AM (#37388470)
I see, thanks for educating me on this with such an in-depth reply! I simply saw "licensing issue" and "Oracle" together and assumed the standard Oracle licensing worry applied.
Re:ZFS for Dummies by jclulow on Tuesday September 05, 2023 @04:59AM (#37389428)
Note that unlike the GPL, the CDDL is file-based. It doesn't require your entire body of software to be CDDL, it just requires that you make modifications to CDDL-licenced files available under the CDDL.
Re:ZFS for Dummies by jakobson14 on Tuesday September 05, 2023 @03:01PM (#37396210)
You have one crucial fact wrong: The CDDL is actually a weak per-file copyleft.

The CDDL is totally ok with the rest of the linux kernel staying GPLv2. The GPL is allergic to any non-GPL (eg. ZFS) code being distributed with the kernel source.

Re:ZFS for Dummies by TheDong on Tuesday September 05, 2023 @06:29PM (#37398681)
Sure, but I don't think that's crucial here.

Does it change any of the other information? I admit I shouldn't have written "both require the entire derived work to be distributed under CDDL/GPL", and instead should have written "both may require, depending on a court's interpretation, the zfs kernel module to be distributed under CDDL or GPL respectively".

I think the rest of the comment stands though, right?

Re:ZFS for Dummies by jakobson14 on Wednesday September 06, 2023 @12:26AM (#37401334)
> "both may require, depending on a court's interpretation, the zfs kernel module to be distributed under CDDL or GPL respectively".

That's wrong, and misses fundamental information about the CDDL.

First, the CDDL gives absolutely no fucks how you licence a resulting binary. It only cares about the specific covered source files, and doesn't care what you bundle them with. So nobody would ever argue that the module should be licenced as CDDL.

On the flip side, you'd have a really hard time arguing that the module should be licensed as GPL.* Sure, there's maybe a GPL wrapper, but lots of modules are non-GPL even when they might feature such a wrapper. ie. nvidia. You also can't argue ZFS is a derived work of the kernel, which is really the crux of the issue. ZFS was first written on solaris, developed for several years on freeBSD, and remains neutral across 3-5 different supported OSs. There's nothing to indicate the GPL should have more claim over ZFS than the wrapper.

The one and only place you'll ever run into trouble is if you start trying to distribute ZFS *source code* in the kernel source tree, as a fully integrated part of the kernel. (EDIT: or compiled together in a single kernel binary, from said combined source) The CDDL doesn't care about the mixing but the GPL does. Literally everyone agrees this is a bad idea and nobody does it.

*Let's ignore for a moment for the moment that you *couldn't* since the GPL requires complete corresponding source code and you can't relicence the CDDL source code files.

Re:ZFS for Dummies by TheDong on Saturday September 09, 2023 @07:35AM (#37444211)
I agree that statement is factually incorrect, but I still don't think it changes the overall information in my comment in any meaningful way. The actual risk is low, the risk if of GPL infringement, not of oracle suing you, and there are people who contest it on either side. Those are the key points, and those are true I believe.

> On the flip side, you'd have a really hard time arguing that the module should be licensed as GPL

That is the very thing that some people do argue, for example the sfconservancy people. As I note, canonical's lawyers take your view (and I personally agree), but it's not clearcut. The sfconservancy outline their reasoning clearly for why they think zfs.ko is a derived work of the linux kernel.

https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/

Re:ZFS for Dummies by jakobson14 on Saturday September 09, 2023 @10:37PM (#37452389)
The SFC can get stuffed. Their argument says nothing in particular about ZFS and is essentially "the kernel is GPL and therefore ALL modules MUST BE GPL.

Their stance isn't just rejected by nvidia and company, but also by the kernel devs most notably torvalds himself.

Torvalds himself has personally refuted the "linking inescapably means you're derived" point on multiple occasions, and yet here they are continuing to wave it around because that's how they wish it worked.

Re:ZFS for Dummies by ggm on Tuesday September 05, 2023 @12:48AM (#37387994)
This is only going to get a shedload of "you do you" responses. BSD licencing is quite content to use ZFS, its pretty much ubiquitous now in widescale deployment and the source would be impossible to re-lock. Worst case is fork.

I very much regret the fragmentation of FS design, it has many mothers. "there can only be one" was never going to work, but we seem to have perhaps 4-5 more than we really need. ZFS manages to wrap up a number of behaviours cohesively with good version-dependent signalling so it should always be possible to know you're risking a non-reversible change to your flags. And, it keeps improving.

But, counter "it keeps improving" so do all the other current, maintained, developed FS and if somebody tells me they prefer to use Hammer, or one of the Linux FS models with a discrete volume-management and encryption layer, I don't think thats necessarily wrong.

Mainly I regret Apple walking away. That was about Oracle behaviour. It wasn't helpful. A lot of Apple's FS design ideas persist. I never got resource/data forks, it only ever appeared on my radar as .files in the UNIX AUFS backend model of them. Obviously inside Apples code, it was dealt with somehow. It felt like the wrong lessons about meta data had been learned. Maybe an Ex-VMS person went to Apple? Also Apple has a rather "yea maybe or no, dunno" view about case-independent or case-dependent naming. Time machine is good. Feels like it should fit ZFS well. Oh well.

Re:ZFS for Dummies by chaxor on Tuesday September 05, 2023 @01:25AM (#37388181)
I wish zfs had a 'tag' type of feature like Apple style fs. I have thought about making an object store to get some of the metadata on the fs level, rather than having to make some cobbles together solution of an SQLite DB of metadata with the files or something, but many of the object store solutions seem to be for fairly large files, rather than a few thousand pictures of family and such. Unsure what a good solution using zfs would be.
Re:ZFS for Dummies by adrian_b on Tuesday September 05, 2023 @01:38AM (#37388227)
I am not sure what kind of "tag" you mean, but in any modern file system which supports extended file attributes you can attach arbitrary metadata to any file.

You just have to define whatever kind of tags you want and choose names for the xattrs in which to store them and then you should use them in your scripts or applications.

I am using all the time xattrs for things like file checksums and content classification.

Re:ZFS for Dummies by vetinari on Tuesday September 05, 2023 @04:05AM (#37389098)
> except Linux tmpfs (which has only a partial support that is useless for those who use custom xattrs, so a file with xattrs cannot be copied to /tmp if tmpfs is used).

tmpfs is getting user xattrs in kernel 6.6, together with quotas.

> You just have to define whatever kind of tags you want and choose names for the xattrs in which to store them and then you should use them in your scripts or applications.

It is still just a oob store, you have to traverse the directory hierarchy old-style; the old BeOS did (and Haiku today does) something more: it indexed the selected xattrs and allowed live queries against those indices. This allowed for very interesting uses.

Re:ZFS for Dummies by adrian_b on Tuesday September 05, 2023 @06:13AM (#37389887)
> tmpfs is getting user xattrs in kernel 6.6

I am aware of this, but the so-called user xattrs support most likely remains too bad to be of any use.

The kernel list message:

"Add support for user xattrs. While tmpfs already supports security xattrs (security.*) and POSIX ACLs for a long time it lacked support for user xattrs (user.*). With this pull request tmpfs will be able to support a limited number of user xattrs."

I am waiting to see which is the meaning of "a limited number", but based on the history of tmpfs, I suspect that this does not mean that the number of extended file attributes is limited to some value, but it means that there is a small list of names of user xattrs that will preserved (which are those used by some application that the tmpfs devs love, while they are hating the other applications), while the other xattrs will be lost, like now.

If "a limited number" is really a number, that would be much better, but this might still mean that the number is excessively low, e.g. ten attributes or less, instead of being high enough to not be a serious limitation, e.g. one hundred or one thousand.

> It is still just a oob store, you have to traverse the directory hierarchy old-style; the old BeOS did (and Haiku today does) something more: it indexed the selected xattrs and allowed live queries against those indices.

You are right and it would be nice if the file system itself would allow faster queries by xattrs than what can be achieved by testing the attributes of files selected by their names.

However, it is not difficult to create a database file indexing the files by attributes, but it may need frequent updates, to remain in sync with the indexed file system. I use such databases, but mainly for file system parts where I keep files for long-term storage, like books, research papers, handbooks, product documentation or older software projects.

Re:ZFS for Dummies by DHowett on Tuesday September 05, 2023 @10:21AM (#37392033)
Looking over the landed patch (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...), I cannot find a hardcoded list of user.* xattrs that will be accepted.

It looks like it will store as many xattrs as there is space for, based on their contents.

Re:ZFS for Dummies by adrian_b on Wednesday September 06, 2023 @01:57AM (#37401754)
That would be nice, but then the "a limited number" expression does not make much sense.

All the file systems that support extended file attributes have some limit for the size of the attributes attached to a file, for instance of 64 kB.

After searching through the patch, it seems that the maximum number of xattrs is 128. There is also a size limit of 128 kB, which might be the limit for the sum of the sizes of all attributes.

If that is the only reason for saying "a limited number", then this is very good, the limits are high enough to not cause any problems in most applications, so tmpfs will finally no longer strip the attributes of the files copied through it.

Re:ZFS for Dummies by rodgerd on Tuesday September 05, 2023 @01:29AM (#37388191)
Do you refuse to use NVidia cards for the same reason?
Re:ZFS for Dummies by dontupvoteme on Tuesday September 05, 2023 @01:33AM (#37388210)
[delayed]
Re:ZFS for Dummies by NavinF on Tuesday September 05, 2023 @03:45AM (#37388993)
New drivers are MIT/GPL2 dual-licensed: https://github.com/NVIDIA/open-gpu-kernel-modules
Re:ZFS for Dummies by mrweasel on Tuesday September 05, 2023 @03:50AM (#37389024)
Doesn't that still require the closed source user space driver to function? I'd love to be wrong on that one.
Re:ZFS for Dummies by vetinari on Tuesday September 05, 2023 @04:50AM (#37389358)
In the short-term yes, in the long term the user space driver would also get open (or replaced with nouveau, since it will be on equal footing with the closed one, when communicating with kernel; just like the open vs closed amd drivers).

However, what will be left is a blob, that runs on the dedicated core on the GPU. The stuff that Nvidia doesn't want to open is moved there, and that's why the open kernel driver works only on Turing and newer.

Re:ZFS for Dummies by mrweasel on Tuesday September 05, 2023 @04:59AM (#37389434)
> However, what will be left is a blob, that runs on the dedicated core on the GPU

But that basically firmware, right? So not all that different from the firmware loaded by a NIC?

Re:ZFS for Dummies by vetinari on Tuesday September 05, 2023 @05:05AM (#37389469)
Yes, exactly the same thing.
Re:ZFS for Dummies by yjftsjthsd-h on Tuesday September 05, 2023 @01:53AM (#37388305)
Honestly the nvidia thing was always annoying because it seems like the linux devs are more aggressive about licensing with an actual full-on Open Source project than the very much closed-source nvidia drivers. They even both use the same approach of having a GPL wrapper to talk to the non-GPL code AFAIK:\
Re:ZFS for Dummies by mrweasel on Tuesday September 05, 2023 @03:46AM (#37389002)
That might be a bad example. I'd trust ZFS and the CDDL before NVIDIA any day. At least the CDDL is designed with the intention of being open attemps to work with other licenses. NVIDIA seems hellbent on working around the GPL.
Re:ZFS for Dummies by teddyh on Tuesday September 05, 2023 @04:39AM (#37389300)
The original intention of the license doesn’t really matter when its current copyright holder is Oracle.
Re:ZFS for Dummies by jakobson14 on Tuesday September 05, 2023 @03:03PM (#37396239)
Oracle is *a* copyright holder, but they have no control over the direction of the OpenZFS fork.
Re:ZFS for Dummies by teddyh on Tuesday September 05, 2023 @03:35PM (#37396649)
I’m not concerned about the direction of development, I’m concerned about Oracle holding the (or at least a significant part of) copyright to some software which is used as a Linux module, but which is licensed under a license which is (arguably) incompatible with the GPL.

If Oracle wanted to, they could easily release their copyright under a dual license or something else to clear up the licensing issue. They have not, and have chosen to retain the ability to sue every Linux user using ZFS. This concerns me.

Re:ZFS for Dummies by NavinF on Tuesday September 05, 2023 @11:15PM (#37400955)
Re:ZFS for Dummies by teddyh on Wednesday September 06, 2023 @04:14AM (#37402525)
It doesn’t really matter on whose “side” the conflict lies; the license is arguably GPL-incompatible, and Linux is not going to change its license. But Oracle could, if they wanted to. And they evidently don’t want to, which is concerning.
Re:ZFS for Dummies by jakobson14 on Thursday September 07, 2023 @12:11PM (#37421145)
It's plainly obvious why they don't. Oracle still sells *their* fork of ZFS.

Back in the pre-oracle days Sun's fishworks division sold ZFS-based storage appliances. Oracle still sells those, and thanks to ZFS' reputation they're still profitable. Oracle simply took that much older fork of ZFS (with no OpenZFS code in it) and made it closed-source.

Before anyone asks, no, that fork is too old and too diverged to be compatible with OpenZFS.

Re:ZFS for Dummies by teddyh on Saturday September 09, 2023 @10:30AM (#37445870)
All the more reason for Oracle to someday do something to put all those Linux ZFS servers into a questionable legal status. They could then stand with a carrot in their other hand, waving their own unquestionably legal version of “ZFS”.
Re:ZFS for Dummies by jakobson14 on Saturday September 09, 2023 @10:35PM (#37452382)
I don't think you understand how this works. Sun already gave all their power away via the CDDL.

We're currently in a legally-stable state, and have been for over a decade now (nearly two). If they could have done something, they would have.

The fact is that so long as nobody tries to merge ZFS code into the mainline kernel, there's nothing they can do. Everybody knows this, everybody agrees that would be a dumb idea, and everybody has agreed not to do it.

Re:ZFS for Dummies by jakobson14 on Thursday September 07, 2023 @12:13PM (#37421187)
Modules don't have to be under licences compatible with the GPL. There is no universe where you could possibly argue ZFS is a derived work of the kernel. None.

Oracle can't actually sue people using ZFS modules. The CDDL licence that sun provided already gave all the relevant rights away. They can only sue from the GPL side as a kernel contributor, and that's ONLY if someone is dumb enough to ship ZFS code in the linux source tree.

You *seriously* underestimate the number of people using ZoL in production that Oracle would love to sue *if they could.*

Re:ZFS for Dummies by jakobson14 on Wednesday September 06, 2023 @12:39AM (#37401395)
The sad part is the kernel devs can't tell the difference. Doesn't matter if something is open source or proprietary, if they can't take it and relicence it under their GPLv2 licence then it deserves no comfort.
Re:ZFS for Dummies by lmm on Tuesday September 05, 2023 @01:30AM (#37388192)
You might want to consider using FreeBSD then, where there is no licensing issue. I've found it has all the stuff I liked about Linux, and less of the stuff I dislike.
Re:ZFS for Dummies by seized on Tuesday September 05, 2023 @02:12AM (#37388420)
There is also Illumos/OpenIndiana/OmniOS, basically continuations of OpenSolaris. I have used OpenIndiana for many years and it's been dead reliable and stable.

There's quite a few "quality of life" differences like boot environments (boot into a pre upgraded OS state, even years old), built in SMB server with NFS v4 style ACLs, dtrace, built in snapshot scheduling and management, and Napp-It is an available web UI for management a la FreeNAS/TrueNAS.

It has a few differences, service management is quite different from other things, but overall very underrated as an OS I think.

Re:ZFS for Dummies by unethical_ban on Tuesday September 05, 2023 @11:14AM (#37392778)
I haven't looked at those distributions in years. I thought they were niche, container host operating systems and not general purpose.. I guess it need to do my research.
Re:ZFS for Dummies by NavinF on Tuesday September 05, 2023 @11:12PM (#37400947)
That's because they are niche container host operating systems. Without support for modern hardware like GPUs, it makes no sense to run them on bare metal
Re:ZFS for Dummies by seized on Friday September 15, 2023 @12:21AM (#37518387)
Sure it does, they are great storage OSs. A NAS doesnt need GPUs.
Re:ZFS for Dummies by kaba0 on Tuesday September 05, 2023 @02:08AM (#37388396)
The license issue in the very worst case (but even that is quite questionable and I would say that Linus is a bit paranoid here) could only be a problem for a distro that ships it by default, you as the end user can freely use both linux and ZFS, as well as their combination.
Re:ZFS for Dummies by tomxor on Tuesday September 05, 2023 @02:49AM (#37388655)
I was concerned about this aspect initially, but ZoL has been going for many years now. At this point even Debian (the most purist in terms of GPL considerations) includes ZoL in it's official repository. It doesn't distribute binaries, apt just builds the kernel module automatically.

Frankly, if even Debian can use it, it's a non-issue.

Re:ZFS for Dummies by unethical_ban on Tuesday September 05, 2023 @11:16AM (#37392804)
ZFS on Linux, and even Ubuntu distributing the binary form in their base OS, has been going on for years. I think you would be a huge event and very unlikely to occur if Oracle tried to pull some licensing bullshit.

As someone ignorant to file system development, I would almost expect something more likely to be BTRFS getting sued for copying a feature of ZFS or something like that.

Re:ZFS for Dummies by jakobson14 on Tuesday September 05, 2023 @03:07PM (#37396300)
There's really no case to be made from the CDDL side. It's a weak per-file copyleft.

If anyone (Oracle or Linus Torvalds) launches a ZFS-related lawsuit, it'll be as an author of GPLv2 kernel code. For the time being the solution has been to ship ZFS separate from the kernel, as any module with a non-GPL licence typically does.

The biggest hurdle to ZFS isn't legal, but technical. The various teams working on ZFS has so far been able to keep up with kernel churn and symbols (eg FPU ones) being made GPL-only.

That said, the kernel devs have made it clear that they don't care if you're open source or proprietary, they will make changes and mark new symbols GPL-only to fuck with you regardless.