News for hackers. Stuff that matters.
Re:An Introduction to ZFS by matheusmoreira on Friday November 20, 2020 @07:58AM (#25159992)
> Essentially, ZFS manages my files for me.

Yes! Managing our files is the whole point of file systems! It's amazing how bad at it most of them are. Linux is still catching up with btrfs...

It's extremely aggravating how most file systems can't create a pool of storage out of many drives. We end up having to manually keep track of which drives have which sets of files, something that the file system itself should be doing. Expanding storage capacity quickly results in a mess of many drives with many file systems...

Unlike traditional RAID and ZFS, btrfs allows adding arbitrary drives of any make, model and capacity to the pool and it's awesome... But there's still no proper RAID 5/6 style parity support.

Re:An Introduction to ZFS by bityard on Friday November 20, 2020 @09:46AM (#25160885)
> It's extremely aggravating how most file systems can't create a pool of storage out of many drives.

Some would argue that is the job of the volume manager, not the filesystem. On Linux it's LVM2, FreeBSD has vinum, for example.

Re:An Introduction to ZFS by matheusmoreira on Friday November 20, 2020 @12:42PM (#25162829)
I would argue that a volume manager is just one of many parts of a good file system. Managing physical devices at the file system level has advantages: the file system can balance its data automatically in the background. Linux LVM can't do that.
Re:An Introduction to ZFS by geerlingguy on Friday November 20, 2020 @02:18PM (#25163847)
I think this gets into the more philosophical part of the debate like with systemd, where it seems like it could be argued that the separation is better—or not—depending on your needs.

And for my own needs, I typically favor the security of not throwing too much complexity on top of my storage system (if it would be impossible for me to recover something without relying on a ton of magic I can't fully understand, I'd rather not use it in production).

Re:An Introduction to ZFS by throw0101a on Friday November 20, 2020 @01:13PM (#25163164)
> Some would argue that is the job of the volume manager, not the filesystem.

ZFS is both at the same time.

That is why there is the zpool command and the zfs command. While tight-coupling is sometimes bad, in this instance it is useful:

* https://web.archive.org/web/20070508214221/http://blogs.sun....

Re:An Introduction to ZFS by cm2187 on Friday November 20, 2020 @08:22AM (#25160188)
And the windows equivalent is Windows Storage Space which sucks hard for parity volumes (max write speed 60MB/s whereas the same set of drives in any other software RAID writes close to 1GB/s).

I would have thought this was a long solved problem. Kind of like chat applications.

Re:An Introduction to ZFS by TillE on Friday November 20, 2020 @03:37PM (#25164667)
It doesn't help that Microsoft has completely abandoned ReFS. Maybe there were good technical reasons why it was a dead-end, I dunno, but it's really no fun being stuck with NTFS.
Re:An Introduction to ZFS by earthscienceman on Friday November 20, 2020 @08:04AM (#25160049)
It really is incredible. ZFS datasets are essentially data collections, and I can categorize my files according to how I want them managed. It's essentially what iOS dreams of but in a much more manageable, configurable, and open way.
Re:An Introduction to ZFS by sly010 on Friday November 20, 2020 @10:02AM (#25161048)
One beef: Moving files between datasets are an unnecessarily expensive operation.
Re:An Introduction to ZFS by seized on Friday November 20, 2020 @11:49AM (#25162203)
Mainly because the dataset is the ZFS version of a partition. In one pool, each dataset can have its own record size, compression settings (on/off or different levels, even different algorithms), be encrypted, have deduplication turned on... Its impossible for it to not be an "expensive operation".
Re:An Introduction to ZFS by AsyncAwait on Friday November 20, 2020 @09:53AM (#25160949)
> Linux is still catching up with btrfs...

ZFS is available on Linux if you want it, in fact FreeBSD is basing its support for ZFS on it.

For a mainline option, am personally holding out hope for bcachefs, rather than btrfs.

Re:An Introduction to ZFS by matheusmoreira on Friday November 20, 2020 @01:54PM (#25163599)
I think the way btrfs manages physical volumes is better. With btrfs, I can slowly buy drives of different capacities and add them to the file system one by one. I don't have to plan ahead of time like traditional RAID and ZFS setups. This is excellent for home data storage!

The only thing that holds it back is the lack of proper parity support. Anyone has an update on that? The kernel wiki says it still has problems...

Re:An Introduction to ZFS by Lex-2008 on Saturday November 21, 2020 @12:52PM (#25171264)
regarding raid5/6 on btrfs - in this recent message [1] Zygo Blaxell writes:

> Not much has changed for raid5/6 since 2014, other than the introduction > of raid1c3 for metadata in 2019 to make filesystem with raid6 data usable. > Almost all of the bugs from 2014 still exist today. Developers have > been fixing more severe and less avoidable bugs in the meantime.

[1]: https://www.spinics.net/lists/linux-btrfs/msg107268.html

Also, worth mentioning another email from same author with guidelines for users running btrfs raid5 array: https://lore.kernel.org/linux-btrfs/20200627032414.GX10769@h...

Re:An Introduction to ZFS by throw0101a on Saturday November 21, 2020 @11:37AM (#25170648)
You may wish to check out the new dRAID mechanism that recently got committed:

* https://www.youtube.com/watch?v=jdXOtEF6Fh0

* https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAI...

Re:An Introduction to ZFS by RMPR on Friday November 20, 2020 @10:50AM (#25161538)
It seems like btrfs will gain a lot of traction, considering that Facebook invest a lot in it and it's the default FS in Fedora now.
Re:An Introduction to ZFS by throw0101a on Friday November 20, 2020 @01:16PM (#25163193)
I first heard about Btrfs over ten years ago: what's taking it so long to 'gain traction'?
Re:An Introduction to ZFS by IgorPartola on Friday November 20, 2020 @01:38PM (#25163412)
My guess is bugs. Early on they had some serious bugs that lost people data. So the project gained this reputation of not being production-ready which meant that only people interested in playing with a new filesystem were using it. And btrfs has always been brought up alongside zfs which has been stable and mature for decades. As far as I can tell the only advantage btrfs has over zfs is the license. Outside of that, what does it offer that zfs does not? (That’s a serious question, I haven’t kept up with it because I’m a happy zfs user.)
Re:An Introduction to ZFS by Lex-2008 on Friday November 20, 2020 @02:37PM (#25164019)
> what does btrfs offer that zfs does not?

As @matheusmoreira mentioned - you can use harddisks of unequal size. I.e. three disks of 4+3+3 Tb in RAID1 will happily give you 5 Tb of usable space. AFAIK, no other filesystem can do that.

Quoting @matheusmoreira further:

> I can slowly buy drives of different capacities and add them to the file system one by one. I don't have to plan ahead of time like traditional RAID and ZFS setups. This is excellent for home data storage!

Re:An Introduction to ZFS by throw0101a on Saturday November 21, 2020 @11:34AM (#25170627)
The new dRAID mechanism in ZFS may help with this:

* https://www.youtube.com/watch?v=jdXOtEF6Fh0

Re:An Introduction to ZFS by matheusmoreira on Monday November 23, 2020 @12:40AM (#25183909)
How? I don't see how it makes ZFS more flexible. Users still need to have every storage device installed on the computer before creating the RAID configuration. They can't start with the drives they already have and then add more drives over time. At least not without essentially recreating the array from scratch.
Re:An Introduction to ZFS by AsyncAwait on Friday November 20, 2020 @12:38PM (#25162789)
Maybe. But I wonder if btrfs has been tainted as 'too unreliable' at this point, whether true or not.
Re:An Introduction to ZFS by jhoechtl on Friday November 20, 2020 @01:42PM (#25163473)
It's been perfectly stable since years for me. Not using Raid 5/6 of course!
Re:An Introduction to ZFS by codys on Friday November 20, 2020 @02:21PM (#25163869)
Are you using snapshots? qgroups (which you have to enable to see snapshot space utilization)?. I've had bad experiences with them on btrfs.
Re:An Introduction to ZFS by Lex-2008 on Friday November 20, 2020 @02:42PM (#25164082)
not OP, but using btrfs for 0.5 year with snapshots, snapper (which creates snapshots every hour) on NVMe under Arch. Haven't noticed any issues yet - qgroups are disabled, and snapper config decreased to have no more than 20 snapshots.
Re:An Introduction to ZFS by codys on Monday November 23, 2020 @04:49PM (#25192350)
Ya, I was using snapper but didn't restrict the number of snapshots so I probably had more than 20

I use zfs now and have many, many, more than 20 snapshots without issue.

I wasn't using a nvme drive with btrfs, it was a sata ssd. So perhaps the higher io ops or lower latency also helps.