Yes! Managing our files is the whole point of file systems! It's amazing how bad at it most of them are. Linux is still catching up with btrfs...
It's extremely aggravating how most file systems can't create a pool of storage out of many drives. We end up having to manually keep track of which drives have which sets of files, something that the file system itself should be doing. Expanding storage capacity quickly results in a mess of many drives with many file systems...
Unlike traditional RAID and ZFS, btrfs allows adding arbitrary drives of any make, model and capacity to the pool and it's awesome... But there's still no proper RAID 5/6 style parity support.
Some would argue that is the job of the volume manager, not the filesystem. On Linux it's LVM2, FreeBSD has vinum, for example.
And for my own needs, I typically favor the security of not throwing too much complexity on top of my storage system (if it would be impossible for me to recover something without relying on a ton of magic I can't fully understand, I'd rather not use it in production).
ZFS is both at the same time.
That is why there is the zpool command and the zfs command. While tight-coupling is sometimes bad, in this instance it is useful:
* https://web.archive.org/web/20070508214221/http://blogs.sun....
I would have thought this was a long solved problem. Kind of like chat applications.
ZFS is available on Linux if you want it, in fact FreeBSD is basing its support for ZFS on it.
For a mainline option, am personally holding out hope for bcachefs, rather than btrfs.
The only thing that holds it back is the lack of proper parity support. Anyone has an update on that? The kernel wiki says it still has problems...
> Not much has changed for raid5/6 since 2014, other than the introduction > of raid1c3 for metadata in 2019 to make filesystem with raid6 data usable. > Almost all of the bugs from 2014 still exist today. Developers have > been fixing more severe and less avoidable bugs in the meantime.
[1]: https://www.spinics.net/lists/linux-btrfs/msg107268.html
Also, worth mentioning another email from same author with guidelines for users running btrfs raid5 array: https://lore.kernel.org/linux-btrfs/20200627032414.GX10769@h...
* https://www.youtube.com/watch?v=jdXOtEF6Fh0
* https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAI...
As @matheusmoreira mentioned - you can use harddisks of unequal size. I.e. three disks of 4+3+3 Tb in RAID1 will happily give you 5 Tb of usable space. AFAIK, no other filesystem can do that.
Quoting @matheusmoreira further:
> I can slowly buy drives of different capacities and add them to the file system one by one. I don't have to plan ahead of time like traditional RAID and ZFS setups. This is excellent for home data storage!
I use zfs now and have many, many, more than 20 snapshots without issue.
I wasn't using a nvme drive with btrfs, it was a sata ssd. So perhaps the higher io ops or lower latency also helps.
ZFS is great. What todays developers will make out of it... is worrysome.
> electron snapshots on each run
...on a raspberry pi, for maximum trendiness, deployed in a check out aisle where you have to wait for it to painfully work through its issues after every interaction. Kill me now.
... for me using ZFS has changed the way I look at files, filesystems, data, and backups for general computing. I've been a linux user for 13 years but never felt the need to have a fileserver. Now being able to plug a drive in and take a snapshot without rsyncing or thinking about what I'm snapshotting, having it be inherent to the filesystem, was a game changer.
Not to mention being able to snapshot important folders to the native drive in case I need to recover a file from a previous state. I run datasets for categories of data and I can choose categories that I want regular local snapshots of (zvol/crypt/Documents, zvol/crypt/scripts, zvol/crypt/Papers)
Essentially, ZFS manages my files for me. And it all comes with things I didn't know I needed, like filesystem compression. I know BTRFS also attempts to provide this, and there's the licensing issues with ZFS, but I wanted MacOS compatibility also. Although that was an adventure on its own.