News for hackers. Stuff that matters.
Re:DwarFS: A fast high compression read-only file system by hachari on Sunday November 29, 2020 @12:40PM (#25246587)
Why not use BTRFS with file deduplication and transparent compression (zstd specifically)?
Re:DwarFS: A fast high compression read-only file system by TimTheTinker on Sunday November 29, 2020 @01:10PM (#25246787)
This is a read-only file system, so it’s able to exploit certain properties of that—- locating similar files next to each other, for example.
Re:DwarFS: A fast high compression read-only file system by hachari on Sunday November 29, 2020 @02:21PM (#25247285)
I don’t see how read only helps at all.

Btrfs can dedupe at the block level.

Re:DwarFS: A fast high compression read-only file system by _flux on Sunday November 29, 2020 @03:34PM (#25247836)
Consider how much of work in btrfs is done to just handle the case of modifying existing files—or reducing file system size.. It is basically the reason it uses b-trees. It's in the name!

For example, when dedupping in block level it needs to know (right?) how many times a block is being used, so it can be collected when it runs out of uses.

ISO9660 can also express dedupped (hardlinked) files with the Rock Ridge extensions. I don't know but I'm wondering it could even do block-level dedupping if the generating program abused the format a bit..

Re:DwarFS: A fast high compression read-only file system by fsiefken on Sunday November 29, 2020 @05:40PM (#25248755)
A compression benchmark of both filesystems would be of interest in this regard (lzo, zstd and zlib), both read speed and compression wise
Re:DwarFS: A fast high compression read-only file system by TimTheTinker on Sunday November 29, 2020 @05:05PM (#25248521)
Sure, but it sounds like block deduping is only one of several optimizations that DwarFS is able to take advantage of because it’s a read-only FS.
Re:DwarFS: A fast high compression read-only file system by tutfbhuf on Sunday November 29, 2020 @01:14PM (#25246808)
Is Btrfs stable yet?
Re:DwarFS: A fast high compression read-only file system by packetlost on Sunday November 29, 2020 @02:36PM (#25247414)
For most use-cases, yes. But not if you plan on using raid5+ [0]

[0]: https://btrfs.wiki.kernel.org/index.php/Gotchas#Parity_RAID

My own experience indicates it's brittle to power failures and will corrupt in annoying ways in the event of a power failure (or hard reboot) as of ~1 year ago.

Re:DwarFS: A fast high compression read-only file system by hachari on Sunday November 29, 2020 @02:39PM (#25247433)
Year isnt as useful as kernel version.
Re:DwarFS: A fast high compression read-only file system by packetlost on Sunday November 29, 2020 @02:59PM (#25247584)
I was running whatever Arch's Linux kernel was at the time. 5.4 I believe? Arch is pretty much bleeding edge, so year is relevant.
Re:DwarFS: A fast high compression read-only file system by tutfbhuf on Sunday November 29, 2020 @03:02PM (#25247603)
Yup, in Arch even month, week and day is relevant.
Re:DwarFS: A fast high compression read-only file system by Diederich on Sunday November 29, 2020 @01:57PM (#25247105)
Re:DwarFS: A fast high compression read-only file system by hachari on Sunday November 29, 2020 @02:23PM (#25247309)
Btrfs is default in Fedora 33.
Re:DwarFS: A fast high compression read-only file system by tutfbhuf on Sunday November 29, 2020 @02:57PM (#25247575)
Fedora also has Wayland as default. If Debian had Btrfs as default, that would be an argument.
Re:DwarFS: A fast high compression read-only file system by Arnavion on Sunday November 29, 2020 @11:49PM (#25250832)
It's also been the default for the root filesystem in openSUSE since 2012. It also integrates with the package manager - zypper creates pre- and post- snapshots when installing updates, and the snapper utility makes it easy to roll back if your system is busted after installing an update. It's saved my bacon on multiple occasions.

(btrfs also used to be the default for /home, though that changed at some point. When I made a new install last year the installer suggested xfs by default.)

Re:DwarFS: A fast high compression read-only file system by amelius on Sunday November 29, 2020 @04:48PM (#25248405)
I'm only ever switching to a new filesystem if its correctness has been formally verified.
Re:DwarFS: A fast high compression read-only file system by tutfbhuf on Sunday November 29, 2020 @06:10PM (#25248969)
For which filesystems is that case?
Re:DwarFS: A fast high compression read-only file system by amelius on Monday November 30, 2020 @06:56AM (#25252888)
None. But the tools are there, so it should be possible.