Re:DwarFS: A fast high compression read-only file system
by TimTheTinker on Sunday November 29, 2020 @01:10PM
(#25246787)
This is a read-only file system, so it’s able to exploit certain properties of that—- locating similar files next to each other, for example.
[ Parent
| on HN ]
Re:DwarFS: A fast high compression read-only file system
by hachari on Sunday November 29, 2020 @02:21PM
(#25247285)
I don’t see how read only helps at all.
Btrfs can dedupe at the block level.
Re:DwarFS: A fast high compression read-only file system
by _flux on Sunday November 29, 2020 @03:34PM
(#25247836)
Consider how much of work in btrfs is done to just handle the case of modifying existing files—or reducing file system size.. It is basically the reason it uses b-trees. It's in the name!
For example, when dedupping in block level it needs to know (right?) how many times a block is being used, so it can be collected when it runs out of uses.
ISO9660 can also express dedupped (hardlinked) files with the Rock Ridge extensions. I don't know but I'm wondering it could even do block-level dedupping if the generating program abused the format a bit..
Re:DwarFS: A fast high compression read-only file system
by fsiefken on Sunday November 29, 2020 @05:40PM
(#25248755)
A compression benchmark of both filesystems would be of interest in this regard (lzo, zstd and zlib), both read speed and compression wise
Re:DwarFS: A fast high compression read-only file system
by TimTheTinker on Sunday November 29, 2020 @05:05PM
(#25248521)
Sure, but it sounds like block deduping is only one of several optimizations that DwarFS is able to take advantage of because it’s a read-only FS.