Re:DwarFS: A fast high compression read-only file system
by aarchi on Sunday November 29, 2020 @02:31PM
(#25247371)
Is it possible to rebuild a DwarFS fs to incorporate changes from an overlay fs without decompressing, then recompressing?
It seems feasible that a second DwarFS fs could be built from an overlay/DwarFS, then delete the original overlay/DwarFS fs. That would require 2N storage as the new DwarFS is being built. Is it possible to patch an existing DwarFS?
By overlay, are you referring to overlayfs [0]?
Re:DwarFS: A fast high compression read-only file system
by mhx77 on Monday November 30, 2020 @01:56PM
(#25257440)
Yes, and I've documented this now:
https://github.com/mhx/dwarfs/blob/main/doc/dwarfs.md#settin...
Re:DwarFS: A fast high compression read-only file system
by JoshTriplett on Sunday November 29, 2020 @02:49PM
(#25247511)
> Overlay the mounted read-only fs with a read-write fs. Then you can install modules as you like and if you want to start fresh, just throw away the read-write fs. That's what I've done in the past.
It would be nice to be able to build a new read-only filesystem in incremental mode: given a compressed filesystem and some new uncompressed data, incorporate the uncompressed data without completely re-doing all the work.
I don't see any way of getting this anywhere near the kernel without a full rewrite. It's C++ and it depends on libraries that aren't even shipped by a lot of distributions (folly & fbthrift). And, tbh, I don't see much benefit given that FUSE these days doesn't seem to be significantly worse in terms of performance.
> I'd need to setup a full build server/system though because I need to regularly update them with new modules.
Overlay the mounted read-only fs with a read-write fs. Then you can install modules as you like and if you want to start fresh, just throw away the read-write fs. That's what I've done in the past.