Funny thing about it is that I've got a similar problem powering https://perl.bot/ (and the associated irc bot). I don't have as many installs as you currently but It's not far off and I want to add more compile time settings to them. I'd need to setup a full build server/system though because I need to regularly update them with new modules.

How opposed would you be to this being reworked to being able to be mainline kernel support too?

> How opposed would you be to this being reworked to being able to be mainline kernel support too?

I don't see any way of getting this anywhere near the kernel without a full rewrite. It's C++ and it depends on libraries that aren't even shipped by a lot of distributions (folly & fbthrift). And, tbh, I don't see much benefit given that FUSE these days doesn't seem to be significantly worse in terms of performance.

> I'd need to setup a full build server/system though because I need to regularly update them with new modules.

Overlay the mounted read-only fs with a read-write fs. Then you can install modules as you like and if you want to start fresh, just throw away the read-write fs. That's what I've done in the past.

Is it possible to rebuild a DwarFS fs to incorporate changes from an overlay fs without decompressing, then recompressing?

It seems feasible that a second DwarFS fs could be built from an overlay/DwarFS, then delete the original overlay/DwarFS fs. That would require 2N storage as the new DwarFS is being built. Is it possible to patch an existing DwarFS?

By overlay, are you referring to overlayfs [0]?

https://wiki.archlinux.org/index.php/Overlay_filesystem

Yes, and I've documented this now:

https://github.com/mhx/dwarfs/blob/main/doc/dwarfs.md#settin...

> Overlay the mounted read-only fs with a read-write fs. Then you can install modules as you like and if you want to start fresh, just throw away the read-write fs. That's what I've done in the past.

It would be nice to be able to build a new read-only filesystem in incremental mode: given a compressed filesystem and some new uncompressed data, incorporate the uncompressed data without completely re-doing all the work.

> "taking up something around 30 gigabytes of disk space, and I was unwilling to spend more than 10% of my hard drive"

I imagine these days you have more than 300GB hard disk space, making this all moot?

256GB SSDs are still everywhere.