I was initially a bit skeptical if the page cache would work with all layering docker does. But OverlayFS, perhaps unsurprisingly, does indeed support that: [1] "OverlayFS supports page cache sharing. Multiple containers accessing the same file share a single page cache entry for that file. This makes the overlay and overlay2 drivers efficient with memory and a good option for high-density use cases such as PaaS." TIL

[1] https://docs.docker.com/storage/storagedriver/overlayfs-driv...

UPX creates anonymous pages that contain the decompressed payload, right? Those aren’t file-backed pages so how would the filesystem even know those pages exist?

That’s correct and I guess only page deduplication might help in that case. What I meant in my post was just to state that docker can use the page cache despite the massive indirection caused by overlays. I was wondering if that was even possible and it indeed is.

The base would consume “up to once”. You will only load pages touched, and they are all reclaimable under memory pressure. Memory used by UPX for decompression is guaranteed to be used, and is not reclaimable!

So in fact, it’s not might use more resources, it’s guaranteed to use more resources.

The only benefit of UPX is saving disk space, which is unlikely to be super useful to most people these days.

The other use case is to distribute images more quickly over the network and improve image pull times, at the expense of slightly more memory use.

Similar to how saving disk is less useful to most people these days, RAM on servers in big docker/kubernetes clusters is also likely not much of a concern for binaries that are well under 100MB if you're not running a ton of copies on each server. In particular, it seems like this tradeoff is most useful for static go binaries, which don't benefit from shared system libraries and aren't likely to have many copies per server, but of course that definitely depends on the use case and I definitely do not recommend blindly using UPX without considering the tradeoffs.

It would be an amazing integration though if it were possible to integrate UPX with docker's layer compression such that the binary is extracted to the FS before running. But given how it all works, that's pretty challenging to implement and the network is only getting faster...

Are you sure UPX actually beats the standard layer compression which is already provided?

After applying UPX, the result should be nearly uncompressable, so if UPX performs worse than layer compression then it might be better for network usage to not use it at all.

In this scenario where you are unpacking it in advance, that eliminates the main benefit of UPX which is that it is self-extracting.

Definitely important to note, I wonder if upx could solve this via execing an memfd_create in some mode. Also wish upx came with a trivial extraction tool so you could use it for distribution but not runtime.

Aren’t your container layers compressed in transit anyway?

UPX does significantly more compression on binaries than gzip can on the layers. It's not just simple compression, it makes a lot of modifications to the binary to make it more compressible.

Perhaps this is what you were asking for in the GP, but it would be interesting then if UPX could apply those transformations without actually doing the final compression step itself. Perhaps the win isn't as great without the ability to smuggle specific hints to the compressor, but it would be interesting to see.

On the other hand, if it could be demonstrated that modest increases to binary size correlated to significantly improved compressability, then maybe that's something that should just be considered by the compilers themselves.

that's true, plus on low-end devices upx will take forever to decompress and is essentially unusable.

Is UPX actually that much slower than extracting a gzipped docker layer? I didn't realize UPX decompression was much slower than other compression algorithms