The problem is when you have more than one such service. Now when one of them changes, all of them need to be rebuilt. You can solve this with multi-stage builds, but those only work if your build result can be easily copied.

Is the issue here that images shouldn't be thought of as layers, but rather a tree of cached directory nodes? I don't quite follow what's meant by building here, are you referring to compiling or merging the resulting build artifacts into a final container image?

Non-copyable build outputs sound a bit wild - you're thinking of builds that encode absolute paths into the output binaries?

In theory yes, but that doesn't work well on github actions for example. The cache layering seems quite bad there for some reason.