Is the issue here that images shouldn't be thought of as layers, but rather a tree of cached directory nodes? I don't quite follow what's meant by building here, are you referring to compiling or merging the resulting build artifacts into a final container image?

Non-copyable build outputs sound a bit wild - you're thinking of builds that encode absolute paths into the output binaries?