> For instance, suppose binaries were typically distributed in a platform-agnostic format, like LLVM intermediate representation or something equivalent.

The Mill does something like this, but only for their own chips. "Binaries" are bitcode that's not specialized to any particular Mill CPU, and get run through the "specializer" which knows the real belt width and other properties to make a final CPU-specific version.