It very well might be that gap has grown, though I think the expectations of optimizations of the code has grown too. On the other hand, we now have more cores than we know what to do with, so an interesting approach would be to start generating code, and inserting calls to the code generator for pieces that hasn't been generated yet, and farm out generation to multiple cores.

If the code is structured right, and you could certainly try to sort the code based on a dependency graph, you don't even need to load the entire file before you can start generating code from SDE (but you'd need to be prepared to stub out forward jumps so they point to something that'll generate or wait for generation of the rest where necessary).

Well, after quickly skimming through at the PDFs linked here, it looks like SDE are a good data structure for IR, and that one could compile one SDE into another.

So it looks like you could do some kind of heavy "portable optimisation" pass first, then distribute the resulting SDE of that, and do a second, lighter pass of "local" optimisations targeting the local architecture.

That's true. There's also always the option of treating SDE as an "installation format" rather than "runtime format" and generate optimized code for a given architecture at installation time. Optionally it opens the door for funky things such as generating code instrumented for profile guided optimization first, and in the background/offline continue to optimize the code.

(in fact, I believe quite a lot of Franz' subsequent research has been in areas like that, though mostly JVM focused)