That's true. There's also always the option of treating SDE as an "installation format" rather than "runtime format" and generate optimized code for a given architecture at installation time. Optionally it opens the door for funky things such as generating code instrumented for profile guided optimization first, and in the background/offline continue to optimize the code.
(in fact, I believe quite a lot of Franz' subsequent research has been in areas like that, though mostly JVM focused)
So it looks like you could do some kind of heavy "portable optimisation" pass first, then distribute the resulting SDE of that, and do a second, lighter pass of "local" optimisations targeting the local architecture.