On the other hand some optimization are so expensive that a JIT just doesn't have the execution budget to perform them.

Probably the optimal system is an hybrid iterative JIT/AOT compiler (which incidentally was the original objective of LLVM).