Maybe there will be two classes of applications. The elite, like browsers that are constantly fed data about how people use them, regularly updated so optimization is compiled right in; and the not so elite, that rely on per-run startup optimization for good performance.
IIRC gcc already links in a runtime system for recovering runtime type information about dynamic cast. It's not hard to imagine C++x20 (or whatever) including some kind of runtime optimizer.
See http://en.wikipedia.org/wiki/Profile-guided_optimization