For everything else, there is Profile Guided Optimization.
Why not use clang compiling to LLVM bytecode and JIT from that?
There are JIT libraries for C++ for building applications which dynamically compile stuff, I'm not aware of any that dynamically recompile the C++ though.
Seriously though this is misguided. PGO is clearly a step in the right direction but its simply deferred static optimization. Either you optimize in C at the compile time, or with PGO by running a few statistically representative executions, but these are not dynamic.
The only way to make truly dynamic optimizations is by having a runtime environment and code that is interpreted.
The argument that its not often necessary is a different argument and with only a few moments thought can be seen to be untrue. Take for example and strstr like operation that you naively implement using by walking through the source string. Now let's say you do this a lot, all is good because the input is short, now you receive (in a parser for example) something much much bigger. A JIT has privy information and can make determinations that building a index on the source string to make repeated finds faster is better, C does not, even with PGO because you may never have run a test that encounters this situation.
What you cannot do in C/C++ that you can in Java is respond to program inefficiencies at runtime, with runtime knowledge.