First, the JIT compiler has often done most of its work during the first in-process iteration, so the plots are not generally following a "one in-process iteration is slow, then the JIT kicks in from in-process iteration 2 onwards" model. Put another way, the JIT compiler is often making even the first in-process iteration run pretty fast.

Second, I would suggest that the often small timing differences are more worrying than they may first appear. In a semi-mature compiler, optimisations are frequently in the range of a 0.5-1% improvement. So if your measurements are only accurate to (say) 2%, most of your attempted optimisations will be misclassified (bad optimisations will sometimes be measured as good; good optimisations will sometimes be measured as bad). Furthermore, when a VM recompiles things, it generally performs several optimisations at once. Thus, if the overall performance gets worse (even if by a small bit), it may suggest that several optimisations performed badly at the same time.