The thing that makes this mostly theoretical is that the underlying assumption is only true when you neglect that an AOT has zero run-time cost while a JIT compiler has to execute the code it's optimizing and the code to decide if it's worth optimizing and generate new code.
So JIT compiler optimizations are a bit different than AOT optimizations since they have to both generate faster/smaller code and the execute code that performs the optimization. The problem is that most optimizations beyond peephole are quite expensive.
There's another thing that AOT compilers don't need to deal with, which is being wrong.Production JITs have to implement dynamic de-optimization in the case that an optimization was built on a bad assumption.
That's why JITs are only faster in theory (today), since there are performance pitfalls in the JIT itself.
They still want to swap in that JIT'd result ASAP since after all by the time it's been flagged for compilation it's already too late & is a hot hot hot function.
Concretely, I think you'd want hardware that supported reservoir sampling of where CPU cycles are spent, sampling of which branches are mispredicted, and which code locations are causing cache misses. You'd also want lightweight hardware recording of execution traces.
The Javascript V8 engine, and the JVM both are popular and supported enough that I expect the teams working on them take advantage of every trick they can for specific CPUs, they have a lot of resources for this. (at least the major x86 and ARM chips - maybe they don't for MIPS or some uncommon variant of ARM...). Of courses there are other JIT engines, some uncommon ones don't have many resources and won't do this.
Not to the extent clang and gcc do, no. V8 does, e.g. use AVX instructions and some others if they are indicated to be available by CPUID. TurboFan does global scheduling in moving out of the sea of nodes, but that is not machine-specific. There was an experimental local instruction scheduler for TurboFan but it never really helped big cores, while measurements showed it would have helped smaller cores. It didn't actually calculate latencies; it just used a greedy heuristic. I am not sure if it was ever turned on. TurboFan doesn't do software pipelining or unroll/jam, though it does loop peeling, which isn't CPU-specific.
They are not very good at it, and can't be. You can look inside them and see the models are pretty simple; the best you can do is optimize for the first step (decoder) of the CPU and avoid instructions called out in the optimization manual as being especially slow. But on an OoO CPU there's not much else you can do ahead of time, since branches and memory accesses are unpredictable and much slower than in-CPU resource stalls.
In theory JIT can be a lot more efficient, optimizing for not only the exact instruction set, and do per CPU architecture optimizations, such as instruction length, pipeline depth, cache sizes, etc.
In reality I doubt most compiler or JIT development teams have the resources to write and test all those potential optimizations, especially as new CPUs are coming out all the time, and each set of optimizations is another set of tests that has to be maintained.