I've never seen in practice any jit do better than hand tuned AOT compiled code.

I've seen lambdas and virtuals inlined dynamically. AOT would need to be statically typed and instantiated all the way to do the same thing.

I could create a sample program that would be hard to make as fast in C or C++, with the constraint that the code needs to be dynamically loaded at runtime.

In practice, it's easier to modify a more dynamic program and be sure you're not breaking much. Highly tuned C tends to bake in assumptions about memory lifetime, layouts, call graph, etc. The flexibility to modify is what gives you the ability to target higher level improvements in a shorter amount of time.

Given any concrete implementation, after it's been designed in a more dynamic language, you can convert it to something fast in something like C. But starting from C is probably not a great idea. And not everything can be written multiple times economically.

E.g. the .net GC was prototyped in Lisp, and initially converted to C++ with a translator.

I have, try to make an AOT compiler remove dynamic dispatch across shared libraries and convert it into either direct calls or even inline them.

It's rare, but it can happen occasionally. A just-in-time compiler has access to tracing and profiling that a ahead-of-time compiler may not, which means that it can do things like inline a virtual method call that is actually monomorphic at runtime.

“occasionally”

That’s the keyword here.

I was responding to a comment that had the word "never" in it.