Specialized hardware has been losing out against general for years.

There were those "LISP machines" in the early 1980s but when Common Lisp was designed they made sure it could be implemented efficiently on emerging 32-bit machines.

Part of the reason is anytime a specialized hardware is found that works, the generalized hardware steals the feature that makes it faster - basically the story of all the extensions to x86 like SSE, etc.

> But the reality was that JIT allows code to get faster over time, as the JIT improves.

Ehh .. PGO is only somewhat better for JIT than AOT. More often for purely-numerical code the win is because the AOT doesn't do per-machine `-march=native`. It's the memory model that kills JVM performance for any nontrivial app though.

Well, code size is another interesting aspect here. A JIT compiler can effectively create any number of versions for a hot method, based on even very aggressive assumptions (an easy one would be that a given object is non-null, or that the interface only has a single instance loaded). The checks for these are cheap (e.g. it could be encoded as trapping an invalid page address in case of NPEs), and invalidation’s cost is amortized.

Contrast this with the problem of specialization in AOT languages, which can easily result in bloated binaries (PGO does help here quite a lot, that much is true). For example, generics might output a completely new function for every type it gets instantiated with - if the function is not that hot, it actually makes sense to rather try to handle more cases with the same code.