Thank you for sharing!

> Fixed guest branches just get turned into host branches and work like normal.

How does that work in case of self-modifying code, or skewed execution (where the same x86 instruction stream has two totally different interpretations based on what offset you start at)?

Skewed execution are just different traces. Basic blocks don't have a requirement that they don't partially overlap with other basic blocks. You want that anyway for optimization reasons even without skewed execution.

Self modifying code is handled with MMU traps on the writes, and invalidation of the relevant traces. It is very much a slow path though. Ideally heavy self modfying code is able to stay in the interpreter though and not thrash in and out of the compiler.

> Self modifying code is handled with MMU traps on the writes, and invalidation of the relevant traces. It is very much a slow path though. Ideally heavy self modfying code is able to stay in the interpreter though and not thrash in and out of the compiler.

This might end up having a bad time running JavaScript VM JITed code, which self-modifies a lot.

But all of that makes sense! Thanks!

Yeah, nesting JITs was kind of always an Achilles heel of this kind of architecture.

IIRC, they had a research project to look at shipping a custom JVM that compiled straight to their internal ISA to skip the impedance mismatch between two JITs. JITed JS (or really any extremely dynamic code that also asks for high perf) probably wasn't even on their radar given the era with even the SmallTalk VM that HotSpot derived from being a strongly typed derivative of SmallTalk.