No, I mean rewrite the critical parts of the "interpreted" program as interpreter primitives/instructions: the best way to eliminate interpreter overhead is to merge N primitives into one.
But I realise we have a different perspective. You seem to consider interpreter VMs as off the shelf components, when I consider VMs as custom components because I'm working mainly with embedded systems. In the embedded world, JIT is often simply not an option either because they are not available for your particular target, or because the resource constrains don't allow it. But still you sometimes want to trade resources for ease of development and/or flexibility.
So far C has been enough. Achieving low (=preferably none) defect rate using C is challenging enough.
Usually FPGAs can take care of performance critical part. They do increase total cost, though.
Rewrite critical parts of interpreter in native code? My point was there's no optimal native code anymore.
Pure interpreters are falling out of fashion anyways. For JITted systems, there's a significant cost for calling native code. At least until JITs actually inline natively called code, possibly even through dynamic library (.so, .dylib, .dll, etc.) call.
We need a lightweight, thin profile guided JIT/AOT engine. No standard library, memory management agnostic. Something that can target different architectures, memory & cache configurations and instruction set extensions that may have been unknown at design phase. A compilation target for some C/C++/Rust/etc. and a runtime that takes care of the architecture details.