As for the mainframes and minis, I never worked with them, besides the AS/400.
But from my collection of old papers and manuals, many of them had "bytecodes" with microcoded execution (more higher level than plain CISC Assembly) burned on their processors.
See for example this paper, where they took a VAX and rewrote its microcode so that it executed instructions for the Warren Abstract Machine, the output format of their Prolog compiler.
It's like you would reprogram your Core i7 to directly consume JVM bytecode.
I was a big Prolog fan during my degree.
And frankly, its design is not that far from the other micros (ISA cards can extend the BIOS by responding to specific memory address ranges, much like a ROM cart on a C64). Its just so damn well wallpapered over these days.
OS choices and OS upgrades happened regularly. You could run all sorts of things on a PDP/11, VAX, DG Nova, etc.