Transparent, yes. Fast, no. To get back to real mode they had to reset the CPU.
There were a couple ways to do this. There was the triple fault technique that phicoh mentioned. I'm not sure how portable that was. A triple fault shuts down the processor. Whether or not that reboots or just sits there halted would depend on the motherboard design.
The official way on the PC AT was to ask the keyboard controller to do it. On the PC AT, the keyboard controller was connected to the reset signal, and had a command to tell it to reset the computer. (The keyboard controller was also used to control the A20 line, to workaround another 286 annoyance).
AT compatibles generally included that behavior in their keyboard controllers for compatibility.
Transparent, yes. Fast, no. To get back to real mode they had to reset the CPU
But from the user perspective (remember when the user perspective used to matter?), it just worked and gave them power and functionality they'd never had before.A modern adware-laden web page on gigabit-speed Internet is dog slow in comparison.
A crazy trick that did work is to cause a triple fault in protected mode. That would reset the 286. Then there was a little bit of support from the BIOS. You could tell the BIOS to not run the power on self tests and load the MBR from disk. Instead the BIOS would jump to a specific memory location.
I think that you are mistaken: I am rather sure that the undocumented LOADALL instruction could make something like that possible:
> http://www.os2museum.com/wp/loadall-strikes-again/
Article found via https://retrocomputing.stackexchange.com/a/220/4889
v8086 mode, like many aspects of the x86 architecture, is a big mess.
But Extenders like DOS/16M[0] did this transparently fast by having a real-mode process handle DOS interactions.
Given the proliferation of IBM "compatible" BIOSes in PC clones of that era (many not all that "compatible"), DOS/16M did a fantastic job of running on almost all of them. I worked with it a lot when Informix was first ported to extended-memory DOS usage using DOS/16M. There was a large deployment for a branch of the US armed forces using the product.
[0] http://www.tenberry.com/dos16m/faq/basics.html
When DOS/16M was first released, the company name was Rational Systems.