Is it theoretically possible to perform the profile generation+apply steps dynamically at runtime?

It would be hard to trust the result.

I wouldn't want to support it, but similar things have been done before.

Alexia Massalin's Synthesis[0] (pdf) operating system did JIT-like optimizations for system calls. Here's a LWN article[1] with a summary. Anyone who's interested in operating systems should read this thesis.

HP's Dynamo[2] runtime optimizer did JIT-like optimizations on PA-RISC binaries; it was released in 2000. DynamoRIO[3] is an open source descendant. Also, DEC had a similar tool for the Alpha, but I've forgotten the name.

[0] https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...

[1] https://lwn.net/Articles/270081/

[2] https://dl.acm.org/doi/pdf/10.1145/349299.349303

[3] https://dynamorio.org/

Nat Freidman developed "GNU Rope"[1] from 1998 which, if memory serves, was inspired by a tool that did the same thing in IRIX (cord, I believe).

[1] http://lwn.net/1998/1029/als/rope.html

This is getting way outside the traditional compiler model, but I believe the .NET JIT has been adding more support for this in the last couple versions. One aspect of it is covered at https://devblogs.microsoft.com/dotnet/performance-improvemen...

I wonder if some JIT systems already do that / might be extended in that direction.