It would be difficult as every package/program would need a step to generate the profile data by executing and running the program like the user would.

Is it theoretically possible to perform the profile generation+apply steps dynamically at runtime?

It would be hard to trust the result.

I wouldn't want to support it, but similar things have been done before.

Alexia Massalin's Synthesis[0] (pdf) operating system did JIT-like optimizations for system calls. Here's a LWN article[1] with a summary. Anyone who's interested in operating systems should read this thesis.

HP's Dynamo[2] runtime optimizer did JIT-like optimizations on PA-RISC binaries; it was released in 2000. DynamoRIO[3] is an open source descendant. Also, DEC had a similar tool for the Alpha, but I've forgotten the name.

[0] https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...

[1] https://lwn.net/Articles/270081/

[2] https://dl.acm.org/doi/pdf/10.1145/349299.349303

[3] https://dynamorio.org/

Nat Freidman developed "GNU Rope"[1] from 1998 which, if memory serves, was inspired by a tool that did the same thing in IRIX (cord, I believe).

[1] http://lwn.net/1998/1029/als/rope.html

This is getting way outside the traditional compiler model, but I believe the .NET JIT has been adding more support for this in the last couple versions. One aspect of it is covered at https://devblogs.microsoft.com/dotnet/performance-improvemen...

I wonder if some JIT systems already do that / might be extended in that direction.

based on what "fishgoesblub" commented, building - read: `emerge -e @world` - a gentoo system with profiling forced, and then using it in that "degraded" state for a while ought to be able to inform PGO, right? if there's a really good speedup from putting hot code together, the hottest code after moderate use should suffice to speed up things, and this could continually be improved.

I'm also certain that if there were a way to anonymously share profiling data upstream (or to the maintainers), that would decrease the "degradation" from the first step, above. I am 100% spitballing here. I'm a dedicated gentoo sysadmin, but i know only a small bit about optimization of the sort being discussed here. So it is possible that every user would have to do the "unprofiled profiler" build first, which, if one cares, is probably a net negative to the planet, unless the idea pans out, then it's a huge positive for the planet - man hours, electricity, wear/endurance on parts, etc.