Escape analysis leads to Rust's memory safety model. I think we will see a similar cross-domain learnings in optimization, and I don't think NN are it. Monte Carlo simulation seems more likely to me, and that can steal ideas from either game AI, property-based testing, or both.

More boringly, treating the compilation metadata more as a database may also be waiting to be exploited. Right now we have feedback-directed optimization which basically takes a 1:many or 1:1 approach of one data set gives you one binary. Next compile you start with new input and run the process again, whereas retaining a history might allow for deeper tree searches.

It's always annoyed me a little that every single time my code loads or gets compiled that the runtime has to start from ground state and build up. If I run the same compile fifty times in a row it's always the same output and it always takes the same amount of time. If a big improvement is just beyond the search budget it will forever remain out of sight. Especially in a CI/CD world my ratio of changes to binaries is very, very low, and so the waste is much more pronounced.

If instead you store a map of decisions and constraints, can I test the constraints, flush all of the decisions whose constraints are violated, and begin my search tree from there? Going a little deeper every time in stable areas of the code?