https://github.com/wichtounet/gcc-loop-fusion/commits/master...
Get an ML model to take a stab at optimizing, have a solid system for validating that two pieces of code are equivalent according to the C abstract machine, throw away the ML-suggested optimizations which change the semantics of the code, keep the ones which keep semantics the same.
More realistically you can use an AI to generate vaguely realistic PGO data (profile guidance is where the real speed comes from). This has already been done at least by researchers.
That's not a bad idea, but PGO is pretty limited. I'm thinking about things like rewriting a loop to use vector instructions; something which compilers are absolutely terrible at but ChatGPT is actually pretty good at.
Enabling PGO when building the D compiler made it almost 50% faster on some parts of the test suite (and fairly reasonable big files at that) - speedup elsewhere was not as dramatic but still a tidy win.
What ideas do you have for doing that in anything under exponential time?
And maybe you can't make a polynomial time verifier which works in the general case, but maybe you can make a useful one which handles a lot of cases.
What we need instead is "always on" profiling/debugging/inspection tools so that I can see directly and immediately the performance impact of my code changes while I'm typing the code (down to the compiler output as disassembly, annotated with performance warnings like predicted cache misses). Basically the same sort of evolution which gave us warning and error squiggles while typing, but for performance (think godbolt.org, but in realtime and with hot code reloading).
Unless the inner loop is something long or complicated (in which case for reading & maintainability it should maybe be broken down into functions), I generally find the fused version easier for both.
For readability: it is immediately obvious that everything is working on exactly the same data, and the same range within that data, and that one section is not expected to have side effects that affect the running of the subsequent ones.
For maintainability: there are less possibilities of an edit to one part that should affect all being missed elsewhere.