However, you can use ML in areas where compilers use heuristics. For example, how many times should you unroll this loop, or should you inline function B into function A? Your program is going to be valid with or without inlining/unrolling, and the compiler needs an "intuition" of what's going to be best for performance. Right now, in most cases, this uses simple hardcoded rules.
The same goes for math. You wouldn't ask a neural network to certify whether a mathematical proof is valid or not, knowing it has been 99% accurate on the test set. You couldn't trust it to verify all the steps. However, you could use a neural network to help you guide a formal theorem prover in searching for a proof, as in which branches of the infinite tree of mathematical expressions should you search.
It is my understanding that a lot of current applications take a ton of training time / computing power and it's not so trivial to do in the moment. I guess this is less of a theoretical problem and more of a practical problem though.
Yeah, exactly. Theoretically, ML can do anything. Practically, not so much.
And then, if ML is as good as we are, and clearly we're not good enough that we had to write a systematic proof system to compile things correctly, could ML have invented this same technique and implemented that compiler correctly?
In the end, we're just back at the same place, can ML be as intelligent as we are. Is it the holy grail of AI? What are its limits?
Either the thing you want to do is impossible or else a learned model can do it at least almost as well as... idk what the alternative even is, something not learned?
Taking this to the extreme, on an example where I have first-hand experience: I would never recommend replacing your compiler passes with transformers. The latter will be buggier, at best marginally faster, and it will take several orders of magnitude more effort to get them to work well enough for production. ML isn't the right tool. But, I mean, you can do it. You shouldn't. But you can.
To be fair to the article, the title is "won't", not "can't". And that's easier to believe, at least for me. It's not that X is unattainable using ML; it's that some other approach will get there first.