> It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress.
So I point out a recent example of progress (of which the sequence of fundamental insights is nearly always incremental), where the theoreetical insight was a theoretical derivation, which could be and probably was derived on paper / blackboard, as a direct counterexample: it does not require "incredible amounts of [...] theory".
Why would you compare this with performing manual backpropagation?
>Everything is easy in hindsight
Every breakthrough is non-trivial, else it would not have been a breakthrough, and yet the breakthrough itself can be a relatively simple calculation...
The concept of "AGI" is undefined and virtually worthless to me. There is only non-trivial insights, i.e. theorem and proof.
Everything is easy in hindsight.
And that's without mentioning that the specific result you are talking about is an infinitesimal progress when compared to "AGI".