Agreed that understanding lags behind when money and cheaper computation (god I hate that term "compute" used as a verb, learn to English my dudes) drives the SOTA rather than actual understanding (on both researchers' and computers' part).

There's a reason we haven't seen a hard takeoff with BERT writing BERT+1 in even better Python and it probably isn't because nobody is trying, it's because BERT is too stupid to write a computer program.