In your previous argument you were comparing what nets did years ago vs now, claiming the only difference is accuracy. Obviously, you either were referring to the results (which now are clearly more accurate), not the methods, or you incorrectly assumed nothing have changed in the methods.

In the first case, your new argument does not make sense, because the complexity of implementation does not matter to the result, and there's a clear improvement to it.

In the second case, I can assure you lots changed. The recent major things being ReLUs, self-supervised learning and attention mechanisms.

I haven’t stated nothing has changed — obviously much has to get better results (and I wasn’t talking about just NN — the field is far bigger). But fundamentally the types of problems being solved — identifying and segmenting images, speech-to-text, etc, are the same. We haven’t gotten anywhere in terms of _understanding_ what’s in that speech that just got turned into text, for example. Sure models like BERT have more command of language than anything previously, but it cannot be used in an AGI sense to make sense of a passage at a fundamental level, or tell you _why_ something it “read” occurred.

> We haven’t gotten anywhere in terms of _understanding_ what’s in that speech that just got turned into text, for example

What test is there to quantify understanding that has been used to determine that it hasn't gotten better?

It seems to me that a lot of the things that are posed as hard things for AI to do are poorly defined phenomenon which we have no empirical test for that we simply infer to be explanations for observed behavior of humans that critics assert that AI can't do without evidence or even a definition upon which a search for evidence could be based.

Here's the problem as I see it: you assume "understanding" and "why" are somehow fundamentally different from what BERT/GPT-2 does, but I see no arguments for that point of view.