> I just think that the current DNN+SGD approaches are just not that good

I'll add even further. The transformers and etc that we are using today are not good either.

That's evidenced by the enormous amount of memory they need to do any task. We have just taken the one approach that was working a bit better for sensorial tasks and pattern matching, and went all in, adding hardware after hardware so we could brute-force some cognitive tasks out of it.

If we do the same to other ML architectures, I don't think they would stay much behind. And maybe some would get even better results.