News for hackers. Stuff that matters.
Re:Douglas Hofstadter: The Shallowness of Google Translate (... by YeGoblynQueenne on Monday November 18, 2019 @05:43AM (#21563262)
aⁿbⁿ is a context-free language, not regular. The fact that it's simple is exactly the point: it is so simple, yet neural nets cannot learn it. They can only learn limited subsets of it and even those, from tens of thousands of examples. I showed how the same language can be learned in full from three examples. It's a task in which neural nets, despite all their successes that you cite, fail dismally. You dismiss it easily, but it's an ebmarrassing failure and it makes clear the weaknesses of the connectionist paradigm.

Despite myself, I am still surprised by the unwillingness of connectionists to admit this simple fact: neural nets generalise very poorly and have absolutely awful sample efficiency.

As to more complex, real-world tasks, this is a recent example on a language task using the same algorithm (but a different implementation):

Bias reformulation for one-shot function induction

https://dspace.mit.edu/handle/1721.1/102524

There's more of that in the literature.

There are no comments on this comment yet.