Re:Douglas Hofstadter: The Shallowness of Google Translate (...
by YeGoblynQueenne on Monday November 18, 2019 @05:43AM
(#21563262)
aⁿbⁿ is a context-free language, not regular. The fact that it's simple is
exactly the point: it is so simple, yet neural nets cannot learn it. They can
only learn limited subsets of it and even those, from tens of thousands of
examples. I showed how the same language can be learned in full from three
examples. It's a task in which neural nets, despite all their successes that
you cite, fail dismally. You dismiss it easily, but it's an ebmarrassing
failure and it makes clear the weaknesses of the connectionist paradigm.
Despite myself, I am still surprised by the unwillingness of connectionists to
admit this simple fact: neural nets generalise very poorly and have absolutely
awful sample efficiency.
As to more complex, real-world tasks, this is a recent example on a language
task using the same algorithm (but a different implementation):
Bias reformulation for one-shot function induction
Despite myself, I am still surprised by the unwillingness of connectionists to admit this simple fact: neural nets generalise very poorly and have absolutely awful sample efficiency.
As to more complex, real-world tasks, this is a recent example on a language task using the same algorithm (but a different implementation):
Bias reformulation for one-shot function induction
https://dspace.mit.edu/handle/1721.1/102524
There's more of that in the literature.