Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022]
by BlueTemplar on Monday January 23, 2023 @06:46AM
(#34487732)
Thanks, this made me understand (without reading the whole paper, might do that now) why they were talking about :
> only networks augmented with structured memory (such as a stack or memory tape) can successfully generalize on context-free and context-sensitive tasks
I guess because in practice (for computers, not brains... at least not yet) we can actually cheat by using "extensible on demand memory" rather than an actually infinite one (which is impossible) ?
Also, I wonder if the difference between countable infinite (a Turing's machine tape) and the continuum (that, assuming ZFC, might be somehow achieved by doubling a number for each slot on the Turing tape) might be of relevance even in their finite, extensible at will (for all practical purposes) equivalents ??
> only networks augmented with structured memory (such as a stack or memory tape) can successfully generalize on context-free and context-sensitive tasks
I guess because in practice (for computers, not brains... at least not yet) we can actually cheat by using "extensible on demand memory" rather than an actually infinite one (which is impossible) ?
Also, I wonder if the difference between countable infinite (a Turing's machine tape) and the continuum (that, assuming ZFC, might be somehow achieved by doubling a number for each slot on the Turing tape) might be of relevance even in their finite, extensible at will (for all practical purposes) equivalents ??