Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022]
by cma on Monday January 23, 2023 @09:55AM
(#34489459)
It seems transformers don't generalize in this class and neural turing machines and neural stack machines do.
You get the general rule (in linguistics known as competency) but can't flawlessly do it (performance). Transformers can't seem to get competence here.
You get the general rule (in linguistics known as competency) but can't flawlessly do it (performance). Transformers can't seem to get competence here.
https://en.m.wikipedia.org/wiki/Linguistic_competence