Re:Recent Advances in Natural Language Processing
by FeepingCreature on Monday August 17, 2020 @12:53AM
(#24184123)
If logic helps you predict the next word in the text, then a text based model will learn logic.
Models of all kinds are always poor surrogates for reality, and models that cannot employ logic or causality CLEARLY cannot model a world in which mechanisms cause all change.
Statistical models can indeed describe many observations, but can never break out from the echo chamber of copy-catting patterns it has already seen. If a probabilistic engine like a deep net hasn’t been exposed to a concept during training, it will never induce its existence. Imagination requires initiative, the proposal of an unknown and unfamiliar outcome via logical inference or causal induction. Until deep nets can employ both of these skills, they will never master many skills humans use routinely to explain the world or extend our understanding of how it works.