Re:Programming with Natural Language Is Going to Work (2010)
by ankurdhama on Wednesday May 24, 2023 @08:31AM
(#36057095)
> The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token.
I think they only develop abstract understanding about the relationships of the words/group of words.
It is not obvious to me that this is true.
The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token.
You'll find different opinions as to wether this happens with todays llm's, but it's definitely not a clear cut issue.