News for hackers. Stuff that matters.

A comment on The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT. Parent comment

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by jimmySixDOF on Monday April 10, 2023 @12:11AM (#35509097)
Solving this puzzle is an excellent example of what Ilya Sutskever said recently in the Lunar Society Podcast ..... "It seems predicting the next token well means that you understand the underlying reality that led to the creation of that token"

It is the phase shift increases at this meta associative layer (which nobody seems to have seen coming from LLMs or so soon) that are responsible such feats of apparent comprehension of the question even when the answer provided at the end is wrong. The question now is if bigger training sets et al will lead to more reliable answers. TBD.

[ Parent | on HN ]
Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by uoaei on Monday April 10, 2023 @07:58AM (#35511308)
Ultimately it seems that is the case. However a 1D model of the world is much less useful than an N-D model of the world that is subsequently projected to 1D. Until we build architectures that allow for more "reason space" in the model, we will continue to see obvious blunders.
[ Parent | on HN ]