Re:John Carmack and Rich Sutton partner to accelerate develo...
by FrustratedMonky on Saturday October 07, 2023 @01:15PM
(#37803522)
Right now we don't know if LLM's are also doing this, or not.
In their calculating the 'next' word, as part of that 'weighting', are the 'simple rules' for future words, that you are saying humans do but LLM's can't.
Re:John Carmack and Rich Sutton partner to accelerate develo...
by taeric on Saturday October 07, 2023 @01:53PM
(#37803922)
Ah, fair. I was only commenting on the idea that you write one word at a time. Seems fairly unlikely to me. You expand out ideas.
Re:John Carmack and Rich Sutton partner to accelerate develo...
by FrustratedMonky on Saturday October 07, 2023 @03:27PM
(#37804730)
LOL. Yes, started as a joke.
Re:John Carmack and Rich Sutton partner to accelerate develo...
by 4death4 on Sunday October 08, 2023 @04:23AM
(#37808756)
It doesn't seem LLMs are capable of "understanding" that they don't "know" something, so there are certainly some observable differences. But the intent of my original comment was precisely to highlight that we don't know. It could be that next token predication is somehow mathematically equivalent to what we call consciousness. That would certainly be a revelation.
Shaping your words to sound correct is very common. Both in speech and in writing. Sometimes it is finding how to fit a word you want to use into a sentence. Sometimes it is building a rhyme.
You may feel that you go a word at a time, but that really shows how embedded language is.