Re:Gemini "duck" demo was not done in realtime or with voice
by modeless on Thursday December 07, 2023 @04:50PM
(#38562292)
You're right, the models do think non-verbally. However, crucially, they can only do so for a fixed amount of time for each output token. What's needed is a way for them to think non-verbally continuously, and decide for themselves when they've done enough thinking to output the next token.
Re:Gemini "duck" demo was not done in realtime or with voice
by Davidzheng on Thursday December 07, 2023 @05:19PM
(#38562619)
Is it clear that humans can think nonverbally (including internal monologue) continuously? As in, for difficult reasoning tasks, do humans benefit a lot from extra time if they are not allowed internal monologue. Genuine question
Insofar as we can say that models think at all between the input and the stream of tokens output, they do it nonverbally. Forcing the structure of reduce some of it to verbal form short of the actual response-of-concern does not change that, just as the fact that humans reduce some of their thought to verbal form to work through problems doesn't change that human thought is mostly nonverbal.
(And if you don't consider what goes on between input and output thought, than chain of thought doesn't force all LLM thought to be verbal, because only the part that comes out in words is "thought" to start with in that case -- you are then saying that the basic architecture, not chain of thought prompting, forces all thought to be verbal.)