Re:How Chain-of-Thought Reasoning Helps Neural Networks Compute
by lupire on Friday March 22, 2024 @09:15AM
(#39790391)
How does the non-COT system generate the second word of its not using the output as input? Or do you mean that non-COT systems use only the latest output word when computing the next word, not all the earlier words from the output?
[ Parent
| on HN ]
Re:How Chain-of-Thought Reasoning Helps Neural Networks Compute
by HarHarVeryFunny on Friday March 22, 2024 @09:32AM
(#39790553)
Every output word is always appended to the prompt, so if prompt is P then input after W1 (word 1) is output is P W1, then P W1 W2, etc. So, the LLM is only looking at it's own PAST words, not planning ahead.
If you do COT (think step-by-step) then the difference is that it has broken the prompt request/problem down into steps, so while it's still only seeing it's own past words, those now include all of step-1, which helps it generate step-2, etc and eventually combine all steps it generated into a complete answer.