Re:Does Reasoning Emerge? Probabilities of Causation in Larg...
by astromaniak on Friday August 16, 2024 @03:07PM
(#41269386)
For some reasons LLMs get a lot of attention. But.. while simplicity is great it has limits. To make model reason you have to put it in a loop with fallbacks. It has to try possibilities and fallback from false branches. Which can be done on a level higher. This can be either algorithm, another model, or another thread in the same model. To some degree it can be done by prompting in the same thread. Like asking LLMs to first print high level algorithm and then do it step by step.
[ Parent
| on HN ]
Re:Does Reasoning Emerge? Probabilities of Causation in Larg...
by layer8 on Friday August 16, 2024 @03:15PM
(#41269436)
Iteration is important, but I don’t think that it can substantively compensate for the abstraction limitations outlined in the GP comment.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg...
by refulgentis on Friday August 16, 2024 @04:27PM
(#41270066)
> To make model reason you have to put it in a loop with fallbacks
Source? TFA, i.e. the thing we're commenting on, tried to, and seems to, show the opposite
Re:Does Reasoning Emerge? Probabilities of Causation in Larg...
by cma on Friday August 16, 2024 @08:56PM
(#41271605)
The comment may just be pattern matching on the topic, not reasoning about TFA.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg...
by astromaniak on Saturday August 17, 2024 @12:43AM
(#41272217)
When the task, or part of it, is np complete there is no way around. Model has to try all options till it find working one. In a loop. And this can be multi-step with partial fallback. That's how humans are thinking. They can see only to some depth. They may first determine promising directions. Select one, go dipper. Fallback if it doesn't work. Pattern matching mentioned is simplest one step solution. LLMs are doing it with no problems.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg...
by llm_trw on Friday August 16, 2024 @05:35PM
(#41270600)
LLMs get a lot of attention because they were the first architecture that could scale to a trillion parameters while still improving with every added parameter.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg...
by Salgat on Friday August 16, 2024 @11:39PM
(#41272073)
LLMs already do this. Their many wide layers allow for this, and as a final fallback, their output adjusts based on every token they generate (it's not all decided at once). All your statement really means is a vague "well it should do it more!" which yeah, is the goal of each iteration of GPT etc.