Re:Were RNNs All We Needed?
by roboboffin on Friday October 04, 2024 @02:31PM
(#41744220)
It’s not reasoning, it retrieval of a pattern, and that pattern may contain reasoning.
The prompt engineering is the real reasoning, provided by the human.
Yeah, these kinds of discussions always devolve purely into debates about what's the proper definition of words. Especially on HN where everyone has their "Pedantic Knob" dialed up to 11.
Re:Were RNNs All We Needed?
by roboboffin on Friday October 04, 2024 @03:45PM
(#41744937)
I understand your point. I apologise, if I am coming across pendantic.
My point is computers already follow algorithms, and algorithms contain reasoning; but the computers are not reasoning themselves. At least, not yet!
You're not being pedantic at all. It's a crucial distinction that people try to wave away in favor of hype. Especially since we are so vulnerable to anthropomorphizing.
You weren't being pedantic yourself. My point is that this discussion is ultimately about the definition of words, and that all by itself, makes the discussion meaningless.
I think a "granule" of "reasoning" happens at each inference, and you think there is no reasoning in a single inference. To discuss it further would be a game of whose definition of any given word is correct.
It's like I'm saying a house is made of bricks. You can build a house of any shape out of bricks. But once bricks have been invented you can build houses. The LLM "reasoning" that even existed as early as GPT3.5 was the "brick" with which highly intelligent agents can be built out of, with no further "breakthroughs" being required.
The basic Transformer Architecture was enough and already has the magical ingredient of reasoning. The rest is just a matter of prompt engineering.