News for hackers. Stuff that matters.
Re:Is the reversal curse in LLMs real? by johnfn on Wednesday November 15, 2023 @02:00AM (#38274055)
You're thinking of LLMs as basically pattern matching "a is b", but that's not really how they work. In fact, the incredible thing that LLMs can do is that they can understand (some) colloquial language and shades of meaning. LLMs can grasp all sorts of strange textual nuance; they can't grasp the cases "Foo is Bar's mother" strictly implies "Bar is Foo's son"?
Re:Is the reversal curse in LLMs real? by og_kalu on Wednesday November 15, 2023 @02:33AM (#38274216)
>they can't grasp the cases "Foo is Bar's mother" strictly implies "Bar is Foo's son"?

Correction. The Model can. Training cannot.

Re:Is the reversal curse in LLMs real? by TeMPOraL on Wednesday November 15, 2023 @02:47AM (#38274287)
I'm starting to wonder, why should it in the first place? Very late stages, or fine-tuning, I'd perhaps understand. But early on? Everything is possible, and "Foo is Bar's mother" doesn't imply much over "these tokens come together, sometimes".

It's not that humans aren't suffering from this too. It's possible to spot it when you're learning, or even easier, when you're teaching someone a new thing. A person who learned that "A is B" does not automatically learn that "B is A"; they need to first process it, perhaps run the reversion explicitly in their head. I'd say that checking if the student can infer "B is A" having learned "A is B" is a good way to tell whether they're starting to comprehend the material, vs. just memorizing it.