News for hackers. Stuff that matters.
Re:Is the reversal curse in LLMs real? by foobarqux on Tuesday November 14, 2023 @09:34PM (#38272640)
But that's the problem isn't it? In some cases they are equivalent and in some cases they are not and a next-word-predictor needs to have "explicit" training data (i.e. it is not doing "reasoning") whereas a human can infer them.

This isn't surprising if you think about how the machine actually works rather than treating it like a sacred magical box.

Re:Is the reversal curse in LLMs real? by og_kalu on Wednesday November 15, 2023 @06:57PM (#38284123)
LLMs can already reverse logical implications. Literally this is not a problem in context. The model can infere all this just fine. Even the original paper makes this clear. This is a recall from training problem, not a logical inference one.
Re:Is the reversal curse in LLMs real? by foobarqux on Thursday November 16, 2023 @06:19PM (#38297067)
I don't understand what you are saying. LLMs can't generally reverse implications, that's what the paper says. This in turn suggests that what you are observing in-context is some other statistical heuristic, not a "logical inference capability".

I don't know in what way this is some trivial "recall" problem as you suggest. Does the reversed implication in question need to be in the training data explicitly or not? If it does I don't understand how you can claim a logical inference capability.

Re:Is the reversal curse in LLMs real? by og_kalu on Thursday November 16, 2023 @08:23PM (#38298065)
>LLMs can't generally reverse implications, that's what the paper says

No that's not what the paper says. It says training won't immediately store this information in a way as to make it reversible.

Let's get one thing straight. This is a common problem for human learning as well. Anyone who has used Anki for language learning will tell you that if you just train on target language word on the front and native language word on the back, you will fail the reverse unless you specifically train for that.

This is specifically a problem of recall. Not a problem of making the logical inference. If you ask the human language learner immediately he has learnt the new word for the reverse direction, he will obviously tell you the correct word.

But later he may not recall the reverse even if he remembers the original. Again this is a problem of recall rather than the ability to make logical inferences.

In the same vein, if you give the LLM the original direction in context and immediately ask the reverse, it will correctly tell you. It can make the logical inference.

Re:Is the reversal curse in LLMs real? by foobarqux on Friday November 17, 2023 @11:41AM (#38305979)
Humans have reasoning capabilities that are separate from memory. If you want to show that LLMs have reasoning capability you try to create carefully constructed adversarial experiments that try to distinguish between that hypothesis and others, you do not just show that there are cases that favor your hypothesis.

And although it's a complete waste of time because the answer is obvious when you do do these types of adversarial experiments, like adversarial blockworld, you find that LLMs are not internalizing abstract principles inherent to reasoning and are instead doing something like string-pattern-patching.

Re:Is the reversal curse in LLMs real? by foobarqux on Friday November 17, 2023 @08:43PM (#38313818)
edit: I meant to say "consistent with your hypothesis" not "favor your hypothesis"