News for hackers. Stuff that matters.
Re:Is the reversal curse in LLMs real? by bastawhiz on Wednesday November 15, 2023 @09:44AM (#38277090)
> "A is B" does not imply "B is A" in the first place

Not to Bill Clinton this, but I think you are considering this for the wrong interpretation of "is". In this case, "RMS is the founder of the FSF" is the sort of statement implied here. Nobody else was or ever will be the founder of the FSF. So yes, for what the author is describing, "B is A" should always hold. B, in this case, isn't a category, it's an exact identity. The author doesn't explicitly state this, but it's clear this is what they're talking about.

Re:Is the reversal curse in LLMs real? by mistermann on Wednesday November 15, 2023 @09:51AM (#38277179)
You are using your mind to determine what the proper usage is in this specific concrete instance, LLM's do not have your mind to perform that step. LLM's have to work at higher levels of abstraction, and there the multiple meanings of "is" (which is also very commonly used to mean it is my opinion that X "is" Y) makes sorting things out very difficult.

This problem exists for both LLM's and humans, I think it may be fundamental to reality itself, as it currently is at least.

I wonder if they changed the training data to consistently use "equals" where possible (even if it sounds weird to the reader) would make a difference.

Re:Is the reversal curse in LLMs real? by bastawhiz on Wednesday November 15, 2023 @10:39AM (#38277854)
> there the multiple meanings of "is" (which is also very commonly used to mean it is my opinion that X "is" Y) makes sorting things out very difficult.

I'm not going to hype the technology more than it needs, but it seems to do a pretty damn good job of just this. And to the author's point, it does get this right.

And in fact, I'd suspect many of the cases where the LLM gets this wrong are caused by lots of examples in the training data which were written at different times, where someone else was chancellor. Or where the LLM gets numbering wrong (which is a well established problem).

Re:Is the reversal curse in LLMs real? by mistermann on Thursday November 16, 2023 @09:30AM (#38289938)
Sure, but here you are describing your subjective[1] opinion on the matter. I do not disagree that this is your opinion, but the fact of the matter is similar but quite different thing.

And this is just one thing that makes sorting out what's going on here difficult - essentially, we have biologically LLM's [thinking they are] decoding silicon LLM's, of course the results are going to be weird and counterintuitive.

[1] Examine the specific language you are using, but also the colloquial ~interpreted/virtualized assertion that one is left with, if they are reading your comment "in good faith".

Re:Is the reversal curse in LLMs real? by thomastjeffery on Wednesday November 15, 2023 @10:40AM (#38277861)
LLMs don't derive or follow rules. LLMs don't have anything to do with rules.

They implicitly recognize (and model) patterns in text, then continue those patterns.

While the patterns that an LLM has modeled may align to the high abstraction we call language, LLMs actually work at a much lower level of abstraction: plain text.

It's the content of that text that is significant. We encode language into text, and LLMs infer something "close enough" to language by blindly modeling it.

Re:Is the reversal curse in LLMs real? by mistermann on Thursday November 16, 2023 @09:31AM (#38289950)
You don't actually know what LLM's are "doing", comprehensively. You are describing your map/intuition of what is going on - it is a cultural convention (and a well enforced one at that), so an easy mistake to make.
Re:Is the reversal curse in LLMs real? by naasking on Wednesday November 15, 2023 @10:40AM (#38277871)
LLMs effectively disambiguate in an arguably similar way humans would: look at the same term in multiple contexts to reduce confidence in incorrect interpretations, leaving only correct interpretations.
Re:Is the reversal curse in LLMs real? by mistermann on Thursday November 16, 2023 @09:33AM (#38289965)
[delayed]
Re:Is the reversal curse in LLMs real? by marcosdumay on Wednesday November 15, 2023 @10:32AM (#38277752)
LLMs are supposed to derive a huge amount of those rules from their training sets. That's the one thing they are good at. They should be pretty good at differentiating the reflexive from the non-reflexive usage of "is" from the context pretty fine.

The fact that the current ones don't is surprising.