News for hackers. Stuff that matters.

A comment on The Geometry of Truth: Do LLM's Know True and False. Parent comment

Re:The Geometry of Truth: Do LLM's Know True and False by ToValueFunfetti on Thursday October 19, 2023 @07:01PM (#37949978)
What cases are there for $a === $b? I guess there's synonyms and rephrasing, but in general it's not a useful relationship in language
[ Parent | on HN ]
Re:The Geometry of Truth: Do LLM's Know True and False by wizzwizz4 on Thursday October 19, 2023 @07:22PM (#37950138)
Batman is Bruce Wayne.
[ Parent | on HN ]
Re:The Geometry of Truth: Do LLM's Know True and False by xcv123 on Thursday October 19, 2023 @08:41PM (#37950784)
From this paper:

https://owainevans.github.io/reversal_curse.pdf

"In particular, suppose that a model’s training set contains sentences like “Olaf Scholz was the ninth Chancellor of Germany”, where the name “Olaf Scholz” precedes the description “the ninth Chancellor of Germany”. Then the model may learn to answer correctly to “Who was Olaf Scholz? [A: The ninth Chancellor of Germany]”. But it will fail to answer “Who was the ninth Chancellor of Germany?” and any other prompts where the description precedes the name."

[ Parent | on HN ]