News for hackers. Stuff that matters.

A comment on How outdated information hides in LLM token generation probabilities. Parent comment

Re:How outdated information hides in LLM token generation pr... by HeatrayEnjoyer on Sunday January 12, 2025 @08:15AM (#42673319)
Attention models learn what to pay attention to.

It's been found that data that begin with "Wikipedia:" are automatically weighted higher by language models during training, completely unsupervised.

[ Parent | on HN ]
Re:How outdated information hides in LLM token generation pr... by blueflow on Sunday January 12, 2025 @08:36AM (#42673428)
But this is the same problem - Wikipedia is a secondary source and should always get overruled by the primary source.
[ Parent | on HN ]
Re:How outdated information hides in LLM token generation pr... by dotancohen on Sunday January 12, 2025 @11:47AM (#42674822)
And there are documented cases of concentrated efforts to manipulate the content of Wikipedia.
[ Parent | on HN ]
Re:How outdated information hides in LLM token generation pr... by brookst on Sunday January 12, 2025 @01:24PM (#42675551)
You’re probably being facetious but just in case:

https://www.bbc.com/news/technology-28481876.amp

https://www.bbc.com/news/technology-58559412.amp

(With apologies for amp links)

[ Parent | on HN ]
Re:How outdated information hides in LLM token generation pr... by dotancohen on Sunday January 12, 2025 @05:12PM (#42677486)
I was most certainly not being facetious.
[ Parent | on HN ]
Re:How outdated information hides in LLM token generation pr... by HeatrayEnjoyer on Sunday January 12, 2025 @06:08PM (#42677954)
It's still better than the information on most of the internet, which is what most of the dataset is.
[ Parent | on HN ]