Attention models learn what to pay attention to.

It's been found that data that begin with "Wikipedia:" are automatically weighted higher by language models during training, completely unsupervised.

But this is the same problem - Wikipedia is a secondary source and should always get overruled by the primary source.

And there are documented cases of concentrated efforts to manipulate the content of Wikipedia.

You’re probably being facetious but just in case:

https://www.bbc.com/news/technology-28481876.amp

https://www.bbc.com/news/technology-58559412.amp

(With apologies for amp links)

I was most certainly not being facetious.

It's still better than the information on most of the internet, which is what most of the dataset is.

Same way as we can: learning which sources are more trustworthy.

There's limits to how far you can go with this — not only do humans make mistakes with this, but even in the abstract theoretical it can never be perfect: https://en.wikipedia.org/wiki/Münchhausen_trilemma — but it is still the "how".

for the last 25+ years we rather not learned, but trusted the top3 of SERPs. Every ranking algorithm will be gamed eventually

I would say that we learned to trust the search engines; but otherwise I agree with you: every ranking algorithm will be gamed eventually.

(I wonder if giving an LLM content with intent to cause its users to spend money they didn't need to, would count as fraud, hacking, both, something else entirely?)

I’m not sure what kind of response you’re looking for, or if this is a rhetorical question or not. But “how could a language model infer…?” can be asked about a whole lot of things that language models have no problem reliably inferring.

> that language models have no problem reliably inferring

... the article did give me a different impression.

I don’t think you read my comment correctly.