News for hackers. Stuff that matters.
Re:Wikidata, with 12B facts, can ground LLMs to improve thei... by gaogao on Friday November 17, 2023 @11:32AM (#38305831)
Training on bot data can be bad when it's ungrounded and basically hallucinations on hallucinations.

Having LLMs help curate something grounded is generally reasonable. Functionally, it's somewhat similar to how some training is using generated subtitles of videos for training video/text pairs; it's very feasible to also go and clean those up, even though it is bot data.

There are no comments on this comment yet.