News for hackers. Stuff that matters.
Re:Reddit Is OpenAI’s Moat? by dontreact on Wednesday June 14, 2023 @10:16AM (#36326338)
One of the big values of reddit is that it serves as an easy index for non-garbage websites on the internet. This is exactly how it was used in the T5 paper: they threw away all the websites from their crawl that were not linked to from Reddit.
Re:Reddit Is OpenAI’s Moat? by dontreact on Wednesday June 14, 2023 @10:17AM (#36326349)
And by garbage I mean literally data/illegible/etc. that would ruin the pretraining of the model.