News for hackers. Stuff that matters.

A comment on X updates Terms to allow it to use data for AI training. Parent comment

Re:X updates Terms to allow it to use data for AI training by nextos on Friday September 01, 2023 @01:39PM (#37353937)
Probably HN is already part of The Pile [1]?

I guess X is harder to scrap without permission.

[1] https://pile.eleuther.ai

[ Parent | on HN ]
Re:X updates Terms to allow it to use data for AI training by capableweb on Friday September 01, 2023 @01:45PM (#37354015)
It is indeed:

We introduce new datasets derived from the fol- lowing sources: PubMed Central, ArXiv, GitHub, the FreeLaw Project, Stack Exchange, the US Patent and Trademark Office, PubMed, Ubuntu IRC, HackerNews, YouTube, PhilPapers, and NIH ExPorter. We also introduce OpenWebText2 and BookCorpus2, which are extensions of the original OpenWebText (Gokaslan and Cohen, 2019) and BookCorpus (Zhu et al., 2015; Kobayashi, 2018) datasets, respectively.

From https://arxiv.org/abs/2101.00027 (The Pile: An 800GB Dataset of Diverse Text for Language Modeling)

[ Parent | on HN ]
Re:X updates Terms to allow it to use data for AI training by reaperducer on Friday September 01, 2023 @02:40PM (#37354628)
...HackerNews...

And there's my incentive to stop posting on HN.

It's been a blast, guys. I'm going back to lurker mode.

[ Parent | on HN ]
Re:X updates Terms to allow it to use data for AI training by JumpinJack_Cash on Friday September 01, 2023 @02:03PM (#37354226)
> > Probably HN is already part of The Pile

Everybody smile for the camera, or we could just moon them, or both!

[ Parent | on HN ]