Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Google's Live from Paris Event Private/Deleted Immediately. Parent comment
WebText and WebText2 referenced in their papers are corpuses based on Reddit submissions which had a 22% weight in their training model.
https://openwebtext2.readthedocs.io/en/latest/
This is larger than Wikipedia (3% weight) or either of their two book corpuses (8% each).
The only other data included was a filtered set from Common Crawl (weighted 60%).
---
I was imprecise with my language before but hopefully that at least provides some clarity.
There are no comments on this comment yet.
WebText and WebText2 referenced in their papers are corpuses based on Reddit submissions which had a 22% weight in their training model.
https://openwebtext2.readthedocs.io/en/latest/
This is larger than Wikipedia (3% weight) or either of their two book corpuses (8% each).
The only other data included was a filtered set from Common Crawl (weighted 60%).
---
I was imprecise with my language before but hopefully that at least provides some clarity.