News for hackers. Stuff that matters.

A comment on MonadGPT – What would have happened if ChatGPT was invented in the 17th century?. Parent comment

Re:MonadGPT – What would have happened if ChatGPT was invent... by thomasahle on Sunday November 26, 2023 @10:42PM (#38427784)
How many stories/tokens did they actually train on? I can't find it in the paper.
[ Parent | on HN ]
Re:MonadGPT – What would have happened if ChatGPT was invent... by yyyk on Monday November 27, 2023 @04:12AM (#38430019)
Maybe this will help?

https://huggingface.co/datasets/roneneldan/TinyStories

Also note

https://arxiv.org/abs/2309.05463 (which is larger - obviously size still does contribute to performance)

[ Parent | on HN ]