News for hackers. Stuff that matters.

A comment on Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20. Parent comment

Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by CamperBob2 on Tuesday May 28, 2024 @01:15PM (#40503003)
Someone will come along and say "Why don't you just mirror Anna's Archive?" in 3...2...1...
[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by sebzim4500 on Wednesday May 29, 2024 @05:32AM (#40510100)
I think between Anna's Archive, fineweb and as many github repos as you can scrape you can get a pretty decent dataset.

I doubt Anna's Archive would produce a good model on its own though.

[ Parent | on HN ]