WeirderScience

arXiv link for the paper: https://arxiv.org/pdf/2506.08257

(Note that the author Lukas Beyer is distinct from the other well-known author in the field, Lucas Beyer)

Their sketchy ad/malware era aside, I do appreciate sourceforge keeping all the old OSS websites and repos alive.

If you find this kind of stuff interesting, I'd strongly recommend checking out the book "Spark: The Life of Electricity and the Electricity of Life" by Jorgensen

Looking forward to Casey Muratori's talk!

I wonder if this is a result of the previously reported clashes between OpenAI and Microsoft over access to the Windsurf IP (under their investment agreement)

Yeah, I suspect you're right. Still, even a list of URLs for a frontier model (assuming it does turn out to be of that level) would be welcome over the current situation.

The open training data is a huge differentiator. Is this the first truly open dataset of this scale? Prior efforts like The Pile were valuable, but had limitations. Curious to see how reproducible the training is.