News for hackers. Stuff that matters.
Re:>10x More Efficient Pretraining by ismael_rr on Thursday September 10, 2026 @11:52AM (#49645831)
Super awesome. Wish they would release the paper about what they did to achieve this. I remember nous released the token superposition paper which improved pretraining FLOPs some, but not 50x: https://nousresearch.com/token-superposition. Wondering if they also found some cool tokenization strategiesa
Re:>10x More Efficient Pretraining by pixl97 on Thursday September 10, 2026 @01:31PM (#49647433)
"Write a paper" < "Sell to a big AI lab for $$$"

Going to be interesting to see what happens to discoveries like this in the future.