Re:Neural Network Diffusion
by killerstorm on Thursday February 22, 2024 @03:10AM
(#39464498)
There's a principle more powerful than the bitter lesson: GIGO.
Training to predict internet dump can only give you so much.
There's a paper called something like "learning from textbooks" where they show that a small model trained on high-quality no-nonsense dataset can beat a much bigger model at a task like Python coding.
Training to predict internet dump can only give you so much.
There's a paper called something like "learning from textbooks" where they show that a small model trained on high-quality no-nonsense dataset can beat a much bigger model at a task like Python coding.