News for hackers. Stuff that matters.

A comment on DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL. Parent comment

Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by Palmik on Tuesday February 11, 2025 @04:17PM (#43018490)
The negative quality impact of quantization is more pronounced for smaller models [0], so I'm surprised this tiny quant works at all.

[0] or rather models closer to saturation, which is a function of model params and amount of training

[ Parent | on HN ]
Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by simonw on Tuesday February 11, 2025 @04:18PM (#43018506)
Yeah I'm very impressed that the tiniest quant version appears partially usable... especially now I've replicated that same S, T, R, A, W, B, E, R, F, U, R, Y bug on a much larger one: https://news.ycombinator.com/item?id=43018494
[ Parent | on HN ]