News for hackers. Stuff that matters.

A comment on DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL. Parent comment

Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by mluo on Tuesday February 11, 2025 @04:27PM (#43018586)
Hi, one of the lead authors for this work.

We recommend using Bfloat16 (not fp16), quantization for small models can really hurt performance!

[ Parent | on HN ]
Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by simonw on Tuesday February 11, 2025 @04:36PM (#43018677)
Is there a GGUF version of your model anywhere that you recommend? I'm on a Mac.
[ Parent | on HN ]
Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by mluo on Tuesday February 11, 2025 @05:16PM (#43019074)
Think there are some people who made GGUFs as branches of our model, try it out!

https://huggingface.co/models?other=base_model:quantized:age...

[ Parent | on HN ]
Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by CamperBob2 on Tuesday February 11, 2025 @04:35PM (#43018668)
Have you compared it to the 1.58 bit dynamic quant model based on the original R1 (i.e., not a distillation)? Whatever unsloth did, it doesn't seem to be giving up much reasoning performance over the full Q8 version.
[ Parent | on HN ]
Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by mluo on Tuesday February 11, 2025 @05:17PM (#43019082)
It's simply bc the model is small (1.5B), making it sensitive to weight perturbations
[ Parent | on HN ]
Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by newman314 on Wednesday February 12, 2025 @03:01PM (#43029139)
Is there a MLX version that can be added to the fullmoon iOS app?
[ Parent | on HN ]