News for hackers. Stuff that matters.
Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by CamperBob2 on Tuesday February 11, 2025 @04:35PM (#43018668)
Have you compared it to the 1.58 bit dynamic quant model based on the original R1 (i.e., not a distillation)? Whatever unsloth did, it doesn't seem to be giving up much reasoning performance over the full Q8 version.
Re:DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Sc... by mluo on Tuesday February 11, 2025 @05:17PM (#43019082)
It's simply bc the model is small (1.5B), making it sensitive to weight perturbations