News for hackers. Stuff that matters.
Re:DeepSeek-R1 by cbo100 on Tuesday January 21, 2025 @01:01AM (#42777061)
I get the right answer on the 8B model too.

It could be the quantized version failing?

Re:DeepSeek-R1 by ein0p on Tuesday January 21, 2025 @02:40PM (#42784370)
My models are both 4 bit. But yeah, that could be - small models are much worse at tolerating quantization. That's why people use LoRA to recover the accuracy somewhat even if they don't need domain adaptation.