Re:Everything around LLMs is still magical and wishful thinking
by ls612 on Saturday July 05, 2025 @01:30AM
(#44470287)
Even when quantized down to 4 bits to fit on a 4090?
[ Parent
| on HN ]
Re:Everything around LLMs is still magical and wishful thinking
by FuckButtons on Saturday July 05, 2025 @06:35PM
(#44476135)
Not in my experience, running qwen3:32b is good, but it’s not as coherent or useful as 3.5 at a 4bit quant. But the gap is a lot narrower than llama 70b.