News for hackers. Stuff that matters.

A comment on Everything around LLMs is still magical and wishful thinking. Parent comment

Re:Everything around LLMs is still magical and wishful thinking by ls612 on Saturday July 05, 2025 @12:50AM (#44470149)
How do the best models that can run on say a single 4090 today compare to GPT 3.5?
[ Parent | on HN ]
Re:Everything around LLMs is still magical and wishful thinking by electroglyph on Saturday July 05, 2025 @01:21AM (#44470251)
Qwen 2.5 32B which is an older model at this point clearly outperforms it:

https://llm-stats.com/models/compare/gpt-3.5-turbo-0125-vs-q...

[ Parent | on HN ]
Re:Everything around LLMs is still magical and wishful thinking by ls612 on Saturday July 05, 2025 @01:30AM (#44470287)
Even when quantized down to 4 bits to fit on a 4090?
[ Parent | on HN ]
Re:Everything around LLMs is still magical and wishful thinking by FuckButtons on Saturday July 05, 2025 @06:35PM (#44476135)
Not in my experience, running qwen3:32b is good, but it’s not as coherent or useful as 3.5 at a 4bit quant. But the gap is a lot narrower than llama 70b.
[ Parent | on HN ]