News for hackers. Stuff that matters.

A comment on Qwen3.8-Max: A New Bar for Coding and Cowork. Parent comment

Re:Qwen3.8-Max: A New Bar for Coding and Cowork by sznio on Monday August 03, 2026 @03:25AM (#49152339)
quantized + offload

I have an RX 6700 XT with 12gb vram and 64gb system ram. running dense models like 27b is difficult, but i can run IQ4/IQ5 qwen 122b-a10b or 35b-a3b at ~20tok/s

[ Parent | on HN ]
Re:Qwen3.8-Max: A New Bar for Coding and Cowork by TonyStr on Monday August 03, 2026 @07:12AM (#49154197)
IQ4 qwen 122b-a10b would mean 61GB total size and 5GB active, so about 5GB of the model loaded into GPURAM plus any generated context, and 61GB of weights loaded into system RAM? I don't know if that math is correct, but does that run well? Wouldn't that only leave 3GB of system RAM?
[ Parent | on HN ]