News for hackers. Stuff that matters.

A comment on Qwen 3.8 27B on my laptop can interpret model weights, too. Parent comment

Re:Qwen 3.8 27B on my laptop can interpret model weights, too by bityard on Tuesday November 03, 2026 @03:48PM (#50357968)
Which quant of QFN are you running on your Strix Halo?

HF is telling me that IQ4-XS is the largest one likely to fit without much contortion.

[ Parent | on HN ]
Re:Qwen 3.8 27B on my laptop can interpret model weights, too by UncleOxidant on Tuesday November 03, 2026 @07:01PM (#50359928)
That's the one I'm running using a fork of the gufo inference engine as gufo upstream doesn't support that quant.
[ Parent | on HN ]
Re:Qwen 3.8 27B on my laptop can interpret model weights, too by Zetaphor on Tuesday November 03, 2026 @07:10PM (#50359991)
I'm running a regular Q4, what about that quant format justifies the fork?
[ Parent | on HN ]