Qwen 3.8 27B on my laptop can interpret model weights, too

Re:Qwen 3.8 27B on my laptop can interpret model weights, too by t0mas88 on Tuesday November 03, 2026 @04:58PM (#50358797)

Agree, I run QFN on my AI Pro R9700 (32GB) using https://github.com/Niko1221/Strata It's even faster than on a Strix Halo. Typically I get around 90 tokens/sec with a small context, down to 70 tokens/sec when it's near the maximum context usage.

[ Parent | on HN ]

Re:Qwen 3.8 27B on my laptop can interpret model weights, too by pxmpxm on Tuesday November 03, 2026 @07:20PM (#50360075)

Err strata is just a fork of llama cpp with fancier uis, right?

[ Parent | on HN ]

Re:Qwen 3.8 27B on my laptop can interpret model weights, too by MadameMinty on Tuesday November 03, 2026 @07:57PM (#50360300)

Strata manages MoE expert offloading completely differently, which for QFN means ludicrous speed.

[ Parent | on HN ]

Re:Qwen 3.8 27B on my laptop can interpret model weights, too by mrheosuper on Tuesday November 03, 2026 @09:15PM (#50360782)

I run QFN I2 XS on my 3060ti + 48GB RAM for it. Get around 2x token with 200k Context window. Good enough for me.

[ Parent | on HN ]