Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

Re:Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac wit... by mulemisterX on Tuesday September 01, 2026 @03:50PM (#49527160)

I have a 48GB M5. I don't need to run larger models. I want more context. I've managed to set the context window at 71,680 using Qwen3.8-27B-oQ4e-fp16-mtp. But I want more. Is anybody, with similar specs, able to set their context window higher?

[ Parent | on HN ]

Re:Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac wit... by ig0r0 on Tuesday September 01, 2026 @04:48PM (#49528004)

yes, with qwen3.8-27b-4bit run via rapid-mlx i can get to about 200k

[ Parent | on HN ]

Re:Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac wit... by metadat on Tuesday September 01, 2026 @05:03PM (#49528166)

Nice.. and how many toks/sec?

[ Parent | on HN ]

Re:Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac wit... by ig0r0 on Wednesday September 02, 2026 @01:57AM (#49532273)

Around 20

[ Parent | on HN ]

Re:Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac wit... by pram on Tuesday September 01, 2026 @05:51PM (#49528749)

You should try Glimmer MTP. Qwen3.8 27B seems to have weird memory and caching issues on oMLX

[ Parent | on HN ]

Re:Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac wit... by hadlock on Tuesday September 01, 2026 @07:23PM (#49529684)

We are running 35b-A3b with 264k context (the model's default max) using vllm and the "frog" jinja templates: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates and had good luck. We are mostly running agentic workloads though, rather than coding. 27b has a slightly higher agentic job completion rate (95% vs 92%) but the 3% trade off is worth it because the A3B is sooooo much faster, and we reprocess the other jobs with a different model. Don't sleep on the froggeric templates.

Qwen: Looking at you for a new ~35B MoE! Please and thank you

[ Parent | on HN ]

Re:Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac wit... by kamranjon on Tuesday September 01, 2026 @08:13PM (#49530124)

I am running 3.8 27b at q6 quant with 160k context on a 32gb video card (arc b70 pro) - I quantized the kv cache at q8 - that is the only trick really - works great.

[ Parent | on HN ]

Re:Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac wit... by jonplackett on Thursday September 03, 2026 @07:42PM (#49558655)

How are you finding it when it gets that high anyway? I’ve got a 64gb Mac so the context _can_ go higher. But I find around 70-80k it goes a bit weird and starts arguing with itself and doesn’t seem to know who it is vs me anymore.

[ Parent | on HN ]