Re:Qwen 3.6 27B is the sweet spot for local development
by bigyabai on Monday June 29, 2026 @08:25PM
(#48727090)
> When it comes to inference speed, you want your model to fit in memory, and then to have as much memory bandwidth as possible.
This is only true when your GPU isn't bottlenecked building a KV cache, which it usually will be on Apple Silicon. The Achilles heel of the M-series chips are their weak, SOC-grade GPU that holds back the Max and Ultra models from having interactive TTFTs on larger models and contexts.
Re:Qwen 3.6 27B is the sweet spot for local development
by fancyfredbot on Tuesday June 30, 2026 @10:32AM
(#48733247)
Normally people refer to the compute-bound phase as "prefill". Nothing wrong with saying it's building the kv cache though, it's accurate just unusual.
Re:Qwen 3.6 27B is the sweet spot for local development
by iagooar on Tuesday June 30, 2026 @03:18AM
(#48729504)
On paper the M4 should be roughly 1/3 of the M5, in practice it is only 1/2. With the right, optimized model like qwen3.6 35B MoE MLX you can get over 40 tok / sec on it. I run dozens of background jobs that are not time-critical on it.
Re:Qwen 3.6 27B is the sweet spot for local development
by bfjvibybd6cuvu6 on Tuesday June 30, 2026 @05:53AM
(#48730486)
What kind of jobs?
Re:Qwen 3.6 27B is the sweet spot for local development
by iagooar on Wednesday July 01, 2026 @03:18AM
(#48743309)
Scanning mailbox, reading and classifying emails. Scanning a knowledge base, reading and improving individual articles, reading support interactions and creating summaries, checking and researching new leads that signed up. So much that is possible.
614 GB/s of memory bandwidth
> MacMini M4 with 64GB of RAM
273 GB/s of memory bandwidth (also only currently available with 48GB)
When it comes to inference speed, you want your model to fit in memory, and then to have as much memory bandwidth as possible. In this case a hypothetical Mini with 1TB of memory would still be over 2x slower with 27-35B models.
And FWIW I have an M4 Max MBP 128GB that I keep on a Roost laptop stand, with a separate keyboard/mouse/video. It does fire up the cooling jets when running local LLMs, but stays within tolerance for me on noise. I haven't heat-tested it on longer runs, but I imagine the risen airflow helps a ton.