Re:GLM-5.3-Flash
by colingauvin on Wednesday August 26, 2026 @10:24AM
(#49449816)
That's 160GB-ish for Q4...how is 256 insufficient?
Looks like the M5 Ultra Studio wait times are going to increase again. Already at 10-12 weeks, I wonder how long it'll go?
Re:GLM-5.3-Flash
by speedgoose on Wednesday August 26, 2026 @10:45AM
(#49450131)
I guess like the M3 Ultra, at some point normal customers won’t be able to buy it.
That M3 had an older type of RAM. Apple hopefully secured sufficient supply of the newer variant for the M5 Ultra.
Both use LPDDR5x, they're not shipping LPDDR6 (yet).
All M3 variants use LPDDR5.
Re:GLM-5.3-Flash
by yonatan8070 on Wednesday August 26, 2026 @03:19PM
(#49454380)
Speaking as someone who isn't really well versed in this, does 18B active parameters mean that you could potentially hold only the 18B parameters in RAM and stream the rest from a fast NVMe SSD for acceptable performance similar to how Colibri works?
Sorta, but you're off by one layer. You can store the 18B in VRAM and stream the rest from RAM. There's still a performance hit relative to storing it all in VRAM, but it's tolerable.
Generally, for local consumer use, these large MOE models are best for unified RAM systems like DGX Spark or Mac Studio.
Normal MoE is switch-weights-per-token so you would se substantial slowdowns that way. Apple did a More that switches weights per prompt (instruction-following pruning, https://arxiv.org/abs/2501.02086) but you have to design the model that way which I don't think the have.
This is pretty hefty for a "flash" model, even a 256 GB setup is insufficient at q4 - and q4 is already the worst-but-still-acceptable quant in my experience. The benchmarks look great, especially since GLM tends to be more honest than the average Chinese lab, but you’ll need to splurge to run it at home.
@edit: so many releases that I forgot to math. This fits just fine in q4, realistically the minimal hardware would be 192gb - so blazing fast on double rtx 6000 pro and usable on 256gb unified memory. You could even go with 5bit quant on 256gb.
… you’ll still need to splurge, though.