Re:Run Kimi K3 using 29 GB of RAM at 0.50 tok/s
by throwawayffffas on Saturday August 01, 2026 @02:49AM
(#49131783)
You get 8 nvmes set them up in raid 0/1 across two full pcie5x16 ports and you could reach up to 4ish tokens per second, presumably.
[ Parent
| on HN ]
Re:Run Kimi K3 using 29 GB of RAM at 0.50 tok/s
by withinboredom on Saturday August 01, 2026 @09:53AM
(#49134478)
The problem is dram bandwidth to the cpu. Each token costs roughly 20gb of traffic and ddr5 is roughly 50-80gb/s, plus you still have to run the compute sequentially. That’s your limit.