News for hackers. Stuff that matters.
Re:Run Kimi K3 using 29 GB of RAM at 0.50 tok/s by throwawayffffas on Saturday August 01, 2026 @02:49AM (#49131783)
You get 8 nvmes set them up in raid 0/1 across two full pcie5x16 ports and you could reach up to 4ish tokens per second, presumably.
Re:Run Kimi K3 using 29 GB of RAM at 0.50 tok/s by withinboredom on Saturday August 01, 2026 @09:53AM (#49134478)
The problem is dram bandwidth to the cpu. Each token costs roughly 20gb of traffic and ddr5 is roughly 50-80gb/s, plus you still have to run the compute sequentially. That’s your limit.