Re:Self-hosting Kimi K3: 20% more hardware cost, 20% better ...
by lcampbell on Wednesday July 29, 2026 @02:40PM
(#49101331)
> Aggregate token throughput is about 30% lower (122 vs 170 tok/s at 16 users)
Is that actually the aggregate throughput? 8xB300 (with 4TB/s/GPU bandwidth) is only pushing 8 tg/s/session? That seems… incredibly low, even for an A100B model.
Is it actually 122 tg/s per session? (1952 tg/s aggregate throughout)?
Is that actually the aggregate throughput? 8xB300 (with 4TB/s/GPU bandwidth) is only pushing 8 tg/s/session? That seems… incredibly low, even for an A100B model.
Is it actually 122 tg/s per session? (1952 tg/s aggregate throughout)?