News for hackers. Stuff that matters.
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ... by coolspot on Wednesday July 23, 2025 @01:32PM (#44661793)
> That would be enough to support a single user. If you want to host a service that provides this to 10k users in parallel your cost per user scales linearly with the GPU costs you posted.

No. Magic of batching allows you to handle multiple user requests in parallel using the same weights with little VRAM overhead per user.

There are no comments on this comment yet.