News for hackers. Stuff that matters.

A comment on Llama 3.1 405B now runs at 969 tokens/s on Cerebras Inference. Parent comment

Re:Llama 3.1 405B now runs at 969 tokens/s on Cerebras Infer... by ryao on Wednesday November 20, 2024 @12:24AM (#42191042)
They claim it is with between 8 and 20 users:

https://x.com/draecomino/status/1858998347090325846

That said, they appear to be giving the per user performance.

[ Parent | on HN ]

There are no comments on this comment yet.