News for hackers. Stuff that matters.
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by deepnotderp on Sunday February 25, 2024 @12:55PM (#39503007)
> GPUs Groq will still come out on top because you can never reduce latency by adding more parallel compute :)

You literally can, in fact that’s the entire reason to use multiple chips.

See eg the TPU group’s paper: https://arxiv.org/abs/2211.05102

There are no comments on this comment yet.