News for hackers. Stuff that matters.
Re:Groq surpasses 1,200 tokens/sec with Llama 3 8B by snhbsqub on Saturday June 08, 2024 @01:24AM (#40615465)
Groq makes zero revenue, needs hundreds on chips to run 1 model, and runs everything at lower precision. SambaNova has a lot of revenue, and runs at that speed at full precision on a single node. It really isn’t a competition.

There are no comments on this comment yet.