News for hackers. Stuff that matters.

A comment on Groq runs Mixtral 8x7B-32k with 500 T/s. Parent comment

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by mlazos on Monday February 19, 2024 @04:38PM (#39435121)
You can’t scale horizontally forever because of communication. I think HBM would provide a lot more flexibility with the number of chips you need.
[ Parent | on HN ]

There are no comments on this comment yet.