News for hackers. Stuff that matters.

A comment on Groq runs Mixtral 8x7B-32k with 500 T/s. Parent comment

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by SilverBirch on Monday February 19, 2024 @04:44PM (#39435184)
Surely once you're scaling over multiple chips/servers/racks you're dealing with retries and checksums and sequence numbers anyway? How do you get around the non-determinacy of networking beyond just hoping that you don't see any errors?
[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by tome on Monday February 19, 2024 @04:54PM (#39435291)
Our interconnect between chips is also deterministic! You can read more about our interconnect, synchronisation, and error correction in our paper.

https://wow.groq.com/wp-content/uploads/2023/05/GroqISCAPape...

[ Parent | on HN ]