Groq runs Mixtral 8x7B-32k with 500 T/s

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by tome on Monday February 19, 2024 @02:53PM (#39433955)

Have a look at section 2.3 of our paper. Between any two chips we get 100 Gbps. The overall bandwidth depends on the connection topology used. I don't know if we make that public.