Groq runs Mixtral 8x7B-32k with 500 T/s

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by karthityrion on Monday February 19, 2024 @01:36PM (#39433119)

What is the underlying architecture of the ASICs. Does it use systolic arrays?

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by tome on Monday February 19, 2024 @02:14PM (#39433506)

Yes, our matrix engine is quite similar to a systolic array. You can find more details about our architecture in our paper:

[ Parent | on HN ]