Re:Groq runs Mixtral 8x7B-32k with 500 T/s
by tome on Monday February 19, 2024 @12:32PM
(#39432365)
That could be possible. It's out of my area of expertise so I can't say for sure. My understanding was HBM forces on you specific access patterns and non-deterministic delays. Our compiler already deals with many other forms of resource-aware scheduling so it could take into account DRAM refreshes easily, so I feel like there must be something else that makes SRAM more suitable in our case. I'll have to leave that to someone more knowledgeable to explain though ...
[ Parent
| on HN ]
There are no comments on this comment yet.