News for hackers. Stuff that matters.

A comment on Groq runs Mixtral 8x7B-32k with 500 T/s. Parent comment

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by amirhirsch on Monday February 19, 2024 @04:44PM (#39435181)
It seems like you are making general purpose chips to run many models. Are we at a stage where we can consider taping out inference networks directly propagating the weights as constants in the RTL design?

Are chips and models obsoleted on roughly the same timelines?

[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by tome on Monday February 19, 2024 @04:47PM (#39435222)
I think the models change far too quickly for that to be viable. A chip has to last several years. Currently we're seeing groundbreaking models released every few months.
[ Parent | on HN ]