News for hackers. Stuff that matters.

A comment on Groq runs Mixtral 8x7B-32k with 500 T/s. Parent comment

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by frognumber on Monday February 19, 2024 @04:51PM (#39435269)
Don't blame you. Been at plenty of startups, resources are finite, and focus is important.

My only point was to, well, perhaps bump this up from #100 on your personal priority list perhaps to #87, to the limited extent that influences your business.

[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by matanyal on Monday February 19, 2024 @06:05PM (#39435896)
Groq Engineer here as well; we actually built our compiler to compile pytorch, TensorFlow, and Onnx natively, so a lot of the amazing work being done by y'all isn't building much of a moat. We got LLama2 working on our hardware in just a couple of days!
[ Parent | on HN ]