News for hackers. Stuff that matters.

A comment on Groq runs Mixtral 8x7B-32k with 500 T/s.

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by supercharger9 on Monday February 19, 2024 @03:19PM (#39434307)
Ignoring latency but not throughput, How does this compare in terms of Cost ( cards Acquisition cost and Power needed) with Nvidia GPU for inference?
[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by matanyal on Monday February 19, 2024 @06:19PM (#39436026)
We intend to be very competitive on cost, power, hardware, TCO, whatever it is. Custom-built silicon+hardware has the advantage in this space.
[ Parent | on HN ]