Re:Groq CEO: 'We No Longer Sell Hardware'
by germanjoey on Sunday April 07, 2024 @08:18PM
(#39965094)
cost effective in what sense? groq doesn't achieve high efficiency, only low latency. but that's not done in a cost-effective way. compare sambanova achieving the same performance with 8 chips instead of 568, and with higher precision.
[ Parent
| on HN ]
Re:Groq CEO: 'We No Longer Sell Hardware'
by halflings on Tuesday April 09, 2024 @05:09PM
(#39984278)
The # of chips is not the most important metric.
Most important, even ignoring latency, is throughput (tokens) per $$$. And according to their own benchmark [1] (famous last words :)), they're quite cost efficient.
[1] https://www.semianalysis.com/p/groq-inference-tokenomics-spe...