So? They aren't performing the same computation. You can't compare the two. What you can compare is power draw at an equivalent tokens/sec on the same model. But you don't have that number.
I’m just estimating here from public numbers. My point is the power consumption could be 100W on that workload and the groq chip could be 1k, both ridiculously optimistic. The whole system is still crazy expensive. H100s will not have latency as fast, but in terms of concurrent users and TCO, I really don’t think groq will be worth it. You could probably get the same concurrent users and throughput with like 8 h100s. Latency won’t be as good, but price could be much lower.
That said I think their arch is super interesting. I just think that demo was way too hype when the actual system is pretty impractical.