News for hackers. Stuff that matters.

A comment on Groq runs Mixtral 8x7B-32k with 500 T/s. Parent comment

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by WiSaGaN on Monday February 19, 2024 @10:47AM (#39431020)
That seems to be per card instead of chip. I would expect it has multiple chips on a single card.
[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by renewiltord on Monday February 19, 2024 @10:52AM (#39431099)
From the description that doesn't seem to be the case, but I don't know this product well

> Accelerator Cards GroqCard low latency AI/ML Inference PCIe accelerator card with single GroqChip

[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by WiSaGaN on Monday February 19, 2024 @10:54AM (#39431121)
Missed that! Thanks for pointing out!
[ Parent | on HN ]