News for hackers. Stuff that matters.
Re:Groqchat by pclmulqdq on Saturday December 23, 2023 @12:12AM (#38741809)
They are putting the whole LLM into SRAM across multiple computing chips, IIRC. That is a very expensive way to go about serving a model, but should give pretty great speed at low batch size.

There are no comments on this comment yet.