News for hackers. Stuff that matters.

A comment on Why DeepSeek is cheap at scale but expensive to run locally. Parent comment

Re:Why DeepSeek is cheap at scale but expensive to run locally by jimmySixDOF on Sunday June 01, 2025 @04:32PM (#44153574)
Groq and Cerebras both take a big chip approach to architecture and, at least in the case of Groq, they only make economic sense under high batch loads.

https://x.com/swyx/status/1760065636410274162?s=46

[ Parent | on HN ]
Re:Why DeepSeek is cheap at scale but expensive to run locally by ryao on Sunday June 01, 2025 @11:37PM (#44155644)
There is nothing big about Groq’s chips. Their individual chips have only 230 MB RAM. Unlike Cerebras, which can load multiple layers into a single chip, grok must divide a layer across many chips.
[ Parent | on HN ]