News for hackers. Stuff that matters.

A comment on The Era of 1-bit LLMs: ternary parameters for cost-effective computing. Parent comment

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by cjbprime on Wednesday February 28, 2024 @04:15PM (#39543531)
You need all of the model in RAM to perform the matmult that gets you the next token from it. There's no shortcut.
[ Parent | on HN ]

There are no comments on this comment yet.