Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on The Coming of Local LLMs. Parent comment
That's for the 7B model. The 30B model needs 24GB quantized (or 64GB for the unquantized model).
There are no comments on this comment yet.
That's for the 7B model. The 30B model needs 24GB quantized (or 64GB for the unquantized model).