News for hackers. Stuff that matters.

A comment on Run a 1T parameter model on a 32gb Mac by streaming tensors from NVMe. Parent comment

Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by zozbot234 on Tuesday March 24, 2026 @06:11PM (#47510207)
The latter link says they get ~1.7 tok/s which is quite impressive for a near-SOTA local model running on ordinary hardware.
[ Parent | on HN ]

There are no comments on this comment yet.