News for hackers. Stuff that matters.

A comment on Run a 1T parameter model on a 32gb Mac by streaming tensors from NVMe. Parent comment

Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by abtinf on Tuesday March 24, 2026 @05:18PM (#47509488)
The lack of a token rate metric for the kimi example is disappointing.
[ Parent | on HN ]
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by zozbot234 on Tuesday March 24, 2026 @06:11PM (#47510207)
The latter link says they get ~1.7 tok/s which is quite impressive for a near-SOTA local model running on ordinary hardware.
[ Parent | on HN ]