News for hackers. Stuff that matters.

A comment on Run a 1T parameter model on a 32gb Mac by streaming tensors from NVMe. Parent comment

Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by WithinReason on Tuesday March 24, 2026 @12:55PM (#47505694)
Have you ever generated access frequency statistics for the experts in these models, something like a histogram?
[ Parent | on HN ]
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by Gracana on Tuesday March 24, 2026 @01:18PM (#47506058)
ktransformers can do dynamic placement of experts and could presumably produce such a histogram, though currently its activation statistics are just a ".pt" file. https://github.com/kvcache-ai/ktransformers/blob/main/doc/en...

FWIW I never got it to work and did not dig into it much.

[ Parent | on HN ]