News for hackers. Stuff that matters.
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by tatef on Tuesday March 24, 2026 @02:25PM (#47507006)
Yes, definitely agree. It's more of a POC than a functional use case. However, for many smaller MoE models this method can actually be useful and capable of achieving multiple tokens/sec.

There are no comments on this comment yet.