News for hackers. Stuff that matters.
Re:iPhone 17 Pro Demonstrated Running a 400B LLM by zozbot234 on Monday March 23, 2026 @03:33PM (#47494086)
A model "where expert switches are less necessary" is hard to tell apart from a model that just has fewer total experts. I'm not sure whether that will be a good approach. "How often to switch" also depends on how much excess RAM has been available in the system to keep layers opportunistically cached from the previous token(s). There's no one-size fits all decision.

There are no comments on this comment yet.