News for hackers. Stuff that matters.
Re:Show HN: Can I run this LLM? (locally) by throwaway314155 on Saturday March 08, 2025 @07:58PM (#43305129)
It still wouldn't fit in 16 GB memory. Further there's too much swapping going on with MoE models to move expert layers to and from gpu without bottlenecks.

There are no comments on this comment yet.