Re:Show HN: Can I run this LLM? (locally)
by throwaway314155 on Saturday March 08, 2025 @07:58PM
(#43305129)
It still wouldn't fit in 16 GB memory. Further there's too much swapping going on with MoE models to move expert layers to and from gpu without bottlenecks.
[ Parent
| on HN ]
There are no comments on this comment yet.