Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Serving 70B-Scale LLMs Efficiently on Low-Resource Edge Devices [pdf]. Parent comment
For vanilla models, you always use all the weights. That isn't true for mixture-of-experts, though, and in that setting, your approach has merit.
There are no comments on this comment yet.
For vanilla models, you always use all the weights. That isn't true for mixture-of-experts, though, and in that setting, your approach has merit.