Re:Serving 70B-Scale LLMs Efficiently on Low-Resource Edge D...
by tgtweak on Thursday October 03, 2024 @12:38PM
(#41732349)
I read it like this too - no drop in weights or model quality just optimizing the lower boundaries of performance when you are splitting from vram to ram to disk (or network).
[ Parent
| on HN ]
There are no comments on this comment yet.