News for hackers. Stuff that matters.
Re:Serving 70B-Scale LLMs Efficiently on Low-Resource Edge D... by tgtweak on Thursday October 03, 2024 @12:38PM (#41732349)
I read it like this too - no drop in weights or model quality just optimizing the lower boundaries of performance when you are splitting from vram to ram to disk (or network).

There are no comments on this comment yet.