Re:Serving 70B-Scale LLMs Efficiently on Low-Resource Edge D...
by wpietri on Thursday October 03, 2024 @01:09PM
(#41732656)
I think the point is that it has the same sort of latency tradeoff that disk swap did: it's awful, but sometimes better than nothing.
[ Parent
| on HN ]
There are no comments on this comment yet.