Re:AirLLM 70B inference with single 4GB GPU
by Alisaqqt on Tuesday August 04, 2026 @04:52AM
(#49165940)
“Can run” and “is useful interactively” are different benchmarks. At this latency, I can still imagine batch or overnight jobs being interesting; for chat, time to first useful answer matters much more than whether the weights technically fit. A workload/latency/energy table would make projects like this easier to evaluate.
[ Parent
| on HN ]
There are no comments on this comment yet.