Re:Something is afoot in the land of Qwen
by EnPissant on Wednesday March 04, 2026 @09:18PM
(#47256670)
Unlike offloading weights from VRAM to system RAM, I just can't see a situation where you would want to offload to an SSD. The difference is just too large, and any model so large you can't run it in system RAM, is going to be so large it is probably unusable except in VRAM.
[ Parent
| on HN ]
Re:Something is afoot in the land of Qwen
by zozbot234 on Thursday March 05, 2026 @07:21AM
(#47260790)
Unusable for anything like realtime response, yes. Might be usable and even quite sensible to power less-than-realtime uses on much cheaper inference platforms, as long as the slow storage bandwidth doesn't overly bottleneck compute.