Re:Real-time LLM Inference on Standard GPUs: 3k tokens/s per...
by wmf on Friday May 29, 2026 @01:47PM
(#48326680)
scenarios where both the model and the hardware are fixed and self-contained
That's basically antirez's DS4 and it works pretty well because there are few leading models and few hardware platforms (Apple, GB10, Strix Halo) that are worth using.
That's basically antirez's DS4 and it works pretty well because there are few leading models and few hardware platforms (Apple, GB10, Strix Halo) that are worth using.