News for hackers. Stuff that matters.

A comment on Real-time LLM Inference on Standard GPUs: 3k tokens/s per request. Parent comment

Re:Real-time LLM Inference on Standard GPUs: 3k tokens/s per... by yjftsjthsd-h on Friday May 29, 2026 @09:14AM (#48322704)
The GPU in my desktop. (A normal-ish decent gaming machine that runs LLMs and txt2img well enough.)

In contrast, not enterprise GPUs that cost as much as a car.

[ Parent | on HN ]

There are no comments on this comment yet.