Real-time LLM Inference on Standard GPUs: 3k tokens/s per request

Re:Real-time LLM Inference on Standard GPUs: 3k tokens/s per... by gaeld on Friday May 29, 2026 @07:00AM (#48321550)

Thanks. To be fair, this number is what we expect to get once we port DeepSeek V4 in our engine on the upcoming generation of GPUs!

[ Parent | on HN ]

There are no comments on this comment yet.