News for hackers. Stuff that matters.
Re:Running Qwen3.8 at 255 tok/s on a single 5090 by Ir0n on Tuesday October 13, 2026 @07:13PM (#50080273)
we achieved this on stock nvfp4 model which is within 5% of fp16 version

we then apply lossless techniques only

dflash2 as spec drafter and patched ninfer inference engine (down projections for short token lengths and adding tool call related schema to drafter shortlist)

see: https://github.com/meshapplied/goethe/pull/4

benchmark numbers are measured using see bench verified traces at avg input 28k and concurrency 1

There are no comments on this comment yet.