News for hackers. Stuff that matters.
Re:Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs by cwyers on Thursday August 07, 2025 @01:52AM (#44821050)
So, the way speculative decoding works, the model begins predicting at the first wrong token, so you still get 'is' for free.

There are no comments on this comment yet.