Re:Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs
by cwyers on Thursday August 07, 2025 @01:52AM
(#44821050)
So, the way speculative decoding works, the model begins predicting at the first wrong token, so you still get 'is' for free.
[ Parent
| on HN ]
There are no comments on this comment yet.