Re:Running Qwen3.8 at 255 tok/s on a single 5090
by MozerJ on Tuesday October 13, 2026 @06:56PM
(#50080116)
That is true, the lack of good benchmarks will be with us for a while I think. Ternary is not a good quant algo, but because it's so drastic, it allows people to use enormous models on worse hardware, or just increase throughput speed. In the meanwhile, there's research going into how this quantisation can be executed while correcting for the loss. In this case we're not just quantising but also post training the model, and we can "trade" some capabilities to others (ex. post-train the experts for math but lose on biology or whatever).
With that said, I find nvfp4 to be far superior. I think right now all you can do is try out the models, maybe run your own benchmarks if you have.
With that said, I find nvfp4 to be far superior. I think right now all you can do is try out the models, maybe run your own benchmarks if you have.