Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update
by whatifitoldyou on Saturday November 14, 2026 @01:58PM
(#50499743)
I will add an average time per task collumn when I get the chance. I feel tokens per second is a very fluid metric though as it varies wildly by hardware, inference engine and configuration as well as MTP config. If you can fit these models in VRAM (aka you have at least 24gigs of VRAM) you should be able to get at least 50/60 tps from the slower dense models and easily more than 100 tps from the MOE models. But it depends on lots of factors.
[ Parent
| on HN ]
There are no comments on this comment yet.