News for hackers. Stuff that matters.

A comment on Qwen3.8-27B at 256K on a 24GB RTX PRO 4000 SFF (432 GB/s): 50 tok/s with MTP.

Re:Qwen3.8-27B at 256K on a 24GB RTX PRO 4000 SFF (432 GB/s)... by Tepix on Monday August 17, 2026 @12:18PM (#49333475)
Always put the quantisation in the title!
[ Parent | on HN ]
Re:Qwen3.8-27B at 256K on a 24GB RTX PRO 4000 SFF (432 GB/s)... by pich on Monday August 17, 2026 @12:26PM (#49333641)
Its not quite that simple here. The iMatrix-guided hybrid uses different quantization levels per tensor/layer, so there isnt one honest Q4/Q5/NVFP4 label I can put in the title
[ Parent | on HN ]