News for hackers. Stuff that matters.
Re:Qwen3-Next by EnPissant on Saturday September 13, 2025 @01:37AM (#45229614)
gpt-oss-120b chooses 4 experts per token and combines them.

I don't know how lmstudio works. I only know the fundamentals. There is not way it's sending experts to the GPU per token. Also, the CPU doesn't have much work to do. It's mostly waiting on memory.

Re:Qwen3-Next by furyofantares on Saturday September 13, 2025 @02:57AM (#45229938)
> There is not way it's sending experts to the GPU per token.

Right, it seems like either experts are stable across sequential tokens fairly often, or there's more than 4 experts in memory and it's stable within the in-memory experts for sequential tokens fairly often, like the poster said.