News for hackers. Stuff that matters.
Re:Qwen3-Next by mcrutcher on Friday September 12, 2025 @09:20AM (#45221827)
MoE models are pretty poorly named since all the "experts" are "the same". They're probably better described as "sparse activation" models. MoE implies some sort of "heterogenous experts" that a "thalamus router" is trained to use, but that's not how they work.

There are no comments on this comment yet.