Re:Qwen3-Next
by mcrutcher on Friday September 12, 2025 @09:20AM
(#45221827)
MoE models are pretty poorly named since all the "experts" are "the same". They're probably better described as "sparse activation" models. MoE implies some sort of "heterogenous experts" that a "thalamus router" is trained to use, but that's not how they work.
[ Parent
| on HN ]
There are no comments on this comment yet.