News for hackers. Stuff that matters.
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by littlestymaar on Thursday April 16, 2026 @11:02AM (#47794190)
That's not how it works. Many people get confused by the “expert” naming, when in reality the key part of the original name “sparse mixture of experts” is sparse.

Experts are just chunks of each layers MLP that are only partially activated by each token, there are thousands of “experts” in such a model (for Qwen3-30BA3, it was 48 layers x 128 “experts” per layer with only 8 active at each token)

There are no comments on this comment yet.