Re:DeepSeek-Prover-V2
by woah on Wednesday April 30, 2025 @06:46PM
(#43851547)
Why do you think that a hand-configured selection between "different domains" is better than the training-based approach in MoE?
[ Parent
| on HN ]
Re:DeepSeek-Prover-V2
by oofbaroomf on Wednesday April 30, 2025 @11:26PM
(#43853403)
First off, they are basically completely different technologies, so it would be disingenuous to act like it's an apples-to-apples comparison.
But a simple way to see it is that when you pick between multiple large models that have different strengths, you have a larger amount of parameters just to work with (e.g. Deepseek R1 + V3 + Qwen + LLaMA ends up being 2 trillion total parameters to pick from), whereas "picking" the experts in an MoE like has a smaller amount of total different parameters you are working with (e.g. R1 is 671 billion, Qwen is 235).