Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

Re:Tongyi DeepResearch – open-source 30B MoE Model that riva... by viraptor on Sunday November 02, 2025 @03:35PM (#45793193)

That's not entirely correct. Most of moe right now are fully balanced, but there is an idea of a domain expert moe where the training benefits fewer switches. https://arxiv.org/abs/2410.07490

[ Parent | on HN ]

Re:Tongyi DeepResearch – open-source 30B MoE Model that riva... by idiotsecant on Monday November 03, 2025 @08:23AM (#45798699)

Yes, explicitly trained experts were a thing for a little while, but not anymore. Yet another application of the Hard Lesson.

[ Parent | on HN ]