News for hackers. Stuff that matters.
Re:Qwen 3.8 27B is excellent, but it defaults to overthinkin... by rao-v on Monday August 17, 2026 @01:20PM (#49334481)
Even Muse Glimmer (as did GPT-OSS I think) does ~4 sliding window attention layers + 1 full attention layer (like Gemma 4). I’m assuming both labs have good reason to think that gated delta nets are not optimal.

Of course it’s possible the labs just stick with the optimal architecture for large models and GDN is best for smaller models.

Re:Qwen 3.8 27B is excellent, but it defaults to overthinkin... by dofm on Monday August 17, 2026 @01:51PM (#49334929)
Thanks for the reply. Sooo much I have to learn.