Astra can read and write weights to improve Qwen

Re:Astra can read and write weights to improve Qwen by fancyfredbot on Monday November 02, 2026 @05:35PM (#50344996)

For the tiny models with a few thousand weights, it seems plausible Astra can actually understand what the network detects and figure out it's e.g. a triangle detector. This is possibly helped by the existence of many breakout playing networks on the web.

For Qwen, even the smallest versions like 3.5-0.8B it's becoming less plausible that it could be interpreted in a forward pass, even with reasoning. I'm wondering if the result here is recalled from many examples of instruction tuned versions of this model.

A smaller Gemma model doesn't fare as well but unclear whether this is due to fewer users and less weight examples or because it's already very well trained.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by vessenes on Monday November 02, 2026 @05:39PM (#50345053)

The Gemma task is harder: add features. Qwen was: fix this looping behavior

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by fancyfredbot on Monday November 02, 2026 @05:49PM (#50345141)

The task was the same - improve math reasoning capabilities by editing the weights. But yes, I agree that in Qwen it could do this through reducing looping behaviour, but Gemma wasn't prone to looping and it didn't find anything to help, possibly because there wasn't much room for improvement.

[ Parent | on HN ]