Re:Astra can read and write weights to improve Qwen
by disgruntledphd2 on Tuesday November 03, 2026 @05:27AM
(#50350201)
It's possible that Astra may have been trained on this kind of work, whereas Claude hasn't been.
Also possible that it's leakage (which honestly does seem like the most likely explanation).
As a reminder, if you see incredible performance from any statistical model, it's most likely leakage.
I had Claude train three language classifiers for short words in four languages. The network was just over 10k weights with a single hidden layer.
I uploaded 6 language word lists and made it train networks to fit random subsets of 4 languages. The languages were picked randomly. Claude could always identify which 4 languages were chosen and in which order. It told me that the language heads were easy to link to letter frequency and each language had distinctive makers. So claim upheld, an LLM can interpret small networks.
I then asked Claude to improve the classification accuracy by proposing an edit to the weights in the above networks. It's proposed improvement reduced classification accuracy significantly. The same thing happened if I gave it a deliberately undertrained network with clear room for improvement.