Astra can read and write weights to improve Qwen

Re:Astra can read and write weights to improve Qwen by kristopolous on Monday November 02, 2026 @09:28PM (#50347229)

so let's assume a completely naive approach. modify any weight arbitrarily, run an eval if eval improves arbitrarily modify another weight and repeat ... how does the naive base case perform on N=20 random trials?

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by cowlevel on Monday November 02, 2026 @10:30PM (#50347699)

What you're describing is a shitty version of gradient descent, which is the technique used to train LLMs and other neural networks. Proper gradient descent simultaneously calculates how much every weight improves the output, and edits them all simultaneously in proportion to that.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by kristopolous on Tuesday November 03, 2026 @02:19AM (#50349041)

obviously. unless you have the equivalent of a coin toss, no metric carries implication. Does naive noise give you nothing? The null hypothesis needs to be established.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by Otterly99 on Tuesday November 03, 2026 @08:41AM (#50351872)

That's exactly what I was thinking. Actually, if you look at the "Is Qwen improving" 25 and 10 forward passes, it doesn't seem to improve much.

Additionally, since it relies on using GSM8K and MATH datasets: how does it compare in terms of compute with other training methods?

[ Parent | on HN ]