Astra can read and write weights to improve Qwen

Re:Astra can read and write weights to improve Qwen by vineyardmike on Monday November 02, 2026 @02:31PM (#50342332)

This does feel big, but the trillion-dollar question: is this trained by OpenAI and can other LLMs do it.

It’s no secret that these big labs see “recursive self improvement” as a primary goal. It seems natural that the models would be explicitly trained to excel at all sorts of ML related tasks as a result. Is this one of those tasks, or is this emergent?

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by jrflo on Monday November 02, 2026 @03:05PM (#50342881)

This seems emergent to me. LLMs are not trained directly on how to read weights, because humans do not understand raw weights, so there is no training data. They're trained on how to use the tools we have like writing python, examining training output and iterating, not writing whole weights from scratch.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by paxy on Monday November 02, 2026 @03:28PM (#50343230)

Humans don’t need to understand the data for models to train on it.

A large enough number of examples of “here are the model weights -> here’s what it does” will achieve this.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by throwaway219450 on Monday November 02, 2026 @03:14PM (#50343015)

In a self-supervised setting you don't need ground truth data, as long as you have a proxy task that you can train on which forces the model to learn something useful. Like cutting out bits of images and telling the model to reconstruct the gaps (i.e. MAEs).

There are also techniques to watermark model weights to confirm what training data were used: https://arxiv.org/html/2512.17075 I'm not an expert in this domain, but it seems like there are lots of pieces that could be used to (a) train a model to infer whether some data were trained on and (b) train a model to understand which weights are relevant for a particular training task in a more explainable way than backprop/gradient descent.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by yorwba on Monday November 02, 2026 @03:16PM (#50343052)

In cases where Astra succeeded, it succeeded using standard techniques also used by humans when they read and write neural network weights. It's a rare skill, mostly used for fun (e.g. https://blog.janestreet.com/can-you-reverse-engineer-our-neu... ) and basic research rather than profit, but Astra is big enough to have learned many rare skills.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by adamsb6 on Monday November 02, 2026 @03:14PM (#50343018)

There is a body of research on mechanistic interpretatibility that probably helped inform this behavior.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by user43928 on Monday November 02, 2026 @02:52PM (#50342680)

I tried it in the Claude web version with Opus 5.5 Medium and it also identified Breakout, even though in one of its explanations it seems to have mixed up some indexes, but the overall reading of the weights was right.

That's also matching what I see when I work with Opus 5.5 to optimize models. It seems to understand what individual layers are for, and can thus sometimes suggest optimizations, where I as a layman would be completely clueless.

The latest models are performing amazingly. Because of this, I disagree with popular comments saying that eg. DeepSeek is good enough, AI-assisted development was where the real value was at, etc.

We see huge progress at the frontier, while the open competition is around GPT-5.5's level, and that model was released in April. Let's see how things develop in the future.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by senordevnyc on Monday November 02, 2026 @04:36PM (#50344150)

The latest models are performing amazingly. Because of this, I disagree with popular comments saying that eg. DeepSeek is good enough, AI-assisted development was where the real value was at, etc.

Completely agree. I often think that those sorts of comments grow out of an understandable lack of imagination. We’ve never had tools like this, and we’ve never had tools that evolved so rapidly, so we easily can get stuck thinking that once a model is good enough to do today’s tasks, that’s really all you need. But the next generation of models often completely obviates the need for those tasks entirely. The software engineers of 2031 won’t be doing tasks anything like the software engineers of today, just like we aren’t doing things like we did in 2021.

(Yes, yes, I know some of you still hand write all your code and think LLMs are today’s crypto bullshit and this is all going to blow over. Not talking to you.)

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by StilesCrisis on Monday November 02, 2026 @08:13PM (#50346719)

All the "competition" is just distilling the frontier, so it makes sense that they are perpetually behind by months.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by caaqil on Monday November 02, 2026 @02:53PM (#50342700)

> It seems natural that the models would be explicitly trained to excel at all sorts of ML related tasks as a result. Is this one of those tasks, or is this emergent?

Maybe there's some general capability, call it g, that can explain all the underlying capabilities and improvements, including on ML tasks? Huh, I am getting a bit of déjà vu here.

[ Parent | on HN ]