Astra can read and write weights to improve Qwen

Re:Astra can read and write weights to improve Qwen by Wheen on Monday November 02, 2026 @02:15PM (#50342060)

It's worth pointing out that this is a multi-trillion (probably) parameter model optimizing an 800 million parameter model. There is no indication that it can improve itself with this same methodology.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by dontknowmuch on Monday November 02, 2026 @02:22PM (#50342196)

If that multi-trillion model is composed of thousands of MOE experts, each expert model might be small enough for it to comprehend.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by shwaj on Monday November 02, 2026 @02:32PM (#50342350)

I don't see how that helps. It would still be one expert model chosen to operate on another expert model of (let's say for this discussion) approximately the same size.

The parent's point still stands, I think. Even if the multi-trillion param model is MOE, each expert is still (assumedly) larger than the model being optimized.

It would be interesting to see future work that characterizes the relative model sizes (and considers MOE vs not-MOE) to see what relative scales this result holds across.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by dyauspitr on Monday November 02, 2026 @07:58PM (#50346579)

> It would still be one expert model chosen to operate on another expert model of (let's say for this discussion) approximately the same size.

This would be earth shattering. That would mean we’re essentially at RSI.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by kzrdude on Monday November 02, 2026 @05:44PM (#50345103)

I think "experts" are a misnomer because it's not that 1 expert = 1 component, like it makes it sound. "Experts" in different layers have no relation.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by elif on Monday November 02, 2026 @09:30PM (#50347245)

I think the indication is that this proprietary and secret machine is extremely capable at reading and writing in weights.

Occams razor would say that it was not only built for this purpose, but possibly multiple generations of good at this.

A year ago LLMs seemed to hit a wall, and then all the sudden they started accelerating as quickly as ever.

No one can legally confirm or deny that it's happening... But it sure feels like it's happening.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by furyofantares on Monday November 02, 2026 @09:57PM (#50347459)

People have been saying LLMs have hit a wall for like 4 years. People still say it today. I don't think it's been true at any point.

[ Parent | on HN ]