Astra can read and write weights to improve Qwen

Re:Astra can read and write weights to improve Qwen by shwaj on Monday November 02, 2026 @02:32PM (#50342350)

I don't see how that helps. It would still be one expert model chosen to operate on another expert model of (let's say for this discussion) approximately the same size.

The parent's point still stands, I think. Even if the multi-trillion param model is MOE, each expert is still (assumedly) larger than the model being optimized.

It would be interesting to see future work that characterizes the relative model sizes (and considers MOE vs not-MOE) to see what relative scales this result holds across.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by dyauspitr on Monday November 02, 2026 @07:58PM (#50346579)

> It would still be one expert model chosen to operate on another expert model of (let's say for this discussion) approximately the same size.

This would be earth shattering. That would mean we’re essentially at RSI.

[ Parent | on HN ]