Astra can read and write weights to improve Qwen

Re:Astra can read and write weights to improve Qwen by augment_me on Monday November 02, 2026 @02:38PM (#50342460)

The math example on Qwen was based on 50 batched iterations, and the fix was essentially to remove a direction in activation space that causes looping instead of an exit, this was already known to be a thing.

So it's not able to insert new knowledge, but it's able to adjust the routing/looping/weighing of existing one.

As I understand, it can replace post-training like LoRA/GRPO, but if it'd cheaper the authors don't go into

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by ex-aws-dude on Monday November 02, 2026 @03:01PM (#50342819)

I don't think its that crazy that you could look at activations and tweak weights directly to get a certain outcome, but isn't the question whether you can do that without side effects or degrading/breaking other parts of the model?

If I want to do that in a generalizable way that preserves performance across all domains then am I not just back to training again?

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by augment_me on Monday November 02, 2026 @03:42PM (#50343421)

Yep youre right, I don't see an eval on this.

I think you could still essentially have a really cheap Lora-Post training on a frozen base model by doing this. And instead of creating 200 checkpoints for different post-trained directions just adjust a single one on the fly.

[ Parent | on HN ]