Astra can read and write weights to improve Qwen

Re:Astra can read and write weights to improve Qwen by supern0va on Monday November 02, 2026 @03:11PM (#50342980)

> It is still very surprising though, considering it apparently displays an "intuitive" understanding of the weights, that make whatever Goodfire did look like primitive monkeys poking at it by comparison.

FYI, Nick/Goodfire used the models extensively in their work, so it's very possible that there were some similarities in their work. I'm really curious to see his response to this.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by krackers on Monday November 02, 2026 @03:58PM (#50343641)

I think this work does vindicate their hypothesis that alignment/superalignment might actually turn out to be "easy", in that you can just train LLMs do to the interpretability work. Maybe this is in fact what OpenAI did, or if it's purely "emergent" then it seems they have great priors that can be made more powerful with some RL

[ Parent | on HN ]