I think it is cool too, but with a caveat. For the large model (Qwen), their Astra used a Python environment, activations info, and iterative feedback to find an edit, it is not the same as looking at just pure weight numbers.

And in small MLPs they used, the I/O representations are kind of made to be more understandable because they explicitly encode objects and coordinates for games. If you set it up like that you can see some patterns yourself in the first/last layers without Astra. It is very different from interpreting distributed, entangled computations inside a large model. So I think the paper shows promising capabilities in weight-space analysis and editing, but not a general understanding of large models from their weights alone.

I'd love to see us reach that point, I am for one working in this area, but I don't think these experiments establish it.