Scale is an interesting point here. Large overparametrized networks don't behave in the same way as mall toy models, this is one of the big caveats of many interp works. I expect naked weight edit will not work for large models. Why? Because an agents needs to know what it edits and reason on it. You could theoretically improve a model on some form of weight recognition, maybe it will work maybe it won't. But again, we come up against a cost/benefit dilemma, it sounds exotic and sci-fi but likely impractical. Parameters of model are connected and relate to each other. Instead, current SOTA auto-interp argues that what we need are better tools for models to know themselves, so rather than direct edit we need to give models better understanding of large parameter geometries, then they can work at scale. This is much harder to do than applying a steering vector.