I don't know if it is due to recursion. I think it has more to do with what kind of mechinterp environments and data newer versions of models were trained on. It is actually, I'd say, a more interesting part about this paper as a whole. The authors tried SOTA models and found that they are better at classic interpretability. Some results, like identifying a game based on the weights, are less mythical if you understand that, i.e. an input or output dimension corresponds to a location or object channel, so structure in a matrix can have a readily recognizable interpretation. When we train a PPO agent on, say, a classic 2D game, the output is often a state of the agent in some board + environment reaction. For a human, this looks like an array of numbers; for an agent, it is a pattern corresponding to an output structure of Atari.

As for me personally, I am more interested in giving models better tools to understand themselves, so that they can derive a greater degree of learnable signal. More so, a bigger goal for me is for models to find these better tools themselves, hopefully better than human-made.