daemonritus

almost all the networks given to astra here are trained on relatively simple single tasks whose geometric/algorithmic structure could be expected to become embedded in surface level weight space geometry, and for a harder task ("which network is sycophantic") where there's less obvious structure to exploit, the method fails. cool but maybe not as paradigm shifting as the authors suggest

by far this is the least math-dense and most intuitive explanation of these ideas that I have seen committed to print anywhere. consider that you might need to do the work of developing improved personal understanding, rather than complaining the expositor hasn't done the work of getting to the conceptual essence. sometimes the conceptual essence is mathematical, cannot be further compressed, and that's okay. I struggle to see how this blog post could possibly be "simplified" beyond defining a few terms more explicitly.