There’s something here…

It’s editing behaviour more efficiently than a psychiatrist, and is in ways acting as a brain surgeon but brain surgeons don’t focus on behaviour and thought loops…

Depends on the brain surgery... a lobotomy at the extreme end. Seizures could be thought of as a loop of firing neurons too.

It's a programmer that can debug, the model weights are part of the source code. The whole thing is just a program in a weird language and language models are good at translating across all sorts of formats. That's what they do when writing SVGs, generating 3D models or even directly outputting 3D models.

And they can write tools that help debug better than demonstrated in that blog post.

In my interpretability tinkering, Fable was the first to be able to write a program that can transpile from a lispy syntax directly to model weights of small neural nets, including transformers. Yesterday, I pointed out in a post that for many kinds of small AIs, you can avoid ML and have the LLM directly write the small neural net, compact decision tree, AD Tree or bayesian network. Since it's "hand"-written, it can be much more efficient and capable than training one of a similar size.