There's a large literature on applying language models to reasoning tasks, but very few on what's actually going on inside them. But https://transformer-circuits.pub/ shows some attempts to do so. Still at a very early stage (see in particular "In-context Learning and Induction Heads").
There's a large literature on applying language models to reasoning tasks, but very few on what's actually going on inside them. But https://transformer-circuits.pub/ shows some attempts to do so. Still at a very early stage (see in particular "In-context Learning and Induction Heads").