> I haven't wrapped my head around 5 and 6

This sentence doesn't align very well with your other comment: https://news.ycombinator.com/item?id=50350548

> In Experiment 5, we ask Astra for networks that play three MinAtar games, and in Experiment 6, for a network that plays Go.

5 and 6 aren't surprising, merely impressive, a direct extrapolation of human abilityies. I've written transformer weights by hand to implement a simple shortest-path graph search. It takes hours, so an LLM that's >100x faster than me obviously can build super complex networks. It's just like creating any intricate system by hand such as writing machine code or designing computational complexity widgets.

On the other hand what is impressive is the remarkably few thinking tokens Astra needed to do things like analyse networks. That is well beyond human capability: experiments have shown that Astra can perform a huge number of arithmetic calculations per single output token, double Fable or any other. Its brain is wired for mathematics.

If Astra uses some sort of looped transformer architecture (and if Fable doesn't), then that could explain why Astra can do more computation per token. It is essentially simulating a much deeper model.

But yea, I was also quite surprised by the low token usage.