If anything it demonstrates the limits of NN. A human brain can learn based on far fewer examples.

Nature's evolution algorithm took millions of years to find the architecture and the base model, which then takes decades to be fine tuned to be able to form this opinion.

It's a knowledge distillation. You can then use this smaller, more efficient models instead of the larger one.

Or maybe it is just memorizing a very large number of games.

They address the possibility of memorization in the PDF:

> This effect cannot be explained by memorization since < 1.41% of the initial puzzle board states appear in our training set.

Seems more like a 'compression' of the large number of games, or even like an approximate 'index' of the database

Is this network smaller than stockfish and by what metric is that?

Searched only once. If this can be applied to other knowledge with this efficiency we're onto something