Incredible results. How can we know that the modifications were not existing on the internet?

"Now let's see if Astra can write a neural network from scratch in zero-shot."

This strongly suggests retrieving pre-trained weights from memory, no?