Interesting, he thinks "there has not been much progress in pruning NNs in the last 10 years". Maybe he meant that it's often possible to train a non-pruned network of the same size as the pruned network, and achieve a similar accuracy.