Could you point to concrete papers, showing that empirically?

(Many things were rediscovered over and over, sure.)

> random steps in high-dimensional spaces unless you align with a ‘large’ eigenvector of the Hessian

In this case, the main observation is different than this one, and has much more to do with sparsity (and an exponentially growing number of connections with each layer).