> If there is a very low-dimensional manifold in parameter space that accounts for all the predictive power, that would be a very easy thing to diagnose from a neural network model.

People wouldn't waste time researching neural network visualization tools, coming up with adversarial examples and developing alternative (more explainable) machine learning models if that was the case.