Interest in neural networks was renewed with Werbos's (1975) backpropagation algorithm. There was continued progress in ANNs all this time.
I think the aversion to ANNs during the 90s was more philosophical and aesthetic - ANNs math is indeed "ugly" compared to symbolic logic, bayesian inference, SVM (in the 00's), and many other traditional AI methods.
https://en.wikipedia.org/wiki/History_of_artificial_neural_n...
https://en.wikipedia.org/wiki/Perceptrons_(book)
applied rigorous math (e.g. when computer science was new) to prove that a certain kind of single-layer neural network couldn't solve certain problems. (Can't learn XOR) It is like proving that it takes N log N comparisons to sort N items.
This dampened interest in neural networks for a long time but the "geometrical thinking in hyperdimensional space" is what the field is all about today.