True, but neural networks existed for a long time. The hardware is what made NNs actually practical. Recent theoretical advances have been only incremental. For example, CNNs are just evolutionary since image recognition has used convolutions long before NNs were practical, and CNNs are just a straightforward translation of existing techniques to NNs.

Sure theoretical advances may be evolutionary, but they resulted in exponential reduction of parameter space and sample complexity. These advances outpace Moore's law by a large margin.

Hardware served as a catalyst, but it was not a necessity.