GPUs were necessary but not sufficient, true. But the real change (at least for supervised learning) was not in the polishing on training methods (an incremental improvement), it was the availability of big enough data sets. No amount of modeling can make up for insufficient data.