We don't have good GPUs that is as fast until 580 (or to some extents, the first Titan). Previous generations only about 2 to 5x faster depending on what types of CPU you compare against.

IMHO, credit should always go to Alex Krizhevsky for the rapid spread of deep learning. He has shown us it was possible. Even without Tensorflow and PyTorch, we will be fine with Caffe, torch, mxnet or Julia.