I remember having lunch with a speech recognition researcher who was using GPUs to train DNNs to do speech recognition in 2011. It really was thought of as niche back then. But the writing was on the wall I guess in the results they were getting.