Deep Learning training is usually I/O bound these days. Definitely on GPU side, I believe the same thing on CPUs. Not clear if many people are using CPUs for the DL training, though. CPUs are used mostly for the inference, when response time is not critical and models are large (with large RAM requirement).