From a computational complexity standpoint, it's not even clear if quantum computing is actually faster. The discrete log problem and integer factoring problem aren't known to be NP-complete, so even assuming P != NP it's possible there are polynomial time algorithms for factoring and discrete log on classical computers as well. BQP vs. P is still, IIRC, unresolved.

Though, from a practical standpoint, I have a feeling that, regardless of how the computational classes shake out, the quantum Fourier transform will prove to be faster than classical techniques. And possibly Grover's algorithm, though its results are somewhat less dramatic.