It's a possibility that it'll be a great threat in the near future. Remember that Babbage's Analytical Engine was designed on paper in 1837, and nothing similar was constructed for over a century.

If it's not the near future, we might move to post-quantum algorithms (or quantum cryptography, a field almost entirely unrelated to classical cryptography which involves carefully moving entangled pairs of particles around) before quantum computers become any good at cryptanalyzing realistically-sized encryption algorithms.