It's very possible we never break the factoring problem. Since we have no properly unified theory of physics (e.g., one that models the observer in QFT), there are outstanding unknowns with quantum computing that may make it so that Shor's algorithm is not practically useful.