I mean, it depends a little bit on what your threat model is. If it takes a week to break a key, and you have hundreds of thousands of tls sessions without knowing which is the relavent one, it is definitely something. But yeah it seems like it would quickly become a minor hurdle once real quantum computers become a thing and presumably have their own moore's law.