This is the primary concern and barrier to most realizations of quantum computers including Google’s most recent efforts, though there are exceptions like linear optical quantum computing which have different problems. So for people working in QC this is in fact the perspective, though they are mostly optimistic that we can continue to engineer/discover better solutions to this problem.