I agree, but at that point if they're going to make this kind of observation on my resume ("hmm, you move around a lot"), the reply is going to be of equal tone ("I move around a lot because the companies I choose tend to make bad decisions and change direction with equal frequency").

There's nothing much to lose by that point.