Yes, you just nailed it. It’s the cognitive dissonance. “That doesn’t make any sense, huh” leads to getting annoyed when the world seems to disagree with something that you just can’t figure out any other way.