You care because "just stop talking altogether" and staring dumbly at the cop doesn't end the confrontation, unless he then leaves (and if you've tried for a while to get a definitive answer, you might as well quit asking for a while). If you get his OK, then he's not detaining you and you can leave.