Or just remove the link from the list view. (Only show the flag link when viewing a particular story.)

(I've also done that by mistake like four times.)