> researchers shouldn't be distributing them with the intent to cause harm

It could be used to cause harm and publishing your work is distributing it (by definition). Similar laws have been (and are) used to target people for unjust reasons.

Defacing street signs is already illegal.

Making various abstract drawings illegal to wear on your clothing (or print on a sticker) is a terrible idea. If the algorithm used by a product can't handle a pedestrian wearing a sweatshirt with an adversarial example on it then that product simply isn't suitable for public use.

Look at drug research. There is plenty of red tape that hinders it. Although, here, the "harm to society" is defined by the nation state.

However, I agree with your proposals in the top-level comment.