If you argue with that kind of unwritten agreement, I'd argue that ad providers should also adhere to the agreement not to track me, manipulate me or to infect my machine with malware. Which they evidently don't. So I don't feel very motivate to respect theirs.
As just one example: If you've agreed to meet with a friend and they don't show, without notice, you'll probably inform them that they are in violation of your unwritten agreement.
(I'm not actually in favour of this line of argument, but I believe it's pretty obvious that unwritten social norms exist, and often serve a purpose)