Are terms of use legally binding? Do the Twitter guidelines (don't rotate the bird etc.) count as terms of use?

What is the worst that could happen here? The organisation behind the Olympics sending a letter to a site owner requesting they remove the link? And what would happen if they refused?