It also allows you to match an e-mail address to a particular account. So, let's say a dating site uses gravatars, now anyone who can see those can match my dating profile to my stackoverflow account (and any other site using gravatars).

I know this ofcourse, and will use a different e-mail address if I need to keep these identities seperate, but many people do not realize any gravatar enabled user profile can be linked to any other gravatar enabled user profile with only a tiny bit of effort in harvesting.

Uh, can't they also link those accounts by noticing that they are the exact same picture? I mean, I thought that was partially the point.

The problem is they can match these profiles even if you haven't signed up for gravatar at all. Your email hash is always sent to gravatar.com.