That's a really good point about the click/no-click. That said, I could see myself down-voting ads for which there's no chance I'd ever click, leaving ads I might be interested in at some point, but didn't actually want to click at that moment.

Also, there's the type of ad where you think you might be interested in it, so you click it to see what it's about, only to find that the product or whatever actually sucks. That's where the down-vote would help.

I don't know if this idea will work, though. Assuming it's used by web surfers, then I think advertisers will want it. But that's a big assumption. What's the incentive for web surfers to use it? Looking at ads, even "high-quality" ones, isn't really a priority of mine. And there are other tools like StumbleUpon or del.icio.us that suggest content to me, but don't really have the commercial agenda that an ad does.