That's still a piece of information though, especially if the same user has actively down-voted other items.

There is no useful information gained if people never touch the system. Clearly non-downvoting means something for people who periodically downvote. But the way I plan to use the service is ... not at all.

I see, I miss-read what you wrote. I had thought we were still talking about "ardent down-voters" Yes, in order for them to glean any information from a particular user there will have to be some clicks. And indeed, the most common way that a user uses this system might very well be "not at all".