I don't care if I agree with someone. I just want them to provide more information and add some value to the discussion. If the information is invalid, I just do what I can to provide counter resources.
I also generally try to approach downvotes the way you're saying as well, though that brings us back to downvotes being "mini flags". I don't know that I disagree with that.
Mostly just bringing up that upvotes and downvotes aren't really defined. They represent the general unspoken opinion of the people voting, which I think is fine.
I try to ask "does it add information to the conversation?" and upvote or downvote based on that.
If you provide guidelines, you're prescribing the behavior you'd like to see and, it seems, doing more to manually try to carve out a culture.
If you leave the barest guidelines, you allow the culture to define what it means organically.
I don't think either of those is right or wrong. I think prescribing behavior through guidelines is more complex and harder to do well, though, but again, not better.
Of course, that could be totally intentional, and I'm not implying it's a bad way to do things. They might not want to nail down the upvote/downvote idea so much and just let them represent a general will of the readers instead of getting into semantic debates over guidelines. Just a few things I'm curious about.