I agree. I think it's fundamentally a flaw of the voting system: the idea that previous users should be able to anonymously and indirectly influence the visibility of a message or post to future users.