It means the votes will be more meaningful. Unless the ratio of votes/comments gets really really low it wouldn't be much of an issue I think. This can even be tuned dynamically.