I think an interesting UX would be to put voting in the post. So you have to go to the comments to upvote/downvote.

That kind of delay encourages discussion and probably helps with the psychological negatives of infinite scroll