You need to first be blessed by the Holy Spirit, aka Paul Graham. Otherwise, you might even see the buttons, but your votes have no effect because you're an Outcast.

you need a certain karma level to be able to downvote

edit: it used to be 100, I dont know if its still the current threshold

Ah, ok thanks. Is there a way I can see unread replies to my comments without having to manually go to all the comments I've seen?

Threads link in the header shows you all the comments you have made and replies to them.

Thanks.

I just got downvoting at around 200 karma.

I think it is more complicated than that. I see for example posters I can down vote and other posters I can not.

This probably isn't the full picture, but you can't vote down comments left directly in response to your own comments (for obvious reasons), nor can you vote down comments over a certain age or that are already of a very low score.