If I said that I don't think a black employee should be allowed to have the same voting rights as me (something which was historically the case and was felt to be reasonable by - for a long time - the majority), how much faith would you have in my not activity discriminating against them in a professional environment?
The difference being that to deny a group voting rights, it means you have something against that group specifically.
Whereas for marriage, it could just be that your notion of marriage as a custom (how you think marriage should be) is incompatible with the group performing it.
If you found a minister of a non-specific religion who refused to marry to black two people just because he just didn't see marriage in his faith as something that happened between black people, how much faith would you have that he had nothing against black people, that it was just about he saw marriage?
Religion can have any arbitrary rule. No marriage for X group. No priesthood for Y group. That doesn't necessarily mean that it also sees Y group in a specific light otherwise.
This is part of the problem with it as a religious argument - most religions don't have a unified view of the subject. To bring it back to the matter at hand there are plenty of Christian's who have no problem with gay marriage so is it really a Christian view, or just a view held by some Christians (which is a somewhat different thing)?
Oh, and religion may have any arbitrary rule but that doesn't mean we have to accept it. There is more biblical justification for stoning than for homophobia yet we're not on side with stoning.
Why?