Religious concerns were only a minor factor. Other objections to having a national identifier include: wasteful "big government" spending, encroachment on states' rights, potential for civil liberties abuses, and lack of consensus over whether it's a real problem that needs to be solved.

I don't necessarily agree with those objections but there have been multiple reasons.