Do people then have any rights in the US? Because conscription is one of the powers the Constitution explicitly grants to the Congress.

The historical background is that serving in the military is what able-bodied male citizens do in a republic. Conscription is not supposed to be something that the government does but what the citizens do by the virtue of being citizens. If there is no conscription, you don't have a republic in the traditional sense, because warfare is no longer a public matter.