Maybe that is how it should be, but that isn't how it is right now.

If you don't want public recognition of your marriage, I'm sure you could write up some private contracts, but it won't confer all of the same rights that a state-recognized marriage gets.