It's a referendum of the states, as was agreed to by the states when the union was formed.

It doesn't have to be people voting directly. It shouldn't be.