The mistake you're making is treating states as primary. States--and, more generally, federalism--are a means to an end. The purpose of government is to serve the interests of people. Rhode Island is a legal fiction delimited by imaginary lines; you and I are real people. The people who live within the imaginary lines of Rhode Island are just as important as anyone, but those million people are no more or less important than any other arbitrarily selected collection of a million people. Why should it matter where they choose to live, or where the imaginary lines are drawn?

In the United States, states are largely a historical accident stemming from the way North America was colonized. No one actually decided to establish a federal democracy in North America and divide it into states, different people from entirely different countries just happened to colonize different parts of the continent without ever intending that New Amsterdam and Jamestown would ever fall under the same American government. The Constitution was a political compromise based on the fact that thirteen of those colonies broke away from the mother country and needed a federal government. The Founding Fathers weren't gods, they were the same breed of hypocritical, self-serving politicians you see in any era, a scarce few wiser than the others for sure, but the same ugly sausage-making all the same.