Given the way that humans have handled one group separating into another, I think this might be a devastating experiment to try. But in theory, it seems plausible that after a number of iterations you would belong to a group that fits your ideals perfectly.
The point of that argument is that no individual has the right to do with the government what he would not be right to do by himself. This means that since he can't individually limit the rights of another, he can't use the government to do it either.
This means that there are no concessions at all to personal freedom to live in a group. Not really anyway. I mean, yes, you can't take someone else's life (unless they attempt to take yours.) But that's hardly a restriction necessary only in large groups of people. It's kind-of implied as soon as you have at least two people.
Tyranny implies government. In the absence of government, therefore, there can be no tyranny. It's not clear to most people whether society without government is possible, of course (or, rather, most would assert that it isn't).
The same "we" that has existed since the beginning of our species, when groups of humans decided to live together.
"We" is an expression of all people - not biased by East or West, Black or White. In all societies the price of living in a group, and enjoying the benefits thereof, are concessions to personal freedom, and the recognition of some authority that is capable of making the decision.
It's also important to note (because it's so frequently confused) that "we" is not necessarily a government. In the absence of a government, "we" will manifest itself in other formats, most frequently tyrannical.
The only way for you to extricate yourself from this (by your implication, malevolent) "we" is to live entirely without dependency on any semblance of society.