Both models (FOSS/Startups) are somewhat vulnerable to those who are the loudest winning by sheer volume. In both cases the BDFL/CEO can intervene but it wastes time/energy when it's needed most.
I am sure in a flat company, compensation is probably the toughest part as it can easily turn into a popularity/tribal thing possibly unless there is some set ranges and agreed upon structure. I wonder in a flat company if compensation needs to be more open.
The difficulty lies in quantifying experience - there loads of ways to pull that off (and none of them quite wrong). Years of experience, type of hackery, FOSS contributions, etc. I'd say the best option would be to pick one and stick with it. If you're properly open about it, those who take issue with your system won't apply.
In OSS, everybody has the same access to the source code. That's the power from the definition, and the OSS license is the explicit rule.
Because of the explicit rule, if someone goes power hungry on one extreme, the rule guarantees that he will be ousted (via forking the source), and other extreme, if someone is not interested in politics (like someone actually working on the code), they won't get screwed, because they will always have the source code access according to the rule.
People in the past tried it, naively, without the explicit power rule. That was the MIT license. But they got screwed (divided and conquered) by people who wanted power, who wanted to close the source. That's why copyleft (i.e. democracy) was invented. (This actually happened on the metagame level - now the BSD type licenses mostly work because people always can fall back to GPL.)
Linus has Linus. Python, Guido. Ruby, Matz. And so on.
Other projects a BDFL can emerge or the original creator is polled for direction but largely they are there to keep trust in the platform.
Here's some really cool code_swarm looks at Guido and his long journey as the sole fighter for Python, bring it to the masses alone for 5+ years: https://vimeo.com/1093745 This video gives a unique look at the meritocracy team behind Python and how they contributed over time, fading in and out.
See all of them here: http://vis.cs.ucdavis.edu/~ogawa/codeswarm/ has Eclipse, Python, Apache, PostgreSQL.
I agree that open source is awesome but successful projects are not at all 'flat' structures. Just think about all the governance processes that are necessary once a project grows beyond a certain size. These allow a project to achieve some kind of longevity. Simply thinking about who has commit access should indicate that there is clearly some form of hierarchy (at least, some form of 'in-crowd').
If you think that the 'naturally' occurring hierarchies work better, then I'd posit that you've only been in the 'in-crowd' for those projects -- of course it's difficult to see the downside from that vantage point.
There will always be hierarchies, but more naturally occurring ones seem to evolve better and work better for the ones involved in the doing/building/engineering part.
Meritocracy yes, but also entirely based on contribution and delivery which is actually quite efficient. No location required in OSS either, it is remote and akin to freedom one might say. Are there problems? Sure, but that is where you fork and diverge, may the best path win.