I think this comes down to the fact that we can keep a mental ledger on the reputations of 50–100 people, so our in-built reputation system breaks down at the current scale.

You could try building a social credit system to scale things up, but that tends to upset people...