My understanding is this is why it's a "Jury of your peers".

Someone from outside the community is an alien and would convict differently than someone who is more integrated with the scenario. Not only is the understanding of the situation different, but the incentives are different.

The scale of society has changed, but imagine what this was like when communities were smaller.