If you work in the field you know the 10-100 people who provide most expertise and you can build trust relationships with them by delivering interesting thoughts and then get their feedback on These ideas. That's the healthy approach. And if you work hard and the stuff you do is reasonable you will also become known in these circles.
Also technology can help weed out the complete b.s., see HN or Stackoverflow for examples on that.
It’s just set up formally so you don’t need to ask everyone individually and there is (an attempt at) the feedback being blind so that people who are unpopular don’t get penalised.
That's not how I have ever seen peer review done by a reputable conference or journal. Why would they be reviewers if they didn't know the topic?
What do you think the word 'peer' means?
In my field the people I'd ask informally for comment are exactly the same people who would peer review my paper.