I suppose there can be some other layer of reviewer meta-review to account for this, much like the role 'acceptance rate' has come to have for journals and conferences. But, following Goodhart's law, even that has come to be gamed now that a meaning - a proxy for journal prestige or quality - has been placed on it.
I quite like the fact that whatever someone says on the internet now, the first comment is usually the best-argued counterpoint. It seems like a dynamic that could be brought to science quite well. I may not have the time to parse an entire paper in a field unfamiliar to me, but if I come across it and the top comment is 'there's a math error in table 6 that throws paragraph 4 into doubt', I can certainly verify that for myself. That sort of 'peer' review would make science much more accessible to the interested layperson.
Of course, in practice, this has the problem that it is already hard to get reviewers, because nobody has time anymore. Removing the anonymity of reviewers leads to the judge becoming open to judgement themselves, and many would not like that. Furthermore, there might be severe repercussions if the reviewed is more powerful than the reviewer, especially in authoritarian societies. But maybe one should just exclude these societies from peer review, and let them do their own thing.
I think peer review has to become a market place. Let everyone choose for themselves which paper they would like to review, and there is both positive and negative credit for both reviewers and authors. The pool of reviewers of a paper shares 25% of the credit that the authors of the paper get. This way you are incentivised to review even outlier papers, because if they are successful you get a substantial amount of credit for them. How the credit is distributed among the reviewers should also incorporate a time factor, so that reviewers rushing to a paper that is already successful don't get nearly as much credit as a lone early reviewer.