Is there a statistical demonstration of the effectiveness of established journals? It seems like you would have to do some form of counterfactual study to prove/disprove this. As far as I know, the argument of the effectiveness of journals only stems from the survivorship bias (e.g. all things considered research until recently has been published by journals).

No, you can look at the stuff people put up on the preprints to see what would happen if there's basically no screening at all.

The preprint model has actually been wildly successful for many communities (https://arxiv.org/, https://www.biorxiv.org/). I don't think anyone is arguing against peer review, but the idea of peer review and an established journal is being conflated. If anything, publishing established journals can create a level of implied authority that reduces much need scrutiny.