In mathematics my impression is that citing arXiv papers is usually done appropriately. If a surprising proof of a longstanding open problem was recently posted on arXiv, people won't usually cite it as authoritative right away. Instead a citation will be worded with something like "a recent claimed proof by so-and-so...", making it clear that it's still preliminary and not to be treated as a sure thing yet (how strongly to add this caveat is a bit of a judgment call).
As a reviewer that's usually what I look for. If someone cites an arXiv paper (or tech report, or even blog post) to give context, to credit other teams who are doing related work, to point out alternative approaches that the current paper isn't investigating, etc., that's perfectly fine. Those are more like informational "see also" citations. If a paper cites an arXiv preprint as authoritative evidence of something that is really critical to the current paper itself being correct, then yes, that's where it can be more iffy and I'd ask the authors to either qualify that claim more, or bolster it with another more solid citation. There is a little bit of a problem with this currently in machine learning, where some people make overly bold/general claims in arXiv preprints that their methodology/data can't really support, and other people sometimes cite these claims a bit too credulously. Important to push back on, but not to the extent of banning arXiv citations.
Not to mention I know examples of respected journals in the field letting total crap through (not opinion, total misapplication of statistical theorems) so the journal stamp of approval isn't everything.
Arxiv is not ideal, but I refuse to think going back to journals as gatekeeper is a step forwards. That would be a step backwards. Some sort of distributed social network peer review of arxiv papers, and having points scored there that count as the review for for papers for funding bodies etc would probably be best. The journal model is outdated and charging too much money to provide little value (I mean, they just coordinate the review process, a piece of software could do the same coordination).