As someone who runs one of those 3rd-party analytics: the JS you add to your site is small, readable, and can be downloaded and included from your own CDN. Hell, you don't even need the script, you can just write your own (not very hard actually). I took a look at Plausible a while ago, and I think the same applies here as well.

With GA, I can do none of that; it's just a massive unreadable blob I can only load from the GA servers. I just have to trust Google doesn't do anything I don't want (including XSS, but also other issues). It's not even easy to figure out what information exactly it collects last time. It's very untransparent and un-auditable and the website equivalent of loading binary kernel blobs.

The concern over xss is important. But the data you send to google opens your users up to a different type of risk.