The argument in favor of Google Fonts and similar services is that visitors likely already have that font cached from visiting another site that uses the same font. It can also decrease bandwidth costs if that's a bottleneck.
That said, serving from your own domain is a great way to make sure that a third party can't break your fonts, in addition to what you mentioned about privacy.
Re:The Fastest Google Fonts
by csswizardry on Friday May 29, 2020 @02:49AM
(#23346955)
OP here. Cross-domain caching is a myth[1] so reusing font files won’t work. In fact, it’s literally impossible in Safari[2] and soon to be made impossible in Chrome[3].
1. https://csswizardry.com/2019/05/self-host-your-static-assets... 2. https://andydavies.me/blog/2018/09/06/safari-caching-and-3rd... 3. https://bugs.chromium.org/p/chromium/issues/detail?id=910708
different fonts give sites different personalities, adding richness to the web (whether you appreciate it or not). for 99.9% of sites, being overly focused on font size/download speed is an unnecessarily trivial optimiztion.
with that said, don't remotely load google fonts because privacy does matter. for most sites, the speed and bandwidth difference doesn't matter. just serve them as first-party, cacheable assets.