You can disable 3rd party fonts.

On Firefox, this is as simple as going to about:config and setting gfx.downloadable_fonts.enabled to false.

On Chrome, it requires adding the flag --disable-remote-fonts to however you're opening it.

This breaks a lot of sites that use custom fonts as compressed replacements for monochrome SVGs.

Those sites are already broken for accessibility.

The "closest available system font" is determined by the font names in a site's stylesheets, not by magic.

There is no way that a browser could figure out what the "closest available" system font is, or that one even exists. It's up to the designer to specify the font(s) to be used.