If you use a modern browser you probably don't need the polyfills anyway, so why are you concerned about blocking them?

While not necessarily related to privacy, I've found sites where blocking self-served JS creates a better browsing experience because it gets rid of annoying effects and things like scrolljacking.