Modern web apps make a mind-boggling number of cross-origin requests, and many of them are to multi-tenant SaaS providers.

You'd be constantly flooded with permission popups, and attackers would just host part of their code on one ot more of the popular domains that everyone had gotten used to clicking "allow" for.