You've never seen the popups? If a WebExtension doesn't request access to all sites, it can only access sites that it's activated on by pressing the toolbar button (MDN refers to them as action buttons if you want to take a look). The problem is that most extensions that aren't site-specific are useful because they have access to all sites, but that doesn't mean that's a requirement of the system.