I'm sure a lot of people also don't realize that Google Chrome has had, for many years, an extension API for accessing USB devices. Has it been a problem? No.
This is just another communication protocol. It's reasonable for browsers to provide a strictly controlled environment to utilize it.
Google, Facebook, and many other technology companies, have conditioned users to click 'Yes' and 'Accept All' and 'Share' such that we can no longer use such prompts to block malicious actors from our computer systems.
I do not trust software engineers to make the correct choice between "ooh a cool feature" and the general well being and privacy of their users. There has been too much precedent and incentivization for the former, including massive profit and promotional tracks.