The problem comes if you install a browser extension (or acquire malware that installs a browser extension for you), forget about it, and then blame the website when the website stops working, often filing bug reports (increasing the support load of the website).
Just look at hacker news. Many posts have someone complaining how the site is broken only to find out they have disabled scripts or have a night mode extension, or disabled webgl but forgot about it, or they programed their router to drop all packets that were going to a url that included "google" anywhere in it (including mysite.com/google-blog-post, true story)
Longer answer: I install any such extension with full knowledge of their potential to cause breakage, and consequently my first reaction to a broken page is to issue a reload with the extension disabled.
The web being programmatically accessible and end-user-malleable is a strength, not a weakness.
It would be nice if there was some way to prevent users from reporting bugs if they have extensions modifying the website in certain ways, or at least warn them to try disabling their extensions first. I wouldn't even mind a whitelist for certain well-behaved ad blockers, but certain extensions are well known for being bad (many extensions _inject_ ads, which is a recipe for website breakage).
I get to choose how this client-side Javascript runs because I get to choose how every kind of software runs on my own computer.