I don't see what else would affect it besides things that you could also verify in theory.
Otherwise, almost every rendezvous you have with the server is an opportunity for methods to be rebound and your cryptosystem to be subverted. If you don't see how, it's a good exercise to investigate.
It would work if the plugin can inspect all browser requests attempt and verify every single asset before it's loaded into the browser.
How do you know which public key belongs to the address? Key servers? Web-of-trust? Look it up on their homepage?
Scramble has an address -> pubkey resolution system which balances security with usability.
At least with Scramble you can create a secure USB stick to boot from with all of that preconfigured. Any solution that uses bare GPG still has severe usability problems.
I salute what you are trying to do, but you have to understand that the Js browser method, as it stands today, is a dead end.
The browser extension would only load vetted assets onto the DOM. In other words, you visit the site by opening a new tab and clicking on the extension. It then loads assets and checks that all the assets are signed by a trusted list of code-vetting signers. If all the signatures look good, then it loads the assets onto the DOM. This is no less secure than having all the client code in the extension.