I felt/feel the same, but I am greatly comforted by the add-on making an attempt to clearly communicate what it uses/needs each of those permissions for: https://addons.mozilla.org/en-US/firefox/addon/tridactyl-vim...

Even though there’s no guarantee that the explanation corresponds exactly to what is implemented behind the scenes, this unusual pattern of user respect and consideration makes the add-on/developers feel more trustworthy than other sketchier ones.