I dearly hope the solution involves not installing webhooks on all one's repositories.

Not sure I understand what the webhook is for. (I've not yet installed the app — because of this issue.) Is this done in order to automatically track Github issue changes? If so, what happens if you want to use this app on a project that you don't have admin rights to?