They could just ask once for defaults not every time and have a per app dialog where the user could tweak the permissions, like browsers do. For instance I have almost everything blocked in the browser: camera, location etc.