Ya I think an option to ask permission per website should be there..it could be subtle like an orange icon that you need to manually click the first time to give permission to that domain