Firefox has both the option of granting the permission only for the current pageload (this is in fact the default behavior!) and exactly the "click the icon and revoke the permission" behavior you are asking for once you have granted the permission. So "browsers" already do this, for some values of "browsers". l)

[Disclaimer: I work on Firefox.]

Many permissions like that are used in ways a typical user doesn't expect. Companies will explain it away by saying you agreed to it in the terms of service, although it's generally acknowledged that almost nobody reads or understands those terms for any company. For example, most people don't realize that when you give an app read access to your photos, it probably will scan every single one of them for data and upload it to the app's maker. Practices like this are so common that I don't even think the developers understand that they're abusing users' trust.

On macOS Safari, any location request through the browser API (not based on IP geolocation server-side) only gives the option to allow it for a day, not permanently. It's a start.