My Android device allows for completely disabling location access (regardless of whether the app requires it) or for allowing access while the app is in use, as you mentioned. I'm never sure if these features are OEM additions or features in standard/AOSP Android versions, so I guess if you're using an Android device YMMV.