Exactly my question. Buried deep in location services is a frequently visited locations that apps have access to even if they only have access "only while using."

http://www.iphonehacks.com/2016/10/how-to-turn-off-hidden-fe...

I'm pretty sure 3rd party apps don't have access to that data anyway...

Yes that exists, but you are wrong about the permission requirement. CLVisit is meant to be a lower power alternative to always tracking, but still requires the same “always” permission: https://developer.apple.com/documentation/corelocation/getti...

iOS has access to these, third party apps do not.

I didn't know of that, thanks a lot for sharing!. Reminds me of the Google Location history on Android devices that shows you scaringly accurate where you went during your day.

In iOS 10 Settings > Privacy > Location, under the "Location Services & Privacy link, it says: "Your iPhone will keep track of places you have recently been, as well as how often and when you visited them, in order to learn places that are significant to you. This data is kept solely on your device and will not be sent to Apple without your consent. It will be used to provide you with personalized services, such as predictive traffic routing." This is consistent with Apple's philosophy of keeping personal data processing on-device as much as possible to protect user privacy. You can always keep the frequent locations feature turned off, or use "Clear History..."

Apps have never had access to location history data. iOS itself tracks this info to enhance some OS-provided features, but apps cannot get at it.

Could you provide a source for your claim that apps have access to location history data?

It's false. Even Apple servers have very limited access to this data.