It's tricky; little things like this may not seem like much, but in aggregate they can be quite frustrating to users. Personally, this is something I do often enough that it noticeably reduces friction for me so I live with it because it's very convenient.

However, it seems like there should still be a way to provide nearly as much convenience to users while still protecting their privacy.