Uh, that's a completely optional feature. I've intentionally had it on for over 3 years now, and it's one of the first things I turn on when I get a new phone.

You can also delete your location history data at any time.