The data something like this collects is so vast that it actually encroaches into the parts of your life you would reasonably expect to be private. For instance, you could determine that I'm seeing a psychiatrist, that I'm cheating on my wife or that I have cancer. That's private information you don't expect to be exposed by simply driving around.

To push the limit of the broad interpretation of your public data, consider the following scenario. Imagine in the not-to-distant future that a technology exists that allows people to scan, from a distance, the electromagnetic field produced by your brain and use that data to map your thoughts and memories. Would me simply stepping into public be a tacit agreement between me and the government to forfeit that privacy?

I think people definitely do have a reasonable expectation that they aren't being followed everywhere they travel, and if you framed it in that way many would see it as a violation of privacy.

It may be true that, in the US, one cannot reasonably expect privacy in the various instants of their public existence, but when you aggregate those instants into a timeline then a reasonable person very much does have an expectation of privacy (otherwise stalking would be perfectly acceptable in society).

Outside the US, I would not be surprised if other cultures have developed a much higher expectation of privacy (or at least discretion and confidentiality) than the US.

Taking the point a bit farther - we have no expectation of privacy for the exterior of our homes. Does this, then, make it OK for the government to use audio recording devices that can pick up sound waves based on the vibrations of the home's panes of glass (because this can actually be done in some cases)? Similarly, the exterior IR signature of our home is theoretically in public view, but should we let the government monitor which rooms are occupied based on changes in the heat signature of the walls?