Perhaps one day increased camera resolutions will allow trackers to identify something humans can't, like retinal scans. This would allow cameras to basically "fingerprint" people, and will be super creepy.
> Consistently identifying a single person from a database of 100Ms/billions of people
I don't need to, there aren't billions of people who are likely to be at the closest ATM to your house.
> constantly gaining weight, growing facial hair, etc
But I see you several times every single day.
> false positives will become an exponentially bigger problem as you increase the candidate space.
To be honest, I think that this is the most likely thing to scare people, that you won't be allowed to fly because a computer made a mistake and everyone trusts it or can't override it. That the police get called when you go to pick up your son because you're the same height and rough build as a child molester who was released from prison yesterday
I think we can assume many agencies have access to ALL Facebook data, Photostream data, etc. With that it is very much possible, dare one say, easy?
[1] No fly lists, for example.