If you look like someone the system is looking for, there's nothing you can do to whitelist yourself other than wear something on your face, you will be flagged, every single time until a human looks at it, and if they're stopping people, that means being stopped, every time.
If the system has the wrong face in the database, bad luck.
If someone finds a way of preventing detection, say, make up, or big glasses, the system doesn't work any more, as far as I know, even the best facial recognition that only relies on cameras and not depth cameras significantly lose accuracy with either of those.
There are a lot of people in London, the false positive and negative rates aren't usually published and anything above zero means those technical problems happen an undisclosed amount too much.
Did you ever hear the story of the guy with a very common name ending up on the no fly list? The no fly list was for a long time, and I think still now, literally only a list of names, meaning if someone with a common name goes on the list, everyone with that name gets stopped at the airport as the database gets checked to see if they're the same person. Now think what happens if someone with a generic looking face ends up in the police database.
Wouldn't such a system (ideally) learn the differences in [who it expected] and [who it found] and attempt to improve its accuracy to prevent false positives in the future? I don't know if this is too basic, but I'd assume training with a set of similar-but-not-an-exact-match data would be vital for accuracy at scale.
At least a couple times a month there's someone who has had repeated issues with law enforcement banging on their door, sometimes even with guns drawn, because someone with a similar name had a warrant out for their arrest, or a criminal background of some type, or someone the police wanted used to live at their address years ago. And there seems to be very little an average person can do to get "the system" to correct itself; even after multiple instances, the police keep coming back again and again and again and again because "the system said so". Massively scaled automated facial recognition will multiply that many times over, especially when officers are trained to just trust "the system" when "the system" says you're a criminal.
That poor guy was effectively fired despite the fact that every human agreed that he actually wasn't and should still be employed.
Perhaps I interpreted it wrong, but I think this speaks volumes for the amount of low-hanging fruit that exists for improvements in such a system: obviously, the above _shouldn't_ happen but does, which raises the question.. why?
I may be (and probably am) wrong, but my guess is that many PDs are still using incredibly outdated systems (at least, mine is), and would vastly benefit from modernizing and improving their systems. There's many other possible reasons (improved systems don't fix this problem, or improved systems have other problems, or administration doesn't care about this problem, etc), but when I read stories like this I think "modern technology would help/fix that" rather than "adding/scaling technology would exacerbate existing problems".
Intrusive CCTV is intrusive. Facial recognition is more intrusive.
It wouldn't be different from an impossibly high number of law enforcement officers keeping watch on everyone at all time like in a high-tech big brother nightmare -- except the tech doesn't work properly, and there are high numbers of false positives that will lead to the wrong innocent people being put under suspicion and potentially having their life ruined by a police investigation that presumes you're guilty until you've been illibated by a court.
They shouldn't be watching or tracking us in the first place.
It belongs to other groups as well. We've seen pretty frequently the state of computer security in governments. Can you also trust that this technology won't be hijacked? Can you tell me that this system can't be used by an invading force to quickly target important individuals?
It isn't always what a technology is being used for that makes it concerning, you also have to consider how it can be abused. This type of technology, and most surveillance technology, has a high potential for abuse. By both our own governments AND adversarial groups.
The differences are cost and scope. The cheaper it is, the easier it is move from doing it to "highly-valuable and highly-like to be guilty targets" to increasingly more innocent people (because "why not?"). That's how the governments think (unless the people strongly oppose these new changes).