It's not a stretch of the imagination to see these things being sold to airports, seaports, mass transit stations, and storefronts as a security feature. Next, your physical mail could be scanned. None of that seems politically unlikely in the current climate.
The tools already revealed for large scale surveillance are cheaper, more effective, and more robust to outside attack than the mentioned ideas. More importantly, they are already there - there is no rollout cost at all! And up until recently it was also easier to keep the public in the dark...
I do see applications at the places you mention, but for a very different reason - border inspections (coupled with human oversight) are an excellent place for automation where a small amount of effort could lead to a massive increase in throughput per person.
The only downside is that officials who deploy these things will want guarantees on effectiveness, which you can never truly give due to statistics. Couple this with the fact that neural networks are very difficult to tune for false negatives and false positives and it would be a difficult sell.
One alternative would be to use these types of networks as black-box preprocessing, followed by a "tunable" algorithm like logistic regression where you could effectively control the ratio of false positives - a high rate of false positives coupled with human oversight could still lead to a large boost in human performance if most of the border inspection process is uninteresting.
But still there are unions... which is a whole separate issue to itself.
China, Hong Kong, Singapore, the UK, and an increasing number of cities worldwide already have massive video surveillance networks allied to local law enforcement, traffic management, and other functions. Adding another stage of image processing would help to leverage and extract actionable data from those (the classic problem of CCTV is that nobody watches it) ... with probably very little additional outlay compared to existing investment.
I can't see anything stopping this commercial progression, in fact I see it as inevitable unless the citizenry can somehow curb their politicians. Good luck with that...
Ultimately, the thing stopping mass surveillance is not a limitation of technology, but of policy. For better or worse, the days of "they don't have the resources to do that" have been replaced by "they aren't allowed to do that".
If you have access to the raw packets going to and from every device, and the accelerometer in almost everyone's pocket, identification can be much simpler than doing full face recognition all the time.
I seriously doubt the dawn of the surveillance state will be heralded by deep neural networks recognizing faces in the streets - hardware and software backdoors on phones are cheaper and more effective.
[1] https://www.facebook.com/publications/546316888800776/