How else could they enforce the city's laws, besides waiting for someone to actively commit a crime with the gun before they can ask if the gun is legally owned?
Right, so we've established that legal firearm ownership is vastly outnumbered by illegal firearm ownership in the above places, and that seeing a legal firearm on the street is in fact extremely rare.
So... you live in an area where there is a substantial known presence of illegal guns, that isn't balanced by a substantial ownership of legal guns. When you see a gun, how is it not at all suspicious?
I do not see your conclusion as a logical extension of constitutional principles, it seems more informed by your own stance re: gun ownership. The constitution grants the right to search upon reasonable suspicion - I'm unsure how a gun's appearance in a place where there are almost no legal ones fails that bar.
Whether or not NY should have such strict gun laws is an entirely separate debate - as it stands it seems perfectly reasonable for the presence of a gun to be suspicious.
The NYC subway system is particularly sensitive to this, as others have pointed out, because so many people use this to work every day. You can say the bus is an alternative, but what about when they make a requirement to get on the bus? (TSA has already been caught at Greyhound stations!)
...or looked at from an entirely different angle, why should there be people trying to determine if I have a gun in a bus, on the subway, or on the streets of New York? Aren't guns legal in America? The fact of the matter is that it is so impossible to lawfully carry a gun in NYC that the cops, now using these scanners, presume you can't have one. Imagine a Texas sheriff with a scanner. "Excuse me sir, I've scanned you and you have a gun on you." "Yeah, so?"