I guess that I find the idea that you should expect privacy when in a public space kind of strange (its right there in the word: public), but that doesn't mean that I think its ok for someone to follow you around recording you (but not because of the actual act of being recorded, but rather because of the targeted nature).
Similarly, I think passive recording (ie non-targeted surveillance) of public spaces should be allowed in and of itself, but that its the use that dictates whether its abusive or not (ie if its done so that people can be identified, then that seems similar to me to following someone around, but if its done for the backdrop of a movie or art project, or its done to study foot traffic on a street.. basically there are many reasons which aren't abusive).
You don't need to outlaw cameras any more than you need to outlaw knives to keep people from stabbing others. But as mentioned, there's a fundamental difference in the idea of privacy, I suppose. It can be understood as "something that happens in a non-public place" or it can be understood as a larger idea that you have a certain right to not be surveilled, recorded and stalked.