Having Americans spy on everyday Germans, that's completely different.
It's one thing for an American on the street to see a German and size him up to see if he's a threat. Does he have a weapon, is he of sound mind, etc. It's completely another to follow him home, peek through the windows, poke through his phone, computer, trash, and then say you're only defending yourself and that everyone does it. The level of intrusion is unreasonable.
The principle of the gag order comes from the implementation of the law. Obviously the public law itself allows gag orders, otherwise Google's lawyers would tell Google to go ahead and publish them. Gag orders have been around in America, at all levels of law enforcement and security, since waaaaay before all this.
The big defect is allowing long-duration (or even permanent) gag orders, and not just for the NSA-style stuff. There's been quite a few court cases where parties settle on something which is public-interest but the judge seals the whole damn case forever.
Of course these laws should be abolished.
But these cases show an important difference: These laws while outrageous, were created transparently and published. This enabled citizens to take actions against them, and hopefully at some point they will be abolished.
In the case of the NSA everything was covered by gag orders under the penalty of committing a felony. That is way more insidious.