The problem with this is it's the same argument that gets used to justify things like the Patriot Act. "The only times you see us use this power is in horrible ways, but we assure you, most of the time we use it for good reasons! We just can't tell you about it, because CLASSIFIED". In short, it's an argument that only works if you already trust the person making the argument.

IMO that shouldn't fly with the Patriot Act, and it shouldn't fly here either.