The NSA only was given such pervasive power to catch terrorists. The fact that privacy seekers are more likely to be "bad actors" is moot unless that means terrorists because regular "bad actors" are supposed to be innocent until guilty, handled by police/fbi, etc. NSA only is allowed to work essentially with no due process because it's for cathing "terrorists". Interestingly, now that terrorists know about the NSA, they no doubt will simply not use the internet (or phones) at all, thus making it so the NSA can't catch a terrorist by any of its methods.
I definitely expect all terrorists are extremely careful about internet activity now that they know the NSA is so invasive, thus making the NSA's actions even less defensible (not that I thought they ever were).
..aaand the ones who get "caught" and are away with a slap on the wrist before you can say "this is a joke, I just cannot believe the hipocrisy of this, double standards much?" (because let's be realistic, it's not instant)
No, that's not the reasoning. People that do bad things try to hide them. Therefore, a good first filter to catch bad people is to target those who hide things. They can narrow the search field afterwards.