Right, if they actually _first_ determine that a person is a terrorist, _then_ they start spying, that'd be one thing. But instead, what we have now(or at least what they're asking for) is almost like this:
http://www.criticalcommons.org/Members/ccManager/clips/DarkK...
Basically, no one person/organization should have this data at their immediate finger tips. It will be abused; it's human nature. A more acceptable way to do this is let Apple/Google make their impossible-to-break-encryption-whatever stuff, then when law enforcement gives a warrant for a particular phone number, that specific phone gets pushed a firmware update or something that disables encryption. Pre-emptively/retroactively having access to everyone's phone data at all times is bad.
The fix here is access control. When we have a problem like terror, civil liberties are the first thing to get thrown under the bus - and trying to fight that will generally end badly. Too many people complain that the NSA has these unconstitutional wire tapping programs - where they have next to no chance of making even the smallest difference to the existence of those programs.
What people should actually be doing is meeting half way and attempting to put up firewalls on the data, to make sure it can't be abused where possible. So no giving the data to other agencies without a warrant - monitor the people who have access, and make sure that if they do searches for specific people, that's logged and audited.
Most reasonable people would probably be fine with having some of their communications monitored, were it in some way guaranteed that their data would only ever to be considered in any detail (i.e. by a human) if they were seriously accused of being a terrorist. Personally, I can easily accept that there are certain cases in which I'd have to give up some of my freedoms in the interest of the nation.
My concern is that this data might made more widely available - for example in the DEA's parallel construction cases (or with RIPA in the UK) - where it's clearly being used in ways against the spirit of the law. Were the NSA clearly told - yes, you can collect this data, but you can't share it with other branches of the government and you may only use it for finding terrorists, that's fine in my opinion - unless I'm a terrorist I'm never even going to find out that I've been monitored, and no one will be able to use the data against me.
There's no way to eradicate "stateless radical Islamists", or even Al Qaeda (anyone can wave a black flag labeled "Al Qaeda"). So unlike a war on (say) Japan, the war on Al Qaeda can't end until a critical mass of politicians make the irrational decision to go on the record as saying we're finished prosecuting terrorists.
Historically, US foreign policy seems to get in the worst trouble when it declares itself a military adversary of concepts as opposed to states. See, for instance, "communism".
On the other hand drug use is still a worthy adversary despite the US having waged a long and bitter war of attrition against it for decades. Also poverty, obesity, probably a bunch of other pet struggles we appropriated the word 'war' for.
The problem is that the NSA can no longer be trusted to confine its use to terrorism. Drug investigations, immigration issues, even IRS investigations are using this data. I half expect speeding to be enforced via phone metadata (GPS). Snowden has also shown us that untold thousands of people have access to this data with little oversight. I trust actual spies. I do not trust thousands of twenty-somethings for whom promotion depends on finding some way of turning my phone records into evidence of crime.
Nor do I want my phone records used in any non-NSA security checks. I work in a field (legal) where I do occasionally come into contact with very bad people. It was a legal clinic. He was my client. The fact that we spoke regularly should not follow me forever.