The hard solution is to secretly duplicate traffic from every data center operated by each of these companies, reverse engineer every HTTP request that goes back and forth so that the data can be parsed, maintain it for every product change that happens at these companies, circumvent HTTPS by compromising the certificate authorities, store it all, and still maintain a massive analytics tool that can make sense of the astounding amount of data coming through.
The easy solution is to avoid all of the technical ugliness of acquiring the data, and just legally make the companies give you the relevant information, neatly structured and packaged. NSLs are the ultimate hack.
You cannot decrypt passive monitoring with a CA's key.
Google started using these for their TLS setup about a year ago, and it's smart. They blogged about it, hoping others would follow suit.
I wonder if perhaps that change, and blog post, were their way of saying without saying.
For example for emails: emails travel unencrypted through the hops, and they would store them all, and then constantly analyzing them. When something suspicious comes up, they would go to the email provider to ask for more data. So for example if gmail address is there, they would go to Google and use their PRISM interface to get more data associated with that gmail adress, if it will be yahoo email, they will go to Yahoo for more data, etc.
This is even more complicated when the data centers are in other countries, and none of the data actually enters the US. So if two EU users were accessing Gmail from the EU, the data may never enter the US at all. This means any network tapping would have to be done in the EU as well, requiring cooperation from many international telecom companies.
It's still easiest to just force Google to hand it over via NSL. Google's still legally bound to deliver the data even if it isn't physically stored in the US.
From their perspective, why not?
A lot of the communication won't be encrypted anyways, and some of it will be, but they may be able to decrypt it at some point in the future.
The hard solution isn't just a little bit harder ... it's several orders of magnitude harder and more expensive. It's also highly vulnerable to simply using encryption. The easy solution works because the US companies are bound by law to cooperate. There's no reason to believe that legal pressure on these companies has failed to get the government what it wants.
Also, not every company is a US company.
I doubt that costs for such an operation would make any difference to the US government, considering how much it is spending per day on its military.
PRISM is a web-app. The slides make it pretty clear its a web-app. The army field manual link helpfully posted before in this discussion outright says its a web-app.
But what I'm afraid of is that this mess with deciding exactly how much access the government has to Google will turn into a distraction from the larger picture. Which is that, in all likelihood, the NSA does not have access to Google. What they do instead, and what the name PRISM implies, is that they connect to the backbone (Verizon/AT&T), scoop up ALL data, and store it in their freshly built data center in Utah.
The slides I saw seemed to indicate when certain applications or filters came online. Such as a filter for Facebook data, or a filter for Google search/map/GPS data, etc. That's how I interpreted the graph, at least. It would indicate the NSA is rolling out specialized applications to handle data coming to and going from specific sites. Which allows them to more intelligently decipher what is being said, in more or less shotgun fashion.
Hence, the name PRISM. It's a project to split the full Internet stream into a Facebook bucket and a Google bucket, etc.