Firsthand experience with these systems, and horror at their capabilities,
is what drove a career intelligence officer to provide PowerPoint slides
about PRISM and supporting materials to The Washington Post in order to
expose what he believes to be a gross intrusion on privacy. “They quite
literally can watch your ideas form as you type,” the officer said.
If the NSA is snooping at the line level, which we know since the AT&T closet revelation a decade ago that they do, then they could see real-time typing in apps like Google Instant.1) Their budget is an order of magnitude higher; 2) PRISM is just an efficient legal data collection framework, an API that companies are forced to comply with
For stuff in FB, FB could be pushing that target-specific data to collection points in real-ish time. Instead of doing daily dumps, you just provide a targeted stream of data.
You seem to know all about that.
This is exactly the same as how the slides saying that the NSA collects data directly from the companies was misinterpreted by WaPo to mean direct access to all the companies' user data. I suspect Snowden jumped to the same conclusion when he came across those slides while sysadminning, and that's why he's in the mess he's in now.
Under rational scrutiny the PRISM story has fallen to pieces. It doesn't make any sense that all of the high-level executives and hundreds of thousands of top engineers have no idea what is happening, while some guy from Booz Allen and a blogger are the only people with the truth.
These allegations are very serious and if by any remote chance they were true, those powerful people would be busy trying to make Greenwald look like a flake and Snowden like a confused tech incompetent.
https://financialcryptography.com/mt/archives/001431.html
Excerpt: "How is this apparent contradiction possible? It is generally done via secret arrangements not with the company, but with the employees. The company does not provide back-door access, but the people do. The trick is to place people with excellent tech skills and dual loyalties into strategic locations in the company. These 'assets' will then execute the work required in secret, and spare the company and most all of their workmates the embarrassment. ..."
In a discussion of this article (on a cryptography list) I observed this incredulous response: "Hmm. So what does that mean a team of ex-military/intelligence security people work there way up or get assistance with contacts and references, replace all the key people in a companies inner security department and start coding up backdoors, APIs and allowing VPN access to it? All without telling anyone or getting noticed by ops people etc."
To which the other party retorted: "Been there. They are noticed, but you get orders from on high to shut up and not notice."
If that's all true, then it sounds like only a very few engineers and managers acting as moles will have specific knowledge of the program. A few non-mole engineers will sense that something's afoot, but they'll stay mum. Maybe that's as far as it goes.
These are highly implausible scenarios.
As for the agent: two words - qualified immunity
To anyone who notices, there's always "shut up and not notice." But there's also "oh that rsync you see there is for our geographically redundant backup facility" or whatever -- in other words, dissembling.
You seem to be assuming that the equipment budget is tied into that. I may have missed something in the details of this program, but I would guess that the NSA's equipment budget is separate. If I had to guess, I would say that the NSA has many software systems that share supercomputer resources (did you think that big new datacenter in Utah was just for storage?); PRISM is one of these, but probably not the only one.
I am guessing that $20M is the price of developing, maintaining, and improving the PRISM software only. The NSA's main problem is not hardware, it is algorithmic -- they are processing a lot of data, and they need software that can scale well and give good results. $20M is a big budget to pay top computer scientists to solve algorithmic problems.
For $20M, you can't do what Prism claims to do for $20M. Even suppose you limited your scope only to Google, AND now you limit only to GMail, AND your development and installation costs are free, and all you're spending on is bandwidth and servers and cooling and maintenance staff, I still doubt you can read all of GMail for $20M/year.
Then there's the bit where they claim that analysts, what's the phrasing, "quite literally can watch your ideas form as you type". Seriously, think for a second what it would mean to be able to do that. Think of how that would work, in terms of network protocols. Now go install wireshark and open a gmail window and start composing an email, or writing half of a chat (without hitting Enter). I haven't done the experiment; maybe you can be the first and break a scoop!
Obviously the people who reported on PRISM have told some lies about what it does. Either it can't do what they say it does, or it cost a lot more than $20M, or both.
(I don't want to minimize the seriousness of the accusations... they should be given due appraisal. But they basically cannot be literally true as written.)
And look. Google gets about 100 "requests" (legal demands) from government per working day, and responds (under legal duress) about 30 per working day in the US alone. You think maybe some kind of workflow management software might be appropriate here, or you're imagining this all goes back and forth over email? Or maybe you're imagining an anonymous FTP server somewhere? Obviously the right thing to do is build a secure custom ticket-tracking system, to manage demands, push-back, authorization, fulfillment, etc. If we want to believe it's all okay, here's a plausible story. Maybe it's true, maybe it's not.