I mean if we know that there are Room 614As [1] all over the country, and that they are diverting nearly ALL communications to their facilities, in addition to them building new facilities to house data all the time [2].
All of this culminating with their shiny new data center [3], which cost 2 billion dollars to build, covers 900,000 sqft, and has a 40 million dollar annual utility bill, it becomes less about proving that they're doing this, but proving that they aren't doing this.
[1] http://en.wikipedia.org/wiki/Room_641A
[2] http://www2.sacurrent.com/news/story.asp?id=69607
[3] http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/al...
Because I personally can't imagine another datastream 1) of such magnitude and 2) worth billions of dollars to build a data centre for.
First, they'd need a shadow Internet backbone capable of transporting all that data to the DC. Second, they'd have to build a system with the same write-rate as all of the internet backbone. This stuff isn't cheap, and the NSA's budget is big, but not that big.
As 1/3rd of the US's internet traffic is porn, which you can really just de-dup by URL, it's a safe bet to say that they're filtering before they transmit data back home. The same goes for streaming video traffic. The remainder is mostly web with a small fraction being actual communication between people. For the NSA's mission, even implemented in the most evil way, they just don't have the money, means, or motivation to record everything. Instead, filtering on *hotmail.com connections, anything over SMTP ports, etc. makes substantially more sense.
We know, thanks to XCD, that 7523 hard drives per second is created by the storage industry (globally, a total of 650 million drives per year). Say that the average storage space is over the last 5 years, around 500GB.
That mean, so long NSA buy's 0.265% of all hard drives produced each year, they will have enough hard drives to record all data transmitted inside the US borders.
They would still need to write the data. One obvious way would be to store it on site, and transport the drives to a central place. Drives are not big, but it is a noticeable work, so if people were doing this, there should be more verifiable proof of it. If we include post-storage compression, finding duplicates and any other tricks, the numbers should be able to be lowered by 50-75% or so, and might be enough to send some through the wire and only the overflow through drives loaded onto trucks.
One could also ask what 0.265% of the storage industry output is in raw cash. To answer that, my answer is, I dont know :).
Your actual data is just a subpoena away anyway in most cases.
Free == didn't come out of the NSA's budget
Consider how big the Internet is. Even if the NSA has 5 100,000 square foot data centers, think about adding up the aggregate data center footprint of Microsoft, Facebook, Apple, Google, Yahoo, Amazon, Rackspace, etc, etc. That comes to a lot more than 500,000 square feet. And this does not even get into the enormous data resources in Asia and Europe.
A much more likely scenario is that they are heavily filtering the data in real time and keeping just what looks useful or suspicious. That is still scary, but less fantastic than the idea that every packet is getting stored by the NSA.
I've looked at my own traffic and how much content useful for storing I'd have and it's probably less than 500Mbytes per month. And that's counting downstream and upstream.
But net is not the only thing NSA is interested in storing - banking, other records, communications over satellites, etc goes in there as well.
What's truly scary is people are sort of "meh" about this. Or they don't believe that this is really happening, as most of this thread seems to be the case in point.
I was thinking something to the effect "I will make all data I save that references your account or IP available to you through the following portal" - I of course, can't do it yet, as I haven't built the portal or figured out how to separate the data, but I think it's a neat solution to some of the problems I have (e.g. I really want to start doing console logging on VPSs 'cause it gives me a wealth of data when someone says "something was wrong with my VPS last night" - but so far I haven't (I do log physical consoles, just 'cause it's impossible to deal with hardware otherwise) due to privacy concerns.)
I need to limit the number of times I give myself extra work as 'motivation' - you get to the point where you spend all your time shooting alligators, and don't have any time at all to drain the swamp.
I mean, sometimes this is better than just not doing it? but not in this case, if you ask me.
Somewhere I read a quote: People under surveillance, are already imprisoned.
The first step is to understand that one is under surveillance.
Paranoids don't make backups, they just stream their hard drive across the Internet and let the NSA archive it.
Now seriously, the government got too big. What are you going to do about it?
Caution! Everything you say here or anywhere is being recorded, and may or may not be used against you in a court of law.
It certainly might be the case that all net traffic is being stored by the NSA, but this man's say-so is hardly proof.
[1] https://publicintelligence.net/binney-nsa-declaration/
[2] https://www.eff.org/cases/jewel