Total HDD shipments are ~135 million per quarter[1]. If every one of those drives was 4TB, entire world production would be about 2 zettabytes yearly. 1 yottabyte would take 500 years of current HDD production.
http://www.storagenewsletter.com/news/marketreport/trendfocu...
The analysis is still good though, assuming that the data center does have a one yottabyte capacity.
However, let's redo the analysis with a 2 zetabyte assumption, keeping in mind that the full data capacity of the center is probably not purchased and set up in a one year period:
We have 2,000 exabytes. That is 2,000,000 petabytes. That is 90909.1 days or 249.1 years. There we go, that sounds better! They can now only store a little over 10x the 22 petabytes per day comfortably.
http://www.wolframalpha.com/input/?i=2000000%2F22+days+to+ye...
The NSA's Utah Data Center will be able to handle and process five zettabytes of data, according to William Binney, a former NSA technical director turned whistleblower.
http://www.npr.org/2013/06/10/190160772/amid-data-controvers...
However, from the same article:
Despite its capacity, the Utah center does not satisfy NSA's data demands. Last month, the agency broke ground on its next data farm at its headquarters at Ft. Meade, Md. But that facility will be only two-thirds the size of the mega-complex in Utah.
Yah right. That would require the ENTIRE worlds production of hard disks for several years.
At 3TB per hard disk you need 350 million of them. Total world production is about 600 million - but the vast majority are much smaller than 3TB.
And that would require a building 1/5 of a mile on each side - just to hold the hard disks, never mind power, cooling, computers or network.
It would require 2.5GW to power just the hard disks - and never mind cool them, or power the computers and routers.
So: Yah Right.
Large scale commercially available SANs advertise de-duplication in the order of 50%[1].
Commercially available backup software advertises de-duplication rates that reduce storage requirements by up to 99% [2]
I don't know how they measure "storage", but the The Utah data center requires 65 MW of power[3] which is a non-trivial amount.
[1] http://www.netapp.com/us/products/platform-os/dedupe.aspx
[2] http://thebackupblog.typepad.com/thebackupblog/2011/07/insid...
[3] http://www.npr.org/2013/06/10/190160772/amid-data-controvers...
Assuming 80% fill factor for error correction & redundancy, it's 3.1 x 10^11
http://www.wolframalpha.com/input/?i=%281+yottabyte+%2F+4+te...
http://www.wolframalpha.com/input/?i=%285.26+x+10%5E8%29+*+1...
What does this tell me? Either the NSA has storage that is 2 or more orders of magnitude denser than current industry parts, or that a lot of their data is not in online storage - it's been moved to tape for warehouse storage (but LTO v6 is not as dense as hard drive - only 2.5TB in a larger device, so the warehouse would still be huge...)
Or most likely, the NSA just doesn't have 1 YB in their data center. Think about the disruption there would be in the hard drive market by someone buying 3.1 billion hard drives, even if they spaced out their purchases. And coming after the factories in Thailand were flooded a few years ago, with the shortage that happened.
So what if they used data compression on disk? Assume they got 90% compression (english text, voice, etc are pretty compressible), that means they only need 1/10th the number of drives. Wolfram Alpha says their data center space requirements go down to 34 square miles. Ok, a big savings. But it's still an area the size of Manhattan.
Let's break this down:
>1,000 gigabytes is a terabyte.
>1,000 terabytes is a petabyte.
>1,000 petabytes is an exabyte.
>1,000 exabytes is a zettabyte.
>1,000 zettabytes is a yottabyte.
Assuming they use the Microsoft standard to count and not 1024 bytes to a kilobyte. Let's assume they have 1 yottabyte of data storage. That's 1000 zettabytes. That's 1,000,000 exabytes. And, finally, that's 1,000,000,000 petabytes.
At a rate of 22 petabytes per day it will take approximately 45454545.45 days or 124,533 years to fill only the Utah data center. (Thank you Wolfram alpha [2])
So the scary answer to your rhetorical question is that "Yes, the NSA has the storage capacity to easily, if not comfortably store 22 petabytes of data per day, or even 500 times that without breaking a sweat."
----------
[1]http://nsa.gov1.info/utah-data-center/
[2]http://www.wolframalpha.com/input/?i=%281%C3%9710%5E9%2F22%2...