News for hackers. Stuff that matters.
Re:Whistleblower Binney says the NSA has dossiers on nearly ... by VikingCoder on Wednesday July 18, 2012 @11:22AM (#4261159)
Math is fun!

You can buy a 2 TB HD for $99.99 right now.

The population of the United States is 311,591,917.

That's 7057 bytes per citizen, for a penny under a Franklin.

The Complete Works of William Shakespeare can be downloaded from Project Gutenberg in EPUB format, in 2.3 MB.

Therefore, if you wanted to have William Shakespeare write as much text about every single citizen in the United States, as he wrote in all of his Complete Works, you would need 341.74 of those HDs.

Therefore, the raw storage would cost you $34,196.58.

The NSA's annual budget is around $3.6 billion.

Every year, the NSA could buy enough hard drive space to store 105,273 times as much text as the Complete Works of William Shakespeare, on every single U.S. citizen.

Re:Whistleblower Binney says the NSA has dossiers on nearly ... by dman on Wednesday July 18, 2012 @12:57PM (#4261639)
Storing the text is easy - deciding what text is relevant enough to store is hard.
Re:Whistleblower Binney says the NSA has dossiers on nearly ... by VikingCoder on Wednesday July 18, 2012 @01:04PM (#4261674)
If you had the NSA's budget, you probably wouldn't need to decide what text is relevant to store.
Re:Whistleblower Binney says the NSA has dossiers on nearly ... by dman on Wednesday July 18, 2012 @01:19PM (#4261752)
Deciding what to store is highly correlated with what information are we most likely going to find useful.
Re:Whistleblower Binney says the NSA has dossiers on nearly ... by VikingCoder on Wednesday July 18, 2012 @01:27PM (#4261801)
When I finally bought an iPod, it had 60 GB of storage. I thought, "Which albums of mine should I upload to it?" Then I remembered, "Oh right - all of them!"

My point is that deciding what to store is expensive and error-prone, with false-positives and false-negatives. Storage is so cheap, that there's an argument to be made that they can just store everything they want to.

Re:Whistleblower Binney says the NSA has dossiers on nearly ... by thereallurch on Wednesday July 18, 2012 @12:55PM (#4261630)
While you make a good point, your end number is wildly inaccurate.

1) I highly doubt the NSA is buying $100 dollar low grade hard drives. I bet this data is backed up at least twice per site, at multiple sites.

2) The data is not unstructured, it will be structured in some sort of database. Storing your facebook photos alone will be potentially hundreds of megabytes.

3) I doubt NSA is spending it's entire budget every year on hard drives.

Re:Whistleblower Binney says the NSA has dossiers on nearly ... by VikingCoder on Wednesday July 18, 2012 @01:04PM (#4261666)
Yes, that's why I quite carefully said, "Every year, the NSA COULD..."

Emphasis on the COULD. It's to emphasize how cheap storage is, and how relatively few U.S. Citizens there are. In our day-to-day lives, we think 311 Million is an enormous number. But in the scale of computing, it's actually quite small.

My real point is: storing the data is no longer a challenge. If you want someone (businesses, governments, etc.) to not store data about every citizen of the U.S., you need laws, regulations, and checks and balances.

Re:Whistleblower Binney says the NSA has dossiers on nearly ... by pessimizer on Wednesday July 18, 2012 @02:33PM (#4262146)
1) If he's three or four orders of magnitude off, his point still stands.

2) Also, the data could be almost completely unstructured. You mine the data that you have to create structure. If you toss anything, or prematurely optimize things, you're missing out on what the algorithms that you come up with 10 years from now will give you.

3) If you're using enough redundancy to multiply $35,000 into $3,000,000,000, you're insane.

Re:Whistleblower Binney says the NSA has dossiers on nearly ... by sliverstorm on Wednesday July 18, 2012 @01:17PM (#4261738)
Why does this matter?