This will never, ever happen. English Wikipedia (w/o pictures) copressed is measly 20 GB. It is hard to quantify "all books ever written", but I have kept copies of some online libraries large enough that they for sure have pretty much every book you can remember and 10 000 you never heard of for every single book you can remember. It's not that much, you can fit it on 1 or 2 regular HDDs.

Now, I did it because I'm that type of guy. There's not that many people who actually do this bullshit, even though it's perfectly doable.

So why don't they? Because it doesn't make much sense, if you aren't afraid of upcoming nuclear winter. Wikipedia is updated and improved every day. You only sometimes want to refer to something old, but you nearly always want to check out something new. Petabytes of video are uploaded to Youtube every year. Probably TB/day wouldn't be an overestimation for audio on Spotify. All data is being updated constantly.

Also, the above is valid for pretty aggressive data compression. Is aggressively compressed data what we want? No. 2h video compressed into about 500 MB was totally fine 15 years ago. If I download a 2h movie today, it's normally around 20 GB. And by no means it's uncompressed.

Seriously, by now you should know for a fact, that if one believes there's such thing as "too much storage space" — he's stuck in the 80s.

And even if there would be such thing — realistically, a cluster of nodes in Google's datacenters can find you a book or a video you are looking for way faster than the most perfect HDD you could theoretically have locally. So, again, normal people wouldn't want to have all this stuff even if they could.

So I was just talking smack, but I think it might be possible.

Remember when we could do 64Kb/s reliably over the web? Then you could do 'voice'. And after that threshold was crossed, you could basically do unlimited voice very quickly.

So Wikipedia - text only - is ballpark 50G - which is to say it would fit on a single mobile phone.

That is bigger than the first Google index!

I don't know how old you are, but the notion that you could walk around with this massive database, literally the size of all of Google right in their pockets in 1999 - would have blown people's minds. It was basically unthinkable.

The rate of growth of storage has slowed down a little in the last decade but there are still jumps to be had, and it's not inconceivable that we get 100-500TB storage in the nex1 10-15 years in regular devices, meaning 10-100x that in a slightly large storage device.

While video data is expanding (4K is much bigger than original HD) it can't go on forever, meaning, just like voice and text, once we cross a certain threshold, then it becomes irrelevantly small relative to storage as well.

So I think there's some value in my point:

In 10-20 years from now, as video storage becomes 'trivial' just as text is today (aka all of wiki text on your phone) - then huge amounts of data become available, instantly.

Though some data sources change a lot - others do not.

It's not inconceivable that we put the 'entire western canon' in everyone's homes.

The other thing no so evident in my comment is that there's only so much use for all of this data.

We are getting massively smaller marginal returns for all this 'big data' we store, frankly, I question in many cases if it's worth it at all. I think a lot of companies have been duped into saving every mouse click or whatever concerning every customer. The world is just not that complicated.

What this means - is as computers miniturize - and storage as well - we may see regular data centers shift away from the cloud back to 'on premise'.

If you can fit 'infinite computing power' in a little closet, and the parts are easily replaced ... then I can see companies doing that.

The promise of cloud computing today largely rests on the economies of scale of physicality: parts take up space, cables, power, heating/cooling, you want a lot of flex/headspace, configurations.

I don't see why in 20 years from no, you can buy 'off the shelf' a 'box' that has the equivalent of 100Ec2s, 500TB of storage and multiple G/s networking cards.

You could run an entire corporate office of 1000 people from just a single box.

The 'physicality' of it all would be mundane and irrelevant. Obviously, it would be 'super complex' and still need 200 IT people to admin all of the software, but physically it could be small and cheap.

'Big maybe' of course, but there are some possibilities in there I think.

The entire libgen archive is roughly 50 terabytes (most of the books ever written). It will be a very long time before we reach that level of storage in a phone.

Only because we changed the goalposts. My 2007 iPod has more hard drive space than my current phone, but it was slow noisy disk storage.

Meanwhile my phone's storage is basically now just a high speed cache for the internet.

Whether it will go back the other way again in the future, I don't know if anyone can know but tech does tend to be cyclical.

To be fair, libgen has a lot of PDFs. Libraries in fb2 or such are orders of magntitude leaner.

I'm not sure what you mean by a very long time. We have 1tb microsdxc now and it is about 1000 times the capacity of sdcards available in 2005.

What's more ... although it would be ugly and awkward, one can easily imagine that 50 micro sd cards could easily be mounted on a single USB like device that sits in your pocket and attaches to a phone. The reason it doesn't exist is primarily nobody needs it rather than "it can't be done".

I share your childlike optimism. But another thing (that neither of the sibling comments mention) is write speed. Having a ton of capacity to store massive amounts of data at rest is not the same thing as being able to easily make near-instantaneous (or even slow) copies. Getting all that data on there is going to be a problem, and so it would only be economical for data sources in high demand, not unlike the way that optical media get the pits and lands stamped in at the factory based on an expensive master. "All of YouTube" might be (probably is) in high enough demand, but it would also require cooperation from the gatekeeper of that content, who at this time has adverse incentives because it's making (many times) more money selling ad impressions for basically every view that it's able to. Even aside from that, what other large datasets are in the same demand category such that the work could or would be subsidized in that way?

I love this thought, but have some doubts.

So far, storage is still following something resembling Moores law, but it will probably hit physical limits way before a year of youtube (probably 30 of these new tapes or so) fits into your hand.

HDDs are hitting physical limits, SSDs still have growing room. You can already buy 100 TB SSDs.

https://www.newegg.com/nimbus-data-dc-100tb/p/2U3-002M-00004

Following Moore's law of 2x density per 18 months, in 10 years we will see single drives of:

100 tb * 2^(10 years/1.5) = 10,159 TB

It's why I pirate at 720p x265, the files are tiny. I'd rather have a extremely large and varied Emby instance than 200 movies in Ultra Bluray Remux.

Love that last line! Wonder what will happen drm wise in the future if we get to this point.

Buying something would just transfer the encryption key

Not going to happen. Once you think you've got everything locally, exponentially more will be generated anew by everyone.

Copyright is a barrier to that.

Relevant article:

https://www.theatlantic.com/technology/archive/2017/04/the-t...