News for hackers. Stuff that matters.
Re:Optical storage – the future of long term data preservati... by jeffbee on Sunday November 14, 2021 @05:33PM (#29221382)
What is the utility of cloud computing for archivists? Is it too expensive? It seems like the longevity is fairly proven at this point. If you put something in S3 in 2006, it's still there. GCS was launched over ten years ago, too. With a multi-cloud strategy you could probably make the case for durability, certainly as a 2nd or 3rd replica that just happens to be dramatically more convenient than optical discs.

Considering that many government records are now produced in the cloud anyway, it makes a certain sense. USGS elevation models for example are hosted on S3. Are they also on tape somewhere else?

Re:Optical storage – the future of long term data preservati... by KennyBlanken on Sunday November 14, 2021 @07:24PM (#29222115)
It is far too expensive due to the power. Remember that you're not keeping one drive per unit of data going, but multiple drives for redundancy. Then there's the power used by the fans, power supply losses, and then the datacenter cooling costs...plus all the network infrastructure and so on. Some of those are pretty small but it adds up to a lot to keep, say, 6TB work of data online vs in a LTO cartridge sitting on a shelf.

Drive manufacturers, storage vendors, and cloud providers have been working on lowering opex/TCO for years. Bigger and bigger drives, helium which reduces aerodynamic drag, tiered storage where stuff is moved to slower and slower spindles the longer it goes without being accessed, for example...in some cases automatically getting pushed to tape. But you need systems to manage all that, systems that take power, have support contracts, and introduce a lot of complexity.

Eventually flash memory will get cheap enough - but flash degrades with time, too.

Re:Optical storage – the future of long term data preservati... by jeffbee on Sunday November 14, 2021 @07:35PM (#29222173)
Eh, but the whole point of the cloud is if you mix a cold storage workload on part of a disk and an active workload on another part of it then the marginal cost of the cold workload in terms of all that stuff you mentioned is almost zero.
Re:Optical storage – the future of long term data preservati... by wmf on Sunday November 14, 2021 @09:59PM (#29223144)
Re:Optical storage – the future of long term data preservati... by selfhoster11 on Sunday November 14, 2021 @11:40PM (#29223780)
Archivists need access to this data decades, if not a century down the line. I trust the Internet Archive to be around somehow for much longer than S3.
Re:Optical storage – the future of long term data preservati... by Kye on Monday November 15, 2021 @08:39AM (#29227192)
Yep. Companies eventually die. Sometimes they're reborn, like IBM, AOL, and Microsoft, but never with the same priorities. The Internet Archive is almost as old as Amazon and archiving is its entire reason for being. The pendulum will swing back as it always has and send things off big [mainframes, clouds], and S3 will be sunset in favor of serving whatever comes next.
Re:Optical storage – the future of long term data preservati... by rbanffy on Sunday November 14, 2021 @05:47PM (#29221500)
> Are they also on tape somewhere else?

I can assure you they are and that there must be a number of copies available for use on any respectable supercomputer.

It just may be that the canonical source of the data is, officially, the cloud and that they release the data by uploading it.