News for hackers. Stuff that matters.
Re:Glass discs that can store 360TB and remain intact for bi... by toyg on Tuesday June 25, 2019 @07:28AM (#20273095)
Seems a bit of a waste to dedicate an entire 360TB disc to a single text document like a bible, which is probably just a few KBs... /s

More seriously, they don't talk about reading capabilities (retrieval speed etc). And what if it gets scratched? What is the error tolerance? At that density, a single speck of dust could have dramatic implications...

I hope this reaches industrial viability, because we desperately need a digital format that can approximate the lifespan of simple paper. At the moment we are chained to a maintenance nightmare of periodic hops between formats, with deadly consequences any time we miss a single jump.

Re:Glass discs that can store 360TB and remain intact for bi... by Cthulhu_ on Tuesday June 25, 2019 @08:18AM (#20273439)
What would IMO be a better solution is something like the pyramids - something huge, with a lot of data written down / chiseled in without any weirdness like parity bits or binary code, and a lot of redundancy. Imagine the pyramids but made out of modern materials, resistant to erosion and earthquakes. Ceramics? Silicon?

Either way it'd have to be legible without the need of a microscope or computers. Write the same text down in a few dozen vastly different languages, so you've got a modern day Rosetta Stone.

And of course, build it somewhere remote and seal it. Bonus points for a stable atmosphere, fill it with a heavy noble gas or make it a vacuum.

(I'm not well versed in any related sciences, I'm just sounding off some random ideas)

Re:Glass discs that can store 360TB and remain intact for bi... by titanomachy on Tuesday June 25, 2019 @09:17AM (#20273845)
There is an ongoing project to carve knowledge on clay tablets and seal it in a salt mine in Austria. Sounds like it checks some of your boxes.

https://www.theatlantic.com/technology/archive/2017/01/human...

Re:Glass discs that can store 360TB and remain intact for bi... by carapace on Tuesday June 25, 2019 @11:27AM (#20275187)
Wow!

https://www.memory-of-mankind.com/how-is-information-kept-le...

> The geology of the mountain will allow the MOM archive to fully close itself by a natural phenomenon: the salt “flows” with a speed of 2 cm/year into any void. This will protect the archive from the greatest threat; man himself.

> The pressure which results from the weight of the mountain and a hypothetical ice shield of 5 km thickness is approximately a fifth of the burst pressure of the used materials.

Re:Glass discs that can store 360TB and remain intact for bi... by rtkwe on Tuesday June 25, 2019 @11:09AM (#20274987)
It depends on your goal, there's long term storage that assumes we'll retain the technology and knowledge required to read it (or it will be easily reproduced, ie the reader technology is trivial to produce in the future because technology has advanced really far) and then there's the civilization restart archive that assumes we'll be anywhere from cave people up to basically where we are today. Really for the second type I think both the macro scale and a micro scale (though not digitally encoded for most of it) in combination along with something like the Rosetta Disk [0] from the Long Now project is probably the way to ensure it's actually readable. The large scale text would provide a boot strap to get optics set up that could read the smaller etched text on archives that could then setup the tech to read the even finer etched stuff and so on maybe eventually reaching disks like the one in the article where we're able to store digital files and the knowledge to read them was introduced in more human readable formats.

[0] http://rosettaproject.org/disk/concept/

Re:Glass discs that can store 360TB and remain intact for bi... by orbital-decay on Tuesday June 25, 2019 @07:35AM (#20273154)
Yeah, actual museum-grade archival is so much more than just having an aging-resistant media. You also need to be able to read and decode it many years into the future, which is really tricky to guarantee - you can't assume the future generations will know how to make a reading device and remember your formats/encodings. So it's a bit like unzip.zip problem.
Re:Glass discs that can store 360TB and remain intact for bi... by oftenwrong on Tuesday June 25, 2019 @08:26AM (#20273486)
http://rosettaproject.org/

>For the extreme longevity version of the Rosetta database, we have selected a new high density analog storage device as an alternative to the quick obsolescence and fast material decay rate of typical digital storage systems. This technology, developed by Los Alamos Laboratories and Norsam Technologies, can be thought of as a kind of next generation microfiche. However, as an analog storage system, it is far superior. A 2.8 inch diameter nickel disk can be etched at densities of 200,000 page images per disk, and the result is immune to water damage, able to withstand high temperatures, and unaffected by electromagnetic radiation. This makes it an ideal backup for a long-term text image archive. Also, since the encoding is a physical image (no 1's or 0's), there is no platform or format dependency, guaranteeing readability despite changes in digital operating systems, applications, and compression algorithms.

>Reading the disk requires a microscope, either optical or electron, depending on the density of encoding and could be combined with an Optical Character Recognition system to read the text back into digital formats relevant at the time of reading. We are keeping our encoding at a scale readable by a 1000X optical microscope, giving us a total disk storage capacity of around 30,000 pages of text.

Re:Glass discs that can store 360TB and remain intact for bi... by jnurmine on Tuesday June 25, 2019 @11:13AM (#20275037)
Wouldn't one have to encase this in a diamond shell or something to avoid scraping it?
Re:Glass discs that can store 360TB and remain intact for bi... by kragen on Tuesday June 25, 2019 @02:16PM (#20277148)
By a delightful coincidence, there's another article on the front page of HN today that is relevant to the interpretability problem you're alluding to; see the discussion at https://news.ycombinator.com/item?id=20264848
Re:Glass discs that can store 360TB and remain intact for bi... by kd5bjo on Tuesday June 25, 2019 @07:52AM (#20273260)
This is where things like microfilm have an advantage- the only knowledge needed to decode it is magnification and the language it’s written in.

I’ve been wondering lately how well you could do with a microfilm strip full of 2d barcodes prefixed by an optical copy of the relevant specifications.

Re:Glass discs that can store 360TB and remain intact for bi... by unhammer on Tuesday June 25, 2019 @07:48AM (#20273234)
relevant: http://www.vpri.org/pdf/tr2015004_cuneiform.pdf talks about archiving computer programs so they may be read by future civilizations
Re:Glass discs that can store 360TB and remain intact for bi... by mapcars on Tuesday June 25, 2019 @08:52AM (#20273682)
>You also need to be able to read and decode it many years into the future, which is really tricky to guarantee

Not really, those future people don't just come out of nowhere. It's a gradual, incremental change as there are still systems using tape storage the technology or formats are not lost but migrated.

Re:Glass discs that can store 360TB and remain intact for bi... by orbital-decay on Tuesday June 25, 2019 @09:16AM (#20273841)
That depends on the time scales we're talking about. 30 years? Probably (and even then there are problems with abandoned/little-documented formats like Word 3.x). 300 years, and a historian will have a hard time decoding a digital file from the year 2019, even assuming the media is still intact. 3000 years and a nuclear war can make a weird shiny disc appear as a currency or a cult item for an archaeologist, not as a knowledge storage.
Re:Glass discs that can store 360TB and remain intact for bi... by nathan_long on Tuesday June 25, 2019 @10:34AM (#20274594)
This assumes an unbroken chain of people devoting time and resources to migrating the data. Any gap would mean that data is lost or requires much more expense to recover.

Find a 3.5" floppy disk from 1995 with WordPerfect files on it and see how much effort it takes you to get the documents back. Now imagine 100 years have passed.

Re:Glass discs that can store 360TB and remain intact for bi... by JudgeWapner on Tuesday June 25, 2019 @01:55PM (#20276918)
think nuclear war that wipes out 99.999% of earths human population. Even the few computers that remain would become useless because there's no power, fall apart due to degredation/weathering, and ignored because people will have much higher priorities like finding food and surviving, than preserving what would then be the previous generation's technology.
Re:Glass discs that can store 360TB and remain intact for bi... by megablast on Tuesday June 25, 2019 @07:49AM (#20273239)
Just put the instructions in a plain txt file as the first file on the disc.
Re:Glass discs that can store 360TB and remain intact for bi... by stefs on Tuesday June 25, 2019 @08:04AM (#20273346)
you'd have to know the (physical) reading mechanism (plus encoding) to read the README.txt itself.
Re:Glass discs that can store 360TB and remain intact for bi... by aardshark on Tuesday June 25, 2019 @08:35AM (#20273560)
Well encoding is not an issue. Any civilisation who have the physical means to read the disc will be able to figure out the encoding fairly simply.
Re:Glass discs that can store 360TB and remain intact for bi... by orbital-decay on Tuesday June 25, 2019 @09:39AM (#20274018)
It isn't that simple, many lost languages have only been restored because of the dual language books.

Similarly, I think what matters here is the higher-order redundancy. A single encoding that can be lost, is not straightforward to figure out, and holds the clue to a massive storage, isn't the most future proof decision; it's a single point of failure.

Re:Glass discs that can store 360TB and remain intact for bi... by TuringTest on Tuesday June 25, 2019 @08:31AM (#20273533)
That's why you put the ASCII encoding as well, plus the blueprints for a digital computer capable of running EMACS to decode it.
Re:Glass discs that can store 360TB and remain intact for bi... by gnode on Tuesday June 25, 2019 @10:04AM (#20274268)
> And what if it gets scratched? What is the error tolerance? At that density, a single speck of dust could have dramatic implications...

My understanding is that it doesn't matter. The data is embedded three dimensionally in glass. Assuming there was some small margin of glass around the patterned region, you could simply grind / polish off the scratches. Of course if you scratched it deep enough to destroy the data region then you'd be out of luck, but this can be made arbitrarily difficult.

Furthermore, you can arbitrarily scale error tolerance with error correcting encodings; data can be distributed, such that damage in one or a few areas is still available elsewhere, similar to RAID 5 and 6.

Re:Glass discs that can store 360TB and remain intact for bi... by notacoward on Tuesday June 25, 2019 @09:43AM (#20274051)
Just to follow up, here's a list of the problems I know about in getting 3D (or 5D) storage to market.

* Materials. You need something that can be manipulated or deformed at reasonable energy levels into two or more stable states to store data, then maintain that state for a useful lifetime without power. That's already a bit of a Goldilocks act, but it must also be non-reactive, not too brittle, not too expensive, etc.

* Media manufacture. Recording even a single layer at these feature sizes is a non-trivial challenge. As layers increase, yield drops exponentially - even before you consider how the layers interact.

* Read/write mechanisms. Focusing on a single layer on a moving medium, with nothing in the way, is also a challenge. Put other data-bearing layers in the way and it becomes much more difficult. Also, the "blast radius" for a single focus/alignment error becomes much larger, so you're going to need some serious error correction over and above what already exists for 2D.

* Transfer speeds. These are already problematic even at 10TB. 360TB without a corresponding increase in transfer speeds would be a nightmare.

A prototype involving a novel set of materials and/or mechanisms working once under ideal conditions is great. Science advances. Kudos for that. But that's only solving about one fifth of the problems that need to be solved to produce something of actual value in the market. You see a similar thing with battery technology. Everyone knows we need better batteries. There are always several companies claiming to have found the next breakthrough Almost invariably the new wonder material turns out to be a poor fit for real-world scale, economics, or conditions. Obviously we should keep trying, and keep learning, but nobody should get their hopes up too much too early.

Re:Glass discs that can store 360TB and remain intact for bi... by bshipp on Tuesday June 25, 2019 @07:31AM (#20273123)
I don't disagree, but it's not really different with spinning rust. the solution would likely be similar: seal it in an enclosure to prevent intrusion of dust, scratches, etc.
Re:Glass discs that can store 360TB and remain intact for bi... by bergoid on Tuesday June 25, 2019 @07:40AM (#20273185)
> spinning rust

TIL

https://etherealmind.com/network-dictionary-spinning-rust/

> Spinning Rust – term for conventional hard disk drives with motors and using ferrous-based platters for data storage. Probably derogatory now that Flash Drives are beginning to replace conventional drives.

Re:Glass discs that can store 360TB and remain intact for bi... by AnIdiotOnTheNet on Tuesday June 25, 2019 @08:12AM (#20273396)
TIL? Goddamnit, how old am I?
Re:Glass discs that can store 360TB and remain intact for bi... by andrey_utkin on Tuesday June 25, 2019 @07:46AM (#20273222)
But the rust which is supposed to be spun, doesn't live long when not being spun?

So far the most viable "solution" is to store data in the cloud service like Backblaze or AWS S3, where it's someone else headache to maintain high enough duplication and swap the drives as they reach their EOL.

Re:Glass discs that can store 360TB and remain intact for bi... by butteroverflow on Tuesday June 25, 2019 @07:35AM (#20273152)
I believe HDDs slowly lose stored data if you don't rewrite it periodically? I remember reading some data recovery forum a few years back with the local experts saying something about 10 years of reliable data storage for an HDD lying on a shelf. After that, all bets are off.
Re:Glass discs that can store 360TB and remain intact for bi... by anentropic on Tuesday June 25, 2019 @07:42AM (#20273197)
and/or add some degree of redundancy and error correction to the storage codec
Re:Glass discs that can store 360TB and remain intact for bi... by syn0byte on Tuesday June 25, 2019 @09:38AM (#20274006)
BD-Rom is already pushing sane limits. It's why an important if under reported "feature" of BD-Rom over DVD was harder, more scratch resistant plastic coating.

If you really need the lifespan of paper... http://ollydbg.de/Paperbak/

Re:Glass discs that can store 360TB and remain intact for bi... by toyg on Tuesday June 25, 2019 @10:16AM (#20274395)
Thanks for that link. It's clearly a joke and obsolete, but I wouldn't mind having something like that to backup a few small things (ssh keys, passwords meant to survive me, etc etc). After all, we will likely always have some sort of image-capturing device to leverage - unlike with discs of any sort.
Re:Glass discs that can store 360TB and remain intact for bi... by bufferoverflow on Tuesday June 25, 2019 @08:37AM (#20273571)
What if HDD platter gets scratched?

What if SSD memory gets scratched?

What if <any storage> gets <X mechanical damage>?

Re:Glass discs that can store 360TB and remain intact for bi... by toyg on Tuesday June 25, 2019 @09:15AM (#20273833)
The point of this tech is precisely to overcome the limitations of previous ones.

> What if <any storage> gets <X mechanical damage>?

It's more like "How much X damage can storage Y tolerate before losing Z amount of data?" You evaluate the different tech then you pick the best option.

Maybe some of this could be mitigated with logical techniques (e.g. make data redunant over 2 or 3 different parts of the disc, sacrificing capacity for durability), but it really depends on whether the process allows for this or completely borks out as soon as a scratch happens. They don't say anything about that and I think it's a pretty important aspect of any long-term storage solution.

Re:Glass discs that can store 360TB and remain intact for bi... by klez on Tuesday June 25, 2019 @08:59AM (#20273720)
Those are not touted as a very-long-term archival solution. This one is.