News for hackers. Stuff that matters.
Re:Sony and Panasonic announce the Archival Disc format by jrochkind1 on Monday March 10, 2014 @08:36AM (#7372546)
It seems this is mainly just a larger capacity (1TB) optical disc, and the 'archival' is just marketing?

For that matter, if they were really marketing at those who are professionally concerned with long-term reliable storage (archivists), even the marketing would include some information on what makes this new media any more reliable over the long-term than existing optical media. The press release includes nothing on this, odd for something branded as 'archival'.

It looks like it's just a larger capacity optical disc (which I'm sure there's a market for), with the 'archival' part just being marketting (odd; apparently they think there's a market for this too, even when it's just spin).

Re:Sony and Panasonic announce the Archival Disc format by keeperofdakeys on Monday March 10, 2014 @09:23AM (#7372729)
CD's don't last forever (http://en.wikipedia.org/wiki/CD-R#Lifespan), so you can't use them for archival purposes. For now, there is no way to tell whether these will be any better.
Re:Sony and Panasonic announce the Archival Disc format by jrochkind1 on Monday March 10, 2014 @10:43AM (#7373151)
Well, no media lasts forever, but we still have to store digital content somewhere for archival purposes. (ideally more than one copy, naturally, routinely checked for data integrity).

But yeah, the long-term reliability of media is often a key consideration in archival storage, which is why I think it's suspicious that they brand this media as 'archival' without providing any information at all about what they think it's long-term reliability is, compared to other (optical or other) media.

Re:Sony and Panasonic announce the Archival Disc format by hga on Monday March 10, 2014 @09:43AM (#7372825)
I believe you can use them for archival purposes if you buy quality discs. Which for me means Made in Japan Taiyo Yuden, although perhaps Mitsui/MAM-A are good. See https://news.ycombinator.com/edit?id=7372810 for my most recent experiences with these discs.
Re:Sony and Panasonic announce the Archival Disc format by acdha on Monday March 10, 2014 @10:44AM (#7373152)
This is really, really, dangerous advice unless you're following proper archival practice and have made multiple copies which are regularly checked for bit errors. For all media, but particularly for CD-Rs, you have to be very careful about environmental conditions – a few degrees temperature difference makes a difference for how fast the dye degrades and unless you actually monitor this, it's easy to have that problem without noticing it until things start failing.

For a small user, you have the extra concern of betting large on a single production run from a single vendor. If you care about archival, you'd really want to burn multiple copies using discs from different production runs and, preferably, different manufacturers.

(This is, of course, why I'd really recommend using CD-Rs only in conjunction with a different technology such as AWS Glacier which has a completely different set of failure modes)

Re:Sony and Panasonic announce the Archival Disc format by hga on Monday March 10, 2014 @11:05AM (#7373293)
There's actually a lot less commonality in a single production run than you'd think, or so I believe having analyzed Taiyo Yuden disc hub numbers. Different physical machines, I suspect. But of course the "chemistry" will be roughly the same, and thus a whole batch or more could go bad early. Although one might hope they do accelerated aging tests on their production.

You are of course correct about the precautions that have to be used, but there's screw cases for everything you can use to back up your data. I was merely responding to the assertion that CD-Rs are entirely unsuited for archival purposes, and backed it up with my own experiences over a decade and a half.

Me, I live a hair's breath from Tornado Alley, and use multiple means to backup my data, local discs, disks in another room (which got trashed enough to be unreliable in a tornado http://www.ancell-ent.com/1715_Rex_Ave_127B_Joplin/images/) which I've switched to LTO-4 tape, some of which live in a safe deposit box, and the really important stuff offsite to rsync.net (which saved the only important data totally lost locally from the tornado, or at least without $$$ to a data recovery firm). And of course the CD-Rs, which at some point I'll start putting into the disk and LTO bases system.

Re:Sony and Panasonic announce the Archival Disc format by acdha on Monday March 10, 2014 @11:12AM (#7373341)
> Although one might hope they do accelerated aging tests on their production

The experienced storage admins I know share that hope but don't trust vendors not to get it wrong. It's just too easy to miss a factor which turns out to matter.

> I was merely responding to the assertion that CD-Rs are entirely unsuited for archival purposes, and backed it up with my own experiences over a decade and a half.

Question: have you done bit-level checksum validation on that old media or is that just the ability to read without errors? There's a little bit of error correction built into the format but I wouldn't trust it for anything important.

Re:Sony and Panasonic announce the Archival Disc format by hga on Monday March 10, 2014 @11:37AM (#7373513)
Bit level CRC-32 checksum validation, absolutely. (Which is actually probably marginal for the file sizes concerned, but that's the standard everyone uses.)

However my words are coming across in this discussion, I'm really not a very trusting guy! Many of my early computer experiences were in the early '80s with surplus hardware from the '70s, including a PDP-11/45 that was a bit beyond the 300th DEC manufactured. Of course everyone used magtape back then for backups, and those were quite reliable (originally intended to be reliable off-line storage).

ADDED: and there are other precautions to take. E.g. I only bought my optical media in April and October, to minimize the environmental stress during shipping.

Re:Sony and Panasonic announce the Archival Disc format by acdha on Monday March 10, 2014 @02:19PM (#7374688)
> Bit level CRC-32 checksum validation, absolutely. (Which is actually probably marginal for the file sizes concerned, but that's the standard everyone uses.)

In the digital preservation community, the standard is at least MD-5 / SHA-1 and most people are moving to SHA-256/512. With a CRC-32 check you're likely to get false-negatives for modern data volumes and there are disturbing reports of CRC failures at higher than expected rates[1] which suggest that the best answer is using multiple, cryptographic hashes particularly since the computation has effectively been free for a least a decade except on unusually CPU-starved storage hosts.

1. I don't recall the paper but I believe it was a followup to http://conferences.sigcomm.org/sigcomm/2000/conf/paper/sigco...

Re:Sony and Panasonic announce the Archival Disc format by rbanffy on Monday March 10, 2014 @09:24AM (#7372733)
"Archival" may also mean "non-consumer" as in "don't ever touch the media with your hands" or "don't even think of having a single copy of anything". The physical tolerances of this are so beyond already fragile Blu Ray media I can only imagine these disks would come encased in airtight cartridges to be manipulated by exquisitely precise robots and read and written within very delicate drives.
Re:Sony and Panasonic announce the Archival Disc format by jrochkind1 on Monday March 10, 2014 @10:42AM (#7373141)
You think the media is going to be LESS reliable for long-term preservation than existing optical media?

Wow, if so, it sure seems sketchy to brand it as 'archival'.

Re:Sony and Panasonic announce the Archival Disc format by rbanffy on Monday March 10, 2014 @02:30PM (#7374765)
I imagine these disks would not survive the kind of mishandling DVDs and BDs survive every day. If it's archive, it should be treated so. Also, having multiple copies makes recovery comparatively faster.

In this context, long term durability has more to do with chemical attack from the atmosphere than being handled by human hands.

Having media that lasts for 100 years is little comfort either. What good would do floppies that last for 100 years when the only computer capable of using them here in my house is an Amiga 600?

Re:Sony and Panasonic announce the Archival Disc format by jrochkind1 on Monday March 10, 2014 @03:00PM (#7374967)
Huh, so to you, you assume that something labelled "archival" means "is especially fragile, must be treated with care", that's what "archival" and "treated so" means to you?

Odd. To me, something labelled "archival" means "is expected to last a really long time, and be especially robust."

I _think_ that's what it means to most people. Although it's hard to say. But compare to wikipedia says "Archival paper is an especially permanent, durable acid-free paper." Especially durable and permanent, not especially fragile and needs to be treated with unusual care!

If they are marketting something as "archival", I am reasonably certain that _many_ people are going to think that means "especially durable and permanent", not "especially fragile, must be treated with care."

But yes, there are many more factors to long-term preservation of digital content than simply media reliability, including inter-operability with future hardward/software. Integrity of the bits on your media is certainly one factor though. And yes, multiple copies and regular integrity checks is crucial, not just counting on your media to last. Media long-term reliability still matters though.

(I work in libraries, and I work with and next to people concerned with digital preservation, although I don't work on long-term digital preservation myself).

Re:Sony and Panasonic announce the Archival Disc format by rbanffy on Monday March 10, 2014 @06:56PM (#7376237)
> Huh, so to you, you assume that something labelled "archival" means "is especially fragile, must be treated with care", that's what "archival" and "treated so" means to you?

No. I just suppose that packing that much information so tightly, encoded in such small features on a plastic disc, will make the recording more fragile. It's like printing all your text with a 3-point font. You can print a lot of text on a single page, but any damage to the page may make large parts unreadable.

As for long term preservation, I'd never suggest keeping your data on the same medium for any significantly long time. And never, ever, under no circumstances, find yourself with a single copy of anything.