News for hackers. Stuff that matters.
Re:Archivists are racing to digitise 300 years of newspapers... by jedberg on Sunday July 05, 2015 @03:03PM (#9834530)
It's interesting that people are currently lamenting the fact that a huge chunk of human history will be lost because it's all digital now, while at the same time we are racing to digitize the past to prevent it from getting lost.

It almost seems inevitable that most of the details of human history will be lost regardless....

Re:Archivists are racing to digitise 300 years of newspapers... by dglass on Sunday July 05, 2015 @10:12PM (#9836278)
I don't mean to self-promote, but I've thought a lot about this too and created http://tracket.com as a side project. I haven't actually talked it publicly until right now.

It's sort of a mix between archive.org and google news. It aggregates and archives articles on the web for current events each day. I've been archiving the top stories each day since December 2014.

Part of my interest in this is to preserve human history in the digital age. We already have the means of storing articles, photos, videos and more of current events as they happen. It's a work in progress, but it's something I spend a couple hours each week hoping to improve it over time.

Edit: Take a look at the top stories in January 2015 at http://tracket.com/2015/01

Re:Archivists are racing to digitise 300 years of newspapers... by waterlesscloud on Sunday July 05, 2015 @03:29PM (#9834629)
The key advantage with digital is that it's easily copied.

For that advantage to be worth anything, the material has to actually be...copied.

Central repositories for this stuff are fine, maybe even essential. But they need to also be spreading around full copies quite liberally.

Re:Archivists are racing to digitise 300 years of newspapers... by jedberg on Sunday July 05, 2015 @03:35PM (#9834651)
> For that advantage to be worth anything, the material has to actually be...copied.

This is they key. I think people are afraid of our all digital world because no one backs anything up.

Re:Archivists are racing to digitise 300 years of newspapers... by agumonkey on Sunday July 05, 2015 @04:57PM (#9834928)
Also, digital is so efficient it accumulates far faster than older mediums. I find it a very telling pattern. Things are easy and so they get out of control.

When I was a kid, a photo album had something like 50 tiny fading photos, and it covered 5 years or so. Each one was a very high density of memories. Now we have thousands of them. Curious.

Re:Archivists are racing to digitise 300 years of newspapers... by jqm on Sunday July 05, 2015 @07:11PM (#9835513)
In the old days someone put a photo album in a trunk in the attic. Then they got old, forgetful, the kids didn't care about the photos, then the person died. Years later someone else found the photo album.

What happens in this scenario with digital photos? Sure, if someone is careful (and most people frankly aren't... I've seen more than a few people lose all their photos) you can keep backing them up. Until you can't anymore or you don't. Then they are gone.

Re:Archivists are racing to digitise 300 years of newspapers... by joshuapants on Sunday July 05, 2015 @04:18PM (#9834763)
Except for the things you don't want anyone to ever see; those will propagate mercilessly.
Re:Archivists are racing to digitise 300 years of newspapers... by bottled_poe on Sunday July 05, 2015 @09:33PM (#9836106)
>The key advantage with digital is that it's easily copied.

That and the fact that we can store an entire library of data in a searchable space the size of an ice cube.

Re:Archivists are racing to digitise 300 years of newspapers... by userbinator on Sunday July 05, 2015 @07:01PM (#9835473)
The fact that digital media is so easily copied is what immensely scares the copyright/DRM advocates, so let's hope they don't get in the way of preserving history.
Re:Archivists are racing to digitise 300 years of newspapers... by cm2187 on Sunday July 05, 2015 @05:39PM (#9835132)
Particularly text. Once OCR-ed, the content of the entire building fits on a consumer HD.
Re:Archivists are racing to digitise 300 years of newspapers... by caseysoftware on Sunday July 05, 2015 @08:49PM (#9835945)
I spent a few years at the Library of Congress working on a similar project.

While it's "easier" to lose digital, it's a possibility as opposed to the inevitability of losing physical forms like paper, LPs, or wire spool recordings. Physical degrades, there's no (economically feasible) way around it.

In the case of digital forms of the same media, they're stored in high resolution, lossless formats for archival purposes and then down sampled for day to day use. For example, our image scans were 300dpi tifs while the day to day versions were simple pngs. Audio files had equivalent things.

The vision is that the lossless versions can be updated to the latest and greatest formats over time and the smaller versions can be recreated as needed.

Re:Archivists are racing to digitise 300 years of newspapers... by javajosh on Sunday July 05, 2015 @03:45PM (#9834708)
It takes time to make paper, it takes time to make ink. It takes time to make nand gates and flash storage. It takes time to write things down, it takes time to impose a patter of electrostatic charge onto a substrate. Call this t_make.

It takes time for paper and ink to decay. It takes time for charges to leak. It takes time for PGA packages to deteriorate. Call this t_decay.

In general, t_make << t_decay even for paper, and with electronic information, it's more profound. A really good number is this ratio, which in this case, t_make/t_decay is roughly 1/300 (assumes this project will take a year, which is probably high). For electronic backup, I'd imagine it's more like 1/10^6 or so.

That said, some human effort is needed to keep information alive over long period of time. But it's only t_make effort every t_decay period, which actually isn't a lot, on it's face. It's an interesting question though what the maximum amount of data humanity can keep alive given a certain level of economic output. E.g. if you make 10^7 hard drives a year that store 10^12 bits and they each last 10 years, then you can store 10^20 bits indefinitely. (10^7 hard drives x 10 years x 10^12 bits).

Re:Archivists are racing to digitise 300 years of newspapers... by sliverstorm on Sunday July 05, 2015 @11:00PM (#9836474)
1) purposeful digital archival and day-to-day throwaway content generation are very different stories in terms of staying power

2) the loss of print media is happening right now, today, while the loss of digital media is not quite as pressing. "Kicking the can" in a sense.

Re:Archivists are racing to digitise 300 years of newspapers... by netcan on Sunday July 05, 2015 @06:23PM (#9835300)
Here's my take, perhaps a little far fetched.

At some point around 5000 years ago writing emerged. People wrote down things like “such and such is the king of all the world. He is the son of such and such.” If these things survived and we have them today, we call that history. There’s a convention of dating the beginning of history to the first writing and calling everything before that prehistory. The specifics are cloudy and debatable. Does it count if we can’t read it? Does it count if it’s not classifiable as “writing.” If people paint a cave and we understand they were talking about hunting aurochses with spears, then we know there were some people here that hunted aurochses with spear. Does that count as writing?

Vague semantic lines aside, there’s an idea here that’s interesting. When people can write about something as it happens and put that information down, that’s history. The historical age. There’s something to that, an important difference between information communicated to us by people of that time rather than us knowing by studying less direct artefacts.

All this writing, all this history, has been accelerating like most technology. Every century (except for a few bumpy patches) we’ve been mooing more of it, making more history. More places, people perspectives, details. We’ve added pictures (if cave painting don’t count, are paparazzi snaps of Kardashian’s arse out too?) and video, 3d images, seismographic data and a whole lot of other stuff to that pile. History is exploding. Terry’s History Monks must be having fits.

Anyway, say we move forward a bit and say the current trends keep going for that bit. CCTVs are now omnidirectional. The cops finally agreed to wear body cams, but insist that the crooks wear them too and the politicians while we’re at it. Computers can rewind every bit that ever flipped. Since everyone has an ear-chip and a groin-chip installed (google glass was the last attempt at okular recording devices, too invasive), our lives are recorded fully. Psychology finally works because instead of asking about your mother, a guy with a 19th century bohemian accent just becomes you as a suckling babe for a bit and figures out what went wrong. Maybe he reads her old tweets.

I think that’s a new age. I know this gets us into naming convention problems when the next age comes, but I’d like to call it posthistory. The period before people recorded their commentary on events, the period when they did and the period when everything was recorded in full.

I actually do think that’s coming and I do think it will be a profound change. I think we’re in a sort of grey transitional period where not everything is recorded, but sufficiently more than in previous times that we’re back to troubling questions “do symbolic representation systems that only have symbols for tax related concepts like cheese, beer and god-kings count as real writing and therefore history,” but with a more posthistoric flavour and copyright issues. If Maldu of unya

Oh, and I think bit rot will get solved, unless zombies or meteors or something. Shuruppak sent a note to Labek telling him not to screw around with married women 5000 years ago and we can still read it. I think we can preserve “LOL UR mom is crazy!!!!!”

Re:Archivists are racing to digitise 300 years of newspapers... by userbinator on Sunday July 05, 2015 @07:15PM (#9835534)
Maybe he reads her old tweets.

http://www.cnn.com/2013/01/07/tech/social-media/library-cong...

Tweets are actually quite small and very compressible, so archiving them all wouldn't take much space; despite Twitter averaging several thousand tweets/s, from the perspective of a machine that's less than 1MB/s and far less than that compressed.

Re:Archivists are racing to digitise 300 years of newspapers... by dreamcompiler on Monday July 06, 2015 @01:22PM (#9840665)
There's an episode of "Black Mirror" that speaks to your "posthistory" idea and it's not pretty. https://en.m.wikipedia.org/wiki/The_Entire_History_of_You
Re:Archivists are racing to digitise 300 years of newspapers... by digi_owl on Sunday July 05, 2015 @03:24PM (#9834603)
Well i think that is inherent in entropy.

If i understand the physics right, any kind of order/information will eventually decay into chaos. With the ultimate of that being the heat (gradient) death of the universe.

Re:Archivists are racing to digitise 300 years of newspapers... by andyjohnson0 on Sunday July 05, 2015 @03:33PM (#9834640)
The scond law of thermodynamics says that the total entropy of a system always increases. It doesn't prohibit local entropy decreases. So a part of a system (newspaper, archive, planet) can maintain or increase its order.
Re:Archivists are racing to digitise 300 years of newspapers... by nitrogen on Monday July 06, 2015 @12:20AM (#9836747)
Note to sibling commenter, contingencies: your comment is dead for whatever reason, so nobody can reply to it. Might want to check with the HN management to see why.