News for hackers. Stuff that matters.

A comment on ArchiveBox: Open-source self-hosted web archive. Parent comment

Re:ArchiveBox: Open-source self-hosted web archive by mysterydip on Sunday March 10, 2019 @05:35AM (#19351175)
One disadvantage is it introduces an application dependency. If the purpose of an archive is to preserve the data for future recall, years or decades later, having it in the most accessible format possible would be a priority.
[ Parent | on HN ]
Re:ArchiveBox: Open-source self-hosted web archive by aloer on Sunday March 10, 2019 @11:42AM (#19352410)
Remembered I read something about this some time ago, looked it up:

SQLite is a recommended storage format (https://www.sqlite.org/locrsf.html) by the library of congress

> As of this writing (2018-05-29) the only other recommended storage formats for datasets are XML, JSON, and CSV

[ Parent | on HN ]
Re:ArchiveBox: Open-source self-hosted web archive by greglindahl on Sunday March 10, 2019 @12:48PM (#19352781)
The Library of Congress stores its web archives as WARCs.
[ Parent | on HN ]
Re:ArchiveBox: Open-source self-hosted web archive by mysterydip on Monday March 11, 2019 @01:51AM (#19356962)
Good to know, thanks!
[ Parent | on HN ]
Re:ArchiveBox: Open-source self-hosted web archive by rakoo on Sunday March 10, 2019 @01:42PM (#19353127)
Very true, but I'd say SQlite is the one application dependency that could have a pass:

- they promise support until at least 2050 (https://sqlite.org/lts.html)

- if a promise isn't enough, SQlite is to be supported for the entire lifetime of the Airbus A350 airframe (https://mobile.twitter.com/copiousfreetime/status/6758345433...). I would assume Airbus will be paying for it.

A real problem with application dependency is that it makes it extremely hard to create a completely independent alternative. A good protocol should have multiple implementations with nothing in common otherwise you rely on specific implementation details. The infamous case of WebSQL should remind us of that.

[ Parent | on HN ]
Re:ArchiveBox: Open-source self-hosted web archive by mysterydip on Monday March 11, 2019 @01:52AM (#19356967)
Also excellent points I waan't aware of, thanks!
[ Parent | on HN ]