Re:Show HN: WarcDB: Web crawl data as SQLite databases
by mynameismon on Sunday June 19, 2022 @01:27PM
(#31801405)
> What is the advantage of moving around .sqlite-files, over just loading the (compressed) WARCs into sqlite databases when you need them?
I suppose there could be made a case for easy extension: You don't need to change the entire spec to add another table in the SQLite database, maybe containing other metadata.
> WARC's real selling pint is that it's such an extremely portable format.
I mean, so is SQLite: it is also apporoved as a LoC archival method. (See SQL Archive [0])
I suppose there could be made a case for easy extension: You don't need to change the entire spec to add another table in the SQLite database, maybe containing other metadata.
> WARC's real selling pint is that it's such an extremely portable format.
I mean, so is SQLite: it is also apporoved as a LoC archival method. (See SQL Archive [0])
[0]: https://www.sqlite.org/sqlar.html