Re:This Page is Designed to Last
by fouc on Thursday December 19, 2019 @06:42PM
(#21840439)
How is viewing the WARC after? Is it the same quality as archive.is/ or archive.org/ ?
[ Parent
| on HN ]
Re:This Page is Designed to Last
by cookiecaper on Friday December 20, 2019 @03:32AM
(#21842535)
It's a pain, like most of the WARC ecosystem. It's been several months since I dug in, so maybe it's had some spitshine the last little bit, but I usually end up using combinations of wpull, grab-site, and a smattering of other utilities to reliably capture a page/set of pages, and have had to make some quick hacks as well as manually merging in some PRs to get things to work with Python3. Once I have the WARC, I typically end up using warcat to extract the contents into a local directory and explore that way.
WARC as a format seems promising, but at least last I checked, open-source tooling to make it a pleasant and/or transparent experience is not really there, and worse, at least as of several months ago, doesn't really seem actively worked on. Definitely an area you'd expect to be further along.
Pretty good, depending on the tooling. Iām having good luck with https://webrecorder.io/ and their related open-source tools.