News for hackers. Stuff that matters.

A comment on The Internet Archive is under a DDoS attack. Parent comment

Re:The Internet Archive is under a DDoS attack by jasonfarnon on Monday May 27, 2024 @08:12PM (#40496066)
My problem with this approach is that the stuff I want to look at in 10 yrs time is never the stuff I think of saving right now. In the 2000s there were browser extensions I've forgotten the names of (shelf? slogger?) that would automatically save local copies of every webpage on page load. But I don't think they're around anymore and have no idea how you could achieve similar functionality with dynamic pages anyway.
[ Parent | on HN ]
Re:The Internet Archive is under a DDoS attack by rightbyte on Monday May 27, 2024 @08:20PM (#40496113)
> But I don't think they're around anymore and have no idea how you could achieve similar functionality with dynamic pages anyway.

It is probably easiest to save the render as a picture and then store text separately for searchability?

[ Parent | on HN ]
Re:The Internet Archive is under a DDoS attack by disqard on Monday May 27, 2024 @08:49PM (#40496290)
There's a way to get "the best of both worlds" (tbh, works most of the time): print to pdf.
[ Parent | on HN ]
Re:The Internet Archive is under a DDoS attack by Intralexical on Tuesday May 28, 2024 @05:01PM (#40505571)
> But I don't think they're around anymore and have no idea how you could achieve similar functionality with dynamic pages anyway.

Chromium's MHTML "Save as…" and the SingleFile WebExtension should both save copies of the rendered DOM.

Apparently Safari has WebArchive and Mozilla had MAFF for similar use cases.

I think WARC is supposed to save enough data about network streams for dynamic pages to work. At least on the Wayback Machine, infinite scrolling and "Load More" buttons do kinda work sometimes. You may have to load the archived pages in a browser and try to use each dynamic feature at least once, to trigger requests for needed resources.

SingleFile: https://github.com/gildas-lormeau/SingleFile

LWN on WARC, tools: https://anarc.at/blog/2018-10-04-archiving-web-sites/

Self-hostable web archives: https://awesome-selfhosted.net/tags/archiving-and-digital-pr...

Wayback Machine addons, bookmarklets: https://help.archive.org/help/save-pages-in-the-wayback-mach...

[ Parent | on HN ]