Ask HN: What’s the best way to archive web sites?

Posted by Apocryphon on Tuesday September 13, 2022 @05:53PM
from the ask-hn dept.

What’s the preferred format for archiving static web pages composed of mostly text and graphics? I had been exporting some as PDFs on Safari, but now I’m wondering if for completeness I should save them in a more complete archive format via Chrome or a different browser.

( 8 comments | 12 points | on HN )

Re:Ask HN: What’s the best way to archive web sites? by rcarmo on Tuesday September 13, 2022 @06:00PM (#32830724)

I personally use an instance of https://archivebox.io/ on my home NAS. It does all the saving and conversion for you (runs Chrome internally and saves to multiple formats) and it’s just a “docker run” away.

Only thing that I wish it did much better is search. Right now it doesn’t do a good job of indexing full text content, but I’m hopeful that will change.

[ Parent | on HN ]

Re:Ask HN: What’s the best way to archive web sites? by trilinearnz on Tuesday September 13, 2022 @06:13PM (#32830845)

This might not be what you want, but you can direct the Wayback Machine on the Internet Archive to explicitly remember specific URLs. More info here: https://help.archive.org/help/save-pages-in-the-wayback-mach...

[ Parent | on HN ]

Re:Ask HN: What’s the best way to archive web sites? by Apocryphon on Tuesday September 13, 2022 @07:14PM (#32831348)

I've used IA a lot, but it's not very good at preserving image assets.

[ Parent | on HN ]

Re:Ask HN: What’s the best way to archive web sites? by prachisingh123 on Wednesday September 14, 2022 @03:36AM (#32834314)

A website can be archived using a variety of methods. A single webpage may be easily saved to your hard drive, you can rely on a CMS backup, or you can utilize free internet archive tools like HTTrack and the Wayback Machine. However, using an automatic archiving system that records every update is the ideal approach to record a site.

[ Parent | on HN ]

Re:Ask HN: What’s the best way to archive web sites? by phendrenad2 on Tuesday September 13, 2022 @11:12PM (#32832908)

HTTrack was the gold standard for backing up sites awhile ago, and I'm guessing that it's still a very good option.

[ Parent | on HN ]

Re:Ask HN: What’s the best way to archive web sites? by sp332 on Tuesday September 13, 2022 @07:21PM (#32831391)

You can direct wget to spider your site and save all the requests and responses to a single Web Archive format file called a WARC. https://wiki.archiveteam.org/index.php/Wget

[ Parent | on HN ]

Re:Ask HN: What’s the best way to archive web sites? by mattpallissard on Tuesday September 13, 2022 @07:30PM (#32831459)

With wget you can also download the files into a directory.[1]. Then you can just point your web-browser at the file path.

1. https://www.linuxjournal.com/content/downloading-entire-web-...

[ Parent | on HN ]

Re:Ask HN: What’s the best way to archive web sites? by altilunium on Tuesday September 13, 2022 @07:32PM (#32831464)

Use SinglePage

https://chrome.google.com/webstore/detail/singlefile/mpiodij...

[ Parent | on HN ]