Wayback: Self-hosted archiving service integrated with Internet Archive

Re:Wayback: Self-hosted archiving service integrated with In... by JamesAdir on Sunday April 16, 2023 @05:04AM (#35588393)

Any recommendation for a tool that can crawl and download an entire website complely and save it locally?

[ Parent | on HN ]

Re:Wayback: Self-hosted archiving service integrated with In... by thunderbong on Sunday April 16, 2023 @05:20AM (#35588462)

HTTrack Website Copier

[ Parent | on HN ]

Re:Wayback: Self-hosted archiving service integrated with In... by JamesAdir on Sunday April 16, 2023 @01:04PM (#35591841)

thanks, tried it, but it has a problem with Unicode URL's unless there is some setting I'm missing.

[ Parent | on HN ]

Re:Wayback: Self-hosted archiving service integrated with In... by gala8y on Sunday April 16, 2023 @02:01PM (#35592559)

try adding +*{unicode} to scan rules. seriously, please try and let me know, if it worked.

[ Parent | on HN ]

Re:Wayback: Self-hosted archiving service integrated with In... by JamesAdir on Tuesday April 18, 2023 @12:37AM (#35610009)

I'm using the Windows version and still try to figure out the options. Will update if I'll find something.

[ Parent | on HN ]

Re:Wayback: Self-hosted archiving service integrated with In... by arboles on Sunday April 16, 2023 @11:47AM (#35591046)

Browsertrix or Brozzler, which crawl using headless browsers for accuracy.

[ Parent | on HN ]