Waybackpack: download the entire Wayback Machine archive for a given URL

Re:Waybackpack: download the entire Wayback Machine archive ... by greglindahl on Tuesday May 03, 2016 @03:55PM (#11623688)

If I read the code correctly, it's one-at-a-time? Which minimizes the stress; if we're slow, it'll slow down.

It'd be nice if it had identification in the UserAgent, so that we could complain to the right people if it was a problem.

[ Parent | on HN ]

Re:Waybackpack: download the entire Wayback Machine archive ... by jsvine on Tuesday May 03, 2016 @04:02PM (#11623742)

Hi, Greg! Library author here. I'd be happy to add a configurable UserAgent. Perhaps the default would be a generic "waybackpack" but could be configurable to add contact info for the user. Does that sound about right? Prefer a different approach?

And, yep, the library is intentionally designed only to request one snapshot at a time.

[ Parent | on HN ]

Re:Waybackpack: download the entire Wayback Machine archive ... by greglindahl on Tuesday May 03, 2016 @04:06PM (#11623779)

waybackpack would be a great default; encouraging the actual user to add contact info would be better for you because we could complain to them instead of you :-)

[ Parent | on HN ]

Re:Waybackpack: download the entire Wayback Machine archive ... by jsvine on Tuesday May 03, 2016 @05:28PM (#11624384)

Updated, merged, and pushed to PyPi as part of v0.1.0: https://github.com/jsvine/waybackpack/pull/5

Thanks again for the feedback. Really appreciate it — and the existence of the Internet Archive and Wayback Machine.

[ Parent | on HN ]