Re:Internet Archive as a default host-of-record for startups
by ivank on Tuesday December 21, 2021 @11:59PM
(#29645754)
https://wiki.archiveteam.org/index.php/ArchiveBot is available if you would like to help save things that are still up.
Re:Internet Archive as a default host-of-record for startups
by smarx007 on Wednesday December 22, 2021 @05:46AM
(#29647655)
I never got turned down sending an IRC message "x is going down in a few days" to the AT channels. https://wiki.archiveteam.org/index.php/Archiveteam:IRC
AT just asks to be given a few weeks/months of notice depending on how many TBs (MBs?) the crawl needs to be and not to throttle/ban their clients.
Re:Internet Archive as a default host-of-record for startups
by chaps on Tuesday December 21, 2021 @11:50PM
(#29645708)
Pretty much. Your idea of getting coverage could probably work well to make sure IA doesn't "turn into a bot" (though it's practically the same, just without the permanent downloading).
I realize it's a hard problem, but I really wish there were a way to automate more of it. Some of these communities are too small for anyone to bother preserving the pages manually, and I don't imagine we'd even show up on ArchiveTeam's radar. But they're not large pages, and basically static. I don't think they'd be a huge burden to store and maintain. It seems like some sort of a coverage + size metric would be pretty effective at guiding an automated scan such that you'd be able to preserve things liked this without needing humans to go and manually archive each and every page.