Re:By removing “extremist content,” platforms are purging hu...
by schoen on Wednesday October 23, 2019 @06:57PM
(#21339944)
I don't think your interpretation of the latter case is quite right. In my understanding, the Archive is willing to disable display of the site content retroactively due to a new robots.txt, but they won't delete that content. It will still be present in their files.
Re:By removing “extremist content,” platforms are purging hu...
by chongli on Wednesday October 23, 2019 @07:01PM
(#21339977)
What you’re describing may have been a recent change that I wasn’t aware of. If so, it’s at least some consolation. We could still do better.
Re:By removing “extremist content,” platforms are purging hu...
by gbear605 on Wednesday October 23, 2019 @10:54PM
(#21341257)
I believe that that has been the case for the entire existence of Wayback, not just recently.
Re:By removing “extremist content,” platforms are purging hu...
by TJSomething on Wednesday October 23, 2019 @07:41PM
(#21340199)
This policy was actually changed in 2017 to completely ignore robots.txt.
[0] https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...
I feel like we’re in the Internet equivalent of the early years of Hollywood. We’ve lost so many precious early movies because there were no backups of the film.