News for hackers. Stuff that matters.
Re:By removing “extremist content,” platforms are purging hu... by chongli on Wednesday October 23, 2019 @06:56PM (#21339936)
Not only that, if they are archived at one point and later decide to be excluded then all of the archives for that domain are deleted retroactively. This means that a valuable site can be archived for years but if the domain lapses and gets bought up by somebody who decides to exclude the Wayback Machine then all of that precious content is destroyed even though the destroyer never owned it!

I feel like we’re in the Internet equivalent of the early years of Hollywood. We’ve lost so many precious early movies because there were no backups of the film.

Re:By removing “extremist content,” platforms are purging hu... by schoen on Wednesday October 23, 2019 @06:57PM (#21339944)
I don't think your interpretation of the latter case is quite right. In my understanding, the Archive is willing to disable display of the site content retroactively due to a new robots.txt, but they won't delete that content. It will still be present in their files.
Re:By removing “extremist content,” platforms are purging hu... by chongli on Wednesday October 23, 2019 @07:01PM (#21339977)
What you’re describing may have been a recent change that I wasn’t aware of. If so, it’s at least some consolation. We could still do better.
Re:By removing “extremist content,” platforms are purging hu... by gbear605 on Wednesday October 23, 2019 @10:54PM (#21341257)
I believe that that has been the case for the entire existence of Wayback, not just recently.
Re:By removing “extremist content,” platforms are purging hu... by TJSomething on Wednesday October 23, 2019 @07:41PM (#21340199)
This policy was actually changed in 2017 to completely ignore robots.txt.

[0] https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...

Re:By removing “extremist content,” platforms are purging hu... by mc32 on Wednesday October 23, 2019 @06:58PM (#21339947)
Deleted or not made available?
Re:By removing “extremist content,” platforms are purging hu... by ryanlol on Thursday October 24, 2019 @04:35AM (#21342705)
If the site goes down and the robots.txt becomes unavailable, the content will become available on internet archive.