Re:Addressing Recent Claims of “Manipulated” Blog Posts in t...
by flycaliguy on Wednesday April 25, 2018 @12:26PM
(#16923135)
Yeah, I've come to appreciate that their frustrating robot.txt policy is actually a wise way for them to avoid chipping away at the complete archive. Obviously they have a mission that stretches far beyond the lifespan of todays internet users and the relevance of robot.txt files.
If that file were to be removed, presumably the archive would again be served up upon request.
The lawyers were asking for the archive store itself to be wiped.