Re:U.K. Parties Prepare for 2015 by Erasing Web Histories
by vidarh on Tuesday November 19, 2013 @05:14AM
(#6760236)
They didn't just remove it. They explicitly added these pages to robots.txt, causing their removal from Internet Archive too.
[ Parent
| on HN ]
Re:U.K. Parties Prepare for 2015 by Erasing Web Histories
by cs02rm0 on Tuesday November 19, 2013 @06:29AM
(#6760467)
Seems like a sensible move to tell crawlers there isn't a page there so don't bother trying to index it. Probably keeps their logs cleaner and reduces the requests they have to serve.
I find it slightly odd that the internet archive then drop historical pages, but the jump some are making to this being Orwellian I find a leap too far.