News for hackers. Stuff that matters.
Re:The Internet Archive Telethon by frik on Sunday December 20, 2015 @01:25AM (#10766254)
If example.com allowed indexing in 1999, a new owner of example.com can hide/delete the 1999-2015 content by changing the robots.txt in 2015.

It would be better if archive.org would adhere to the robots.txt of the requested date/year (show content of example.com from 1999-2014).

Re:The Internet Archive Telethon by mapt on Sunday December 20, 2015 @02:21AM (#10766371)
The fact that all popular URLs which fall out of registration are now picked up by squatter-spambots is also troubling. An Archive.org entry should not cease to exist when the registration lapses if the squatter-spambots decide to robots.txt everything. That would defeat its purpose completely.